{"id":37547,"date":"2022-04-28T12:11:07","date_gmt":"2022-04-28T19:11:07","guid":{"rendered":"https:\/\/app14743.cloudwayssites.com\/?page_id=37547"},"modified":"2022-05-09T08:41:15","modified_gmt":"2022-05-09T15:41:15","slug":"visual-testing","status":"publish","type":"page","link":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/","title":{"rendered":"Learn &#8211; Concepts &#8211; Visual Testing"},"content":{"rendered":"\n<div class=\"wp-block-group pt-sm container-10\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<h3 class=\"wp-block-heading\" id=\"block-d18f348b-b2b9-4e03-b8ff-08048050ba74\"><br>Foundations<\/h3>\n\n\n\n<ul id=\"block-10267d8b-90af-4718-9265-d347f98d2caa\" class=\"is-style-default\"><li><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/functional-testing\/\">Functional Testing<\/a><\/li><li><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/\">Visual Testing<\/a><\/li><li><a href=\"\/learn\/concepts\/performance-testing\">Performance Testing<\/a><\/li><li><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/accessibility-testing\/\">Accessibility Testing<\/a><\/li><li><strong><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/mobile-testing\/\">Mobile Testing<\/a><\/strong><\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"block-1845be72-f492-42d5-86cc-7c6d06e2eaf2\">Frameworks<\/h3>\n\n\n\n<ul id=\"block-0e8076c4-e72c-4c8f-92b4-240d56b60d9e\" class=\"is-style-default\"><li><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/frameworks\/cypress\/\">Cypress<\/a><\/li><li><a href=\"\/learn\/frameworks\/selenium\">Selenium<\/a><\/li><li><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/frameworks\/playwright\/\">Playwright<\/a><ul><li><a href=\"https:\/\/app14743.cloudwayssites.com\/learn\/frameworks\/playwright\/verifying-sortable-tables\/\">Verifying sortable tables<\/a><\/li><\/ul><\/li><li><a href=\"\/learn\/frameworks\/puppetteer\">Puppetteer<\/a><\/li><li><a href=\"\/learn\/frameworks\/appium\">Appium<\/a><\/li><\/ul>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<h2><strong>What is Meant By Visual Testing?<\/strong><\/h2>\n<p><a href=\"https:\/\/app14743.cloudwayssites.com\/visual-testing\/\">Visual testing<\/a> evaluates the visible output of an application and compares that output against the results expected by design. In other words, it helps catch &#8220;visual bugs&#8221; in the appearance of a page or screen, which are distinct from strictly functional bugs.<\/p>\n<p>You can run visual tests at any time on any application with a visual user interface. Most developers run visual tests on individual components during development, and on a functioning application during end-to-end tests.<\/p>\n<p>In today\u2019s world, in the world of HTML, web developers create pages that appear on a mix of browsers and operating systems. Because HTML and CSS are standards, frontend developers want to feel comfortable with a \u2018write once, run anywhere\u2019 approach to their software. Which also translates to \u201cLet QA sort out the implementation issues.\u201d QA is still stuck checking each possible output combination for visual bugs.<\/p>\n<p>This explains why, when I worked in product management, QA engineers would ask me all the time, \u201cWhich platforms are most important to test against?\u201d If you\u2019re like most QA team members, your test matrix has probably exploded: multiple browsers, multiple operating systems, multiple screen sizes, multiple fonts &#8212; and dynamic responsive content that renders differently on each combination.<\/p>\n<p>If you are with me so far, you\u2019re starting to answer the question: why do visual testing?<\/p>\n<h2>Why is Visual Testing Important?<\/h2>\n<p>We do visual testing because visual errors happen &#8212; more frequently than you might realize. Take a look at this visual bug on Instagram\u2019s app:<\/p>\n<p><blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\"><p lang=\"en\" dir=\"ltr\">This <a href=\"https:\/\/twitter.com\/LinkedIn?ref_src=twsrc%5Etfw\">@LinkedIn<\/a> post by <a href=\"https:\/\/twitter.com\/techgirl1908?ref_src=twsrc%5Etfw\">@techgirl1908<\/a> has struck a nerve&#8230;take a look! <a href=\"https:\/\/t.co\/uwt29BY2Zh\">https:\/\/t.co\/uwt29BY2Zh<\/a><a href=\"https:\/\/twitter.com\/hashtag\/GUIGoneWrong?src=hash&amp;ref_src=twsrc%5Etfw\">#GUIGoneWrong<\/a><\/p>&mdash; Applitools (@Applitools) <a href=\"https:\/\/twitter.com\/Applitools\/status\/1111337607680552960?ref_src=twsrc%5Etfw\">March 28, 2019<\/a><\/blockquote><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>The text and ad are crammed together. If this was your ad, do you think there would be a revenue impact? Absolutely.<\/p>\n<p>Visual bugs happen at other companies too: <a href=\"https:\/\/twitter.com\/EranBarlev\/status\/1020013582199676928\">Amazon<\/a>. <a href=\"https:\/\/twitter.com\/deefex\/status\/1242574176050970626\">Google<\/a>.&nbsp;<a href=\"https:\/\/twitter.com\/Applitools\/status\/1349462419404308481\">Slack<\/a>. <a href=\"https:\/\/twitter.com\/pri_tech_mom\/status\/1369900152362655746\">Robin Hood<\/a>. <a href=\"https:\/\/twitter.com\/Applitools\/status\/1181531941096570880\">Poshmark<\/a>. <a href=\"https:\/\/twitter.com\/Applitools\/status\/1180037598561128448\">Airbnb.<\/a> <a href=\"https:\/\/twitter.com\/Applitools\/status\/1123619551982211072\">Yelp<\/a>. <a href=\"https:\/\/twitter.com\/pri_tech_mom\/status\/1416865694411489281\">Target<\/a>. <a href=\"https:\/\/twitter.com\/Applitools\/status\/1015271543209771009\">Southwest<\/a>. <a href=\"https:\/\/twitter.com\/EranBarlev\/status\/1019984879151284224\">United<\/a>. <a href=\"https:\/\/twitter.com\/Applitools\/status\/1112738531376394240\">Virgin Atlantic<\/a>. <a href=\"https:\/\/twitter.com\/techgirl1908\/status\/1050074470554030080\">OpenTable<\/a>. These aren\u2019t cosmetic issues. In each case, visual bugs are blocking revenue.<\/p>\n<p>If you need to justify spending money on visual testing, share these examples with your boss.<\/p>\n<p>All these companies are able to hire some of the smartest engineers in the world. If it happens to Google, or Instagram, or Amazon, it probably can happen to you, too.<\/p>\n<p>Why do these visual bugs occur? Don\u2019t they do functional testing? They do &#8212; but it\u2019s not enough.<\/p>\n<p>Visual bugs are rendering issues. And rendering validation is not what functional testing tools are designed to catch. Functional testing measures functional behavior.<\/p>\n<h3>Why can&#8217;t functional test cover visual issues?<\/h3>\n<p>Sure, functional test scripts can validate the size, position, and color scheme of visual elements. But if you do this, your test scripts will soon balloon in size due to checkpoint bloat.<\/p>\n<p>To see what I mean, let\u2019s look at an Instagram ad screen that\u2019s properly rendered. There are 21 visual elements by my count &#8212; various icons, text. (This ignores iOS elements at the top like WiFi signal and time, since those aren\u2019t controlled by the Instagram app.)<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5078 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image9.png\" alt=\"\" width=\"301\" height=\"532\"><br>If you used traditional checkpoints in a functional testing tool like <a href=\"https:\/\/www.seleniumhq.org\/projects\/webdriver\/\">Selenium Webdriver<\/a>, <a href=\"https:\/\/www.cypress.io\/\">Cypress<\/a>, <a href=\"https:\/\/webdriver.io\/\">WebdriverIO<\/a>, or <a href=\"http:\/\/appium.io\/\">Appium<\/a>, you\u2019d have to check the following for each of those 21 visual elements:<\/p>\n<ol>\n<li>Visible (true\/false)<\/li>\n<li>Upper-left x,y coordinates<\/li>\n<li>Height<\/li>\n<li>Width<\/li>\n<li>Background color<\/li>\n<\/ol>\n<p>That means you\u2019d need the following number of assertions:<\/p>\n<p style=\"text-align: center;\"><strong>21 visual elements x 5 assertions per element = 105 lines of assertion code<\/strong><\/p>\n<p>Even with all this assertion code, you wouldn\u2019t be able to detect all visual bugs. Such as whether a visual element can\u2019t be accessed because it\u2019s being covered up, which blocked revenue in the above examples from <a href=\"https:\/\/twitter.com\/Applitools\/status\/1123619551982211072\">Yelp<\/a>, <a href=\"https:\/\/twitter.com\/Applitools\/status\/1015271543209771009\">Southwest<\/a>, <a href=\"https:\/\/twitter.com\/EranBarlev\/status\/1019984879151284224\">United<\/a>, and <a href=\"https:\/\/twitter.com\/Applitools\/status\/1112738531376394240\">Virgin Atlantic<\/a>. And, you\u2019d miss subtleties like the brand logo, or the red dot under the heart.<\/p>\n<p>But it gets worse: if OS, browser, screen orientation, screen size, or font size changes, your app\u2019s appearance will change as a result. That means you have to write another 105 lines of functional test assertions. For EACH combination of OS\/browser\/font size\/screen size\/screen orientation\/font size.<\/p>\n<p>You could end up with thousands of lines of assertion code &#8212; any of which might need to change with a new release. Trying to maintain that would be sheer madness. No one has time for that.<\/p>\n<p>You need visual testing because visual errors occur. And you need visual testing because you cannot rely on functional tests to catch visual errors.<\/p>\n<h2>What is Manual Visual Testing?<\/h2>\n<p>Because automated functional testing tools are poorly suited for finding visual bugs, companies find visual glitches using manual testers. Lots of them (more on that in a bit).<\/p>\n<p>For these manual testers, visual testing behaves a lot like this spot-the-difference game:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5083 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image14.png\" alt=\"\" width=\"864\" height=\"339\"><\/p>\n<p>To understand how time-consuming visual testing can be, get out your phone and time how long it takes for you to find all six visual differences. I took a minute to realize that the writing in the panels doesn\u2019t count. It took me about 3 minutes to spot all six. Or, you can cheat and look at the answers.<\/p>\n<p>Why does it take so long? Some differences are difficult to spot. In other cases, our eyes trick us into finding differences that don\u2019t exist.<\/p>\n<p>Manual visual testing means comparing two screenshots, one from your known good baseline image, and another from the latest version of your app. For each pair of images, you have to invest time to ensure you\u2019ve caught all issues. Especially if the page is long, or has a lot of visual elements. Think &#8220;Where\u2019s Waldo&#8221;\u2026<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5070 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image1.jpg\" alt=\"\" width=\"866\" height=\"616\"><\/p>\n<h3>Challenges of manual testing<\/h3>\n<p>If you\u2019re a manual tester or someone who manages them, you probably know how hard it is to visually test.<\/p>\n<p>If you are a test engineer reading this paragraph, you already know this: web page testing only starts with checking the visual elements and their function on a single operating system, browser, browser orientation, and browser dimension combination. Then continue on to other combinations. And, that\u2019s where a huge amount of test effort lies &#8211; not in the functional testing, but in the inspection of visual elements across the combination of an operating system, browser, screen orientation, and browser dimensions.<\/p>\n<p>To put it in perspective, imagine you need to test your app on:<\/p>\n<ul>\n<li>5 operating systems: Windows, MacOS, Android, iOS, and Chrome.<\/li>\n<li>5 popular browsers: Chrome, Firefox, Internet Explorer (Windows only) Microsoft Edge (Windows Only), and Safari (Mac only).<\/li>\n<li>2 screen orientations for mobile devices: portrait and landscape.<\/li>\n<li>10 standard mobile device display resolutions and 18 standard desktop\/laptop display resolutions from XGA to 4G.<\/li>\n<\/ul>\n<p>If you\u2019re doing the math, you think it\u2019s the browsers running on each platform (a total of 21 combinations) multiplied by the two orientations of the ten mobiles (2&#215;10)=20 added to the 18 desktop display resolutions.<\/p>\n<p style=\"text-align: center;\"><strong>21 x (20+18) = 21 x 38 = 798 Unique Screen Configurations to test<\/strong><\/p>\n<p>That\u2019s a lot of testing &#8212; for just one web page or screen in your mobile app.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5071 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image2.png\" alt=\"\" width=\"737\" height=\"694\"><\/p>\n<p>Except that it\u2019s worse. Let\u2019s say your app has 100 pages or screens to test.<\/p>\n<p style=\"text-align: center;\"><strong>798 Screen Configurations x 100 Screens in-app = 79,800 Screen Configurations to test<\/strong><\/p>\n<p>Meanwhile, companies are releasing new app versions into production as frequently as once a week, or even once a day.<\/p>\n<p>How many manual testers would you need to test 79,800 screen configurations in a week? Or a day? Could you even hire that many people?<\/p>\n<p>Wouldn\u2019t it be great if there was a way to automate this crazy-tedious process?<\/p>\n<p>Well, yes there is&#8230;<\/p>\n<h2>What is Automated Visual Testing?<\/h2>\n<p>Automated visual testing uses software to automate the process of comparing visual elements across various screen combinations to uncover visual defects.<\/p>\n<p>Automated visual testing piggybacks on your existing functional test scripts running in a tool like <a href=\"https:\/\/www.seleniumhq.org\/projects\/webdriver\/\">Selenium Webdriver<\/a>, <a href=\"https:\/\/www.cypress.io\/\">Cypress<\/a>, <a href=\"https:\/\/webdriver.io\/\">WebdriverIO<\/a>, or <a href=\"http:\/\/appium.io\/\">Appium<\/a>. As your script drives your app, your app creates web pages with static visual elements. Functional testing changes visual elements, so each step of a functional test creates a new UI state you can visually test.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5079 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image10.jpg\" alt=\"\" width=\"623\" height=\"350\"><\/p>\n<p>Automated visual testing evolved from functional testing. Rather than descending into the madness of writing assertions to check the properties of each visual element, automated visual testing tools visually check the visual appearance of an entire screen with just one assertion statement. This leads to test scripts that are MUCH simpler and easier to maintain.<\/p>\n<p>But, if you\u2019re not careful, you can go down an unproductive rat hole. I\u2019m talking about <strong>Snapshot Testing<\/strong>.<\/p>\n<h3>What is Snapshot Testing?<\/h3>\n<p>First generation automated visual testing uses a technology called <em><strong>snapshot testing<\/strong><\/em>. With snapshot testing, a bitmap of a screen is captured at various points of a test run and its pixels are compared to a baseline bitmap.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-5082 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image13.png\" alt=\"\" width=\"567\" height=\"230\"><\/p>\n<p>Snapshot testing algorithms are very simplistic: iterate through each pixel pair, then check if the color hex code is the same. If the color codes are different, raise a visual bug.<\/p>\n<p>Because they can be built relatively easily, there are a number of open-source and commercial snapshot testing tools. Unlike human testers, snapshot testing tools can spot pixel differences quickly and consistently. And that\u2019s a step forward. A computer can highlight the visual differences in the Hocus Focus cartoon easily. A number of these tools market themselves as enabling \u201cpixel perfect testing\u201d.<\/p>\n<p>Sounds like a good idea, right?<\/p>\n<h3>What are Problems With Snapshot Testing?<\/h3>\n<p>Alas, pixels aren\u2019t visual elements. Font smoothing algorithms, image resizing, graphics cards, and even rendering algorithms generate pixel differences. And that\u2019s just static content. The actual content can vary between any two interfaces. As a result, a tool that expects exact pixel matches between two images can be filled with pixel differences.<\/p>\n<p>If you want to see some examples of bitmap differences affecting snapshot testing, take a look at <a href=\"https:\/\/app14743.cloudwayssites.com\/blog\/why-screenshot-image-comparison-tools-fail\">the blog post we wrote on this topic last year<\/a>.<\/p>\n<p>Unfortunately, while you might think snapshot testing makes intuitive sense, practitioners like you are finding that the conditions for running successful bitmap comparisons require a stationary target, while your company continues to develop dynamic websites across a range of browsers and operating systems. You can try to force your app to behave a certain way &#8211; but you may not always succeed.<\/p>\n<h3>Can you share some details of Snapshot Testing Problems?<\/h3>\n<p>For example, when testing on a single browser and operating system:<\/p>\n<ul>\n<li>Identify and isolate (mute) fields that change over time, such as radio signal strength, battery state, and blinking cursors.<\/li>\n<li>Ignore user data that might otherwise change over time, such as visitor count.<\/li>\n<li>Determine how to support testing content on your site that must change frequently &#8211; especially if you are a media company or have an active blog.<\/li>\n<li>Consider how different hardware or software affects antialiasing.<\/li>\n<\/ul>\n<p>When doing <strong>cross-browser testing<\/strong>, you must also consider:<\/p>\n<ul>\n<li>Text wrapping, because you cannot guarantee the locations of text wrapping between two browsers using the same specifications. The text can break differently between two browsers, even with identical screen size.<\/li>\n<li>Image rendering software, which can affect the pixels of font antialiasing as well as images and can vary from browser to browser (and even on a single browser among versions).<\/li>\n<li>Image rendering hardware, which may render bitmaps differently.<\/li>\n<li>Variations in browser font size and other elements that affect the text.<\/li>\n<\/ul>\n<p>If you choose to pursue snapshot testing in spite of these issues, don\u2019t be surprised if you end up joining the group of <a href=\"https:\/\/medium.com\/@tomgold_48918\/why-i-stopped-using-snapshot-testing-with-jest-3279fe41ffb2\">experienced testers who have tried, and then ultimately abandoned, snapshot testing tools<\/a>.<\/p>\n<h3>Can I See Some Snapshot Testing Problems In Real Life?<\/h3>\n<p>Here are some quick examples of these real-life bitmap issues.<\/p>\n<p>If you use pixel testing for mobile apps, you\u2019ll need to deal with the very dynamic data at the top of nearly every screen: network strength, time, battery level, and more:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5077 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image8.png\" alt=\"\" width=\"633\" height=\"46\"><\/p>\n<p>When you have dynamic content that shifts over time &#8212; news, ads, user-submitted content &#8212; where you want to check to ensure that everything is laid out with proper alignment and no overlaps. Pixel comparison tools can\u2019t test for these cases. Twitter\u2019s user-generated content is even more dynamic, with new tweets, like, retweet, and comment counts changing by the second.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5073 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image4.png\" alt=\"\" width=\"598\" height=\"620\"><\/p>\n<p>Your app doesn\u2019t even need to change to confuse pixel tools. If your baselines and test screenshots were captured on different machines with different <a href=\"https:\/\/en.wikipedia.org\/wiki\/Spatial_anti-aliasing\">display settings for anti-aliasing<\/a>, that can turn pretty much the entire page into a false positive, like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-5080 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image11.png\" alt=\"\" width=\"690\" height=\"177\"><\/p>\n<p style=\"text-align: center;\">Source: storybook.js.org<\/p>\n<p>If you\u2019re using pixel tools and you still have to track down false positives and expose false negatives, what does that say about your testing efficiency?<\/p>\n<p>For these reasons, many companies throw out their pixel tools and go back to manual visual testing, with all of its issues.<\/p>\n<p>There\u2019s a better alternative: using AI &#8212; specifically computer vision &#8212; for visual testing.<\/p>\n<h2>How Do I Use AI for Automated Visual Testing?<\/h2>\n<p>The current generation of automated visual testing uses a class of artificial intelligence algorithms called <a href=\"https:\/\/blog.algorithmia.com\/introduction-to-computer-vision\/\">computer vision<\/a> as a core engine for visual comparison. Typically these algorithms are used to identify objects with images, such as with facial recognition. We call them visual AI testing tools.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5075 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image6.png\" alt=\"\" width=\"875\" height=\"365\"><\/p>\n<p>AI-powered automated visual testing combines a learning algorithm to interpret the relationship between a rendered page and intended display of visual elements with actual visual elements and locations. Like pixel tools, AI-powered automated visual testing takes page snapshots as your functionally tests run. Unlike pixel-based comparators, AI-powered automated visual test tools use algorithms instead of pixels to determine when errors have occurred.<\/p>\n<p>Unlike snapshot testers, AI-powered automated visual testing tools do not need special environments that remain static to ensure accuracy. Testing and real-world customer data show that AI testing tools have a high degree of accuracy even with dynamic content because the comparisons are based on relationships and not simply pixels.<\/p>\n<p>Here\u2019s a comparison of the kinds of issues that AI-powered visual testing tools can handle compared to snapshot testing tools:<\/p>\n\n<table id=\"tablepress-1\" class=\"tablepress tablepress-id-1\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Visual Testing Use Case<\/th><th class=\"column-2\">Snapshot Testing<\/th><th class=\"column-3\">Visual AI<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">Cross-browser testing<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">Account balances<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">Mobile device status bars<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">News content<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">Ad content<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-7\">\n\t<td class=\"column-1\">User submitted content<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-8\">\n\t<td class=\"column-1\">Suggested content<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-9\">\n\t<td class=\"column-1\">Notification icons<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-10\">\n\t<td class=\"column-1\">Content shifts<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-11\">\n\t<td class=\"column-1\">Mouse hovers<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-12\">\n\t<td class=\"column-1\">Cursors<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-13\">\n\t<td class=\"column-1\">Anti-aliasing settings<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<tr class=\"row-14\">\n\t<td class=\"column-1\">Browser upgrades<\/td><td class=\"column-2\">No<\/td><td class=\"column-3\">Yes<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-1 from cache -->\n<p>Some AI-powered test tools have been tested at a false positive rate of 0.001% (or 1 in every 100,000 errors).<\/p>\n<h3>AI-Powered Test Tools In Action<\/h3>\n<p>An AI-powered automated visual testing tool can test a wide range of visual elements across a range of OS\/browser\/orientation\/resolution combinations. Just running the first baseline of rendering and functional test on a single combination is sufficient to guide an AI-powered tool to test results across the range of potential platforms<\/p>\n<p>Here are some examples of how AI-powered automated visual testing improves visual test results by awareness of content.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-5072\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image3.png\" alt=\"\" width=\"1598\" height=\"740\"><\/p>\n<p>This is a comparison of two different USA Today homepage images. When an AI-powered tool looks at the layout comparison, the layout framework matters, not the content. Layout comparison ignores content differences; instead, layout comparison validates the existence of the content and relative placement. Compare that with a bitmap comparison of the same two pages (also called \u201cexact comparison:):<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-5081\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image12.png\" alt=\"\" width=\"1600\" height=\"743\"><\/p>\n<p>Literally, every non-white space (and even some of the white space) is called out.<\/p>\n<p>Which do you think would be more useful in your validation of your own content?<\/p>\n<p><\/p>\n<h2>When Should I Use Visual Testing?<\/h2>\n<p>You can do automated visual testing with each check-in of front-end code, after <a href=\"https:\/\/en.wikipedia.org\/wiki\/Unit_testing\">unit testing<\/a> and <a href=\"https:\/\/en.wikipedia.org\/wiki\/API_testing\">API testing<\/a>, and before <a href=\"https:\/\/en.wikipedia.org\/wiki\/Functional_testing\">functional testing<\/a> &#8212; ideally as part of your <a href=\"https:\/\/semaphoreci.com\/blog\/cicd-pipeline\">CI\/CD pipeline<\/a> running in Jenkins, Travis, or another <a href=\"https:\/\/en.wikipedia.org\/wiki\/Continuous_integration\">continuous integration<\/a> tool.<\/p>\n<p>How often? On days ending with \u201cy\u201d. \ud83d\ude42<\/p>\n<p>Because of the accuracy of AI-powered automated visual testing tools, they can be deployed in more than just functional and visual testing pre-production. AI-powered automated visual testing can help developers understand how visual element components will render across various systems. In addition to running in development, test engineers can also validate new code against existing platforms and new platforms against running code.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5076 aligncenter\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image7.png\" alt=\"\" width=\"818\" height=\"508\"><\/p>\n<p style=\"text-align: center;\">AI-powered tools like Applitools allow different levels of smart comparison.<\/p>\n<p>AI-powered visual testing tools are a key validation tool for any app or web presence that requires a regular change in content and format. For example, media companies change their content as frequently as twice per hour use AI-powered automated testing to isolate real errors that affect paying customers without impacting. And, AI-powered visual test tools are key tools in the test arsenal for any app or web presence going through brand revision or merger, as the low error rate and high accuracy lets companies identify and fix problems associated with major DOM, CSS and Javascript changes that are core to those updates.<\/p>\n<h2>Talk to Applitools<\/h2>\n<p>Applitools is the pioneer and leading vendor in AI-powered automated visual testing. Applitools has a range of options to help you become incredibly productive in application testing. We can help you test components in development. We can help you find the root cause of the visual errors you have encountered. And, we can run your tests on an <a href=\"https:\/\/app14743.cloudwayssites.com\/product-ultrafast-test-cloud\/\">Ultrafast Grid<\/a> that allows you to recreate your visual test in one environment across a number of others on various browser and OS configurations. Our goal is to help you realize the vision we share with our customers &#8211; you need to create functional tests for only one environment and let Applitools run the validation across all your customer environments after your first test has passed. We&#8217;d love to talk testing with you &#8211; feel free to <a href=\"https:\/\/app14743.cloudwayssites.com\/contact\/\">reach out to contact us anytime<\/a>.<\/p>\n<h2>More To Read About Visual Testing<\/h2>\n<p>If you liked reading this, here are some more Applitools posts and webinars for you.<\/p>\n<ol>\n<li><a href=\"https:\/\/app14743.cloudwayssites.com\/blog\/visual-testing-for-mobile-apps\/\">Visual Testing for Mobile Apps<\/a> by Angie Jones<\/li>\n<li><a href=\"https:\/\/app14743.cloudwayssites.com\/blog\/visual-ai-hype-or-reality\/\">Visual Assertions \u2013 Hype or Reality?<\/a> &#8211; by Anand Bagmar<\/li>\n<li><a href=\"https:\/\/app14743.cloudwayssites.com\/blog\/many-uses-of-visual-testing\/\">The Many Uses of Visual Testing<\/a> by Angie Jones<\/li>\n<li><a href=\"https:\/\/app14743.cloudwayssites.com\/blog\/visual-testing-as-an-aid-to-functional-testing\">Visual UI Testing as an Aid to Functional Testing<\/a> by Gil Tayar<\/li>\n<li><a href=\"https:\/\/app14743.cloudwayssites.com\/blog\/visual-testing-a-guide-for-front-end-developers\/\">Visual Testing: A Guide for Front End Developers<\/a> by Gil Tayar<\/li>\n<li><a href=\"https:\/\/app14743.cloudwayssites.com\/visual-testing\/\">Visual Testing FAQ<\/a><\/li>\n<\/ol>\n<p>Find out more about Applitools. Setup a <a href=\"https:\/\/app14743.cloudwayssites.com\/request-demo\">live demo<\/a> with us, or if you\u2019re the do-it-yourself type, <a href=\"https:\/\/app14743.cloudwayssites.com\/users\/register\">sign up for a free Applitools account<\/a> and follow one of our <a href=\"https:\/\/app14743.cloudwayssites.com\/tutorials\">tutorials<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Foundations Functional Testing Visual Testing Performance Testing Accessibility Testing Mobile Testing Frameworks Cypress Selenium Playwright Verifying sortable tables Puppetteer Appium What is Meant By Visual Testing? Visual testing evaluates the&#8230;<\/p>\n","protected":false},"author":94,"featured_media":0,"parent":37510,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v24.5 (Yoast SEO v24.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>What Is Visual Testing? - Learn Foundations of Testing<\/title>\n<meta name=\"description\" content=\"Foundations Functional Testing Visual Testing Performance Testing Accessibility Testing Mobile Testing Frameworks Cypress Selenium Playwright Verifying sortable tables Puppetteer Appium What is Meant By Visual Testing? Visual testing evaluates the...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Learn - Concepts - Visual Testing\" \/>\n<meta property=\"og:description\" content=\"Foundations Functional TestingVisual TestingPerformance TestingAccessibility TestingMobile Testing Frameworks CypressSeleniumPlaywrightVerifying sortable\" \/>\n<meta property=\"og:url\" content=\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"AI-Powered End-to-End Testing | Applitools\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-09T15:41:15+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"16 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/\",\"name\":\"What Is Visual Testing? - Learn Foundations of Testing\",\"isPartOf\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image9.png\",\"datePublished\":\"2022-04-28T19:11:07+00:00\",\"dateModified\":\"2022-05-09T15:41:15+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#primaryimage\",\"url\":\"\",\"contentUrl\":\"\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/app14743.cloudwayssites.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Learn\",\"item\":\"https:\/\/app14743.cloudwayssites.com\/learn\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Learn &#8211; Concepts\",\"item\":\"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Learn &#8211; Concepts &#8211; Visual Testing\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#website\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/\",\"name\":\"Applitools Visual AI\",\"description\":\"Applitools delivers full end-to-end test automation with AI infused at every step.\",\"publisher\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/app14743.cloudwayssites.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#organization\",\"name\":\"Applitools\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png\",\"contentUrl\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png\",\"width\":156,\"height\":28,\"caption\":\"Applitools\"},\"image\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"What Is Visual Testing? - Learn Foundations of Testing","description":"Foundations Functional Testing Visual Testing Performance Testing Accessibility Testing Mobile Testing Frameworks Cypress Selenium Playwright Verifying sortable tables Puppetteer Appium What is Meant By Visual Testing? Visual testing evaluates the...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/","og_locale":"en_US","og_type":"article","og_title":"Learn - Concepts - Visual Testing","og_description":"Foundations Functional TestingVisual TestingPerformance TestingAccessibility TestingMobile Testing Frameworks CypressSeleniumPlaywrightVerifying sortable","og_url":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/","og_site_name":"AI-Powered End-to-End Testing | Applitools","article_modified_time":"2022-05-09T15:41:15+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"16 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/","url":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/","name":"What Is Visual Testing? - Learn Foundations of Testing","isPartOf":{"@id":"https:\/\/app14743.cloudwayssites.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#primaryimage"},"image":{"@id":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2019\/05\/image9.png","datePublished":"2022-04-28T19:11:07+00:00","dateModified":"2022-05-09T15:41:15+00:00","breadcrumb":{"@id":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#primaryimage","url":"","contentUrl":""},{"@type":"BreadcrumbList","@id":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/visual-testing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/app14743.cloudwayssites.com\/"},{"@type":"ListItem","position":2,"name":"Learn","item":"https:\/\/app14743.cloudwayssites.com\/learn\/"},{"@type":"ListItem","position":3,"name":"Learn &#8211; Concepts","item":"https:\/\/app14743.cloudwayssites.com\/learn\/concepts\/"},{"@type":"ListItem","position":4,"name":"Learn &#8211; Concepts &#8211; Visual Testing"}]},{"@type":"WebSite","@id":"https:\/\/app14743.cloudwayssites.com\/#website","url":"https:\/\/app14743.cloudwayssites.com\/","name":"Applitools Visual AI","description":"Applitools delivers full end-to-end test automation with AI infused at every step.","publisher":{"@id":"https:\/\/app14743.cloudwayssites.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/app14743.cloudwayssites.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/app14743.cloudwayssites.com\/#organization","name":"Applitools","url":"https:\/\/app14743.cloudwayssites.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/","url":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png","contentUrl":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png","width":156,"height":28,"caption":"Applitools"},"image":{"@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/pages\/37547"}],"collection":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/users\/94"}],"replies":[{"embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/comments?post=37547"}],"version-history":[{"count":0,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/pages\/37547\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/pages\/37510"}],"wp:attachment":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/media?parent=37547"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}