{"id":18453,"date":"2020-05-15T20:23:46","date_gmt":"2020-05-15T20:23:46","guid":{"rendered":"https:\/\/app14743.cloudwayssites.com\/?page_id=18453"},"modified":"2024-02-06T09:03:19","modified_gmt":"2024-02-06T17:03:19","slug":"automated-visual-testing-best-practices-guide","status":"publish","type":"page","link":"https:\/\/app14743.cloudwayssites.com\/automated-visual-testing-best-practices-guide\/","title":{"rendered":"Learn – Automated Visual Testing Best Practices Guide"},"content":{"rendered":"\n
\n

Introduction<\/h2>\n\n\n\n

This guide, authored by Angie Jones, provides a set of best practices and our recommendations for implementing and maintaining automated visual tests with Applitools based on our customers\u2019 experiences.<\/p>\n\n\n\n

Many of the top high-performing companies in software, financial services, and healthcare verticals are using Applitools Eyes for their visual test automation needs. <\/p>\n\n\n\n

We\u2019ve talked with these companies to gain a better understanding of how they are using the product and to identify common practices that are working well for teams. <\/p>\n\n\n\n

This guide provides a set of best practices and our recommendations for implementing and maintaining automated visual tests<\/a> with Applitools based on our customers\u2019 experiences.<\/p>\n\n\n

\n
\n

Looking for a PDF version of the Automated Visual Testing Best Practices Guide?<\/p>\n <\/div>\n

\n Download PDF<\/a>\n <\/div>\n <\/div><\/div><\/div>\n\n\n\n
\n

Summary of Recommendations<\/h2>\n\n\n\n

Here is a summary of our top recommendations contained within this guide. Details on each of these, including when to detour from the recommendation can be found within the following sections of the guide.<\/p>\n\n\n

\n
\n
\n \"\"\n
\n

Use Eyes for both visual & functional assertions<\/h2>\n

The Applitools Eyes API should be used to complement or encompass functional assertions with less lines of code and more test coverage.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Verify the entire page<\/h2>\n

While there are various options to visually verify your application with Applitools Eyes, we recommend verifying the entire page of your application to obtain the most coverage from your visual testing.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Use the Strict match level<\/h2>\n

By default, Applitools Eyes uses the Strict match level algorithm (our recommended comparison method), which employs AI to compare your baseline image with the current image from a regression run.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Use Ignore annotation for data that is not relevant to the test<\/h2>\n

We recommend using our Ignore annotation to ignore parts of your application\u2019s page that are not pertinent to the test.
\nRead more<\/a><\/p>\n

 <\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Programmatically apply annotations when possible<\/h2>\n

When annotations are needed, use fluent checks to programmatically apply them to baseline images before the test is executed.
\nRead more<\/a><\/p>\n

 <\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Use steps to include multiple visual checkpoints in a single test<\/h2>\n

It\u2019s recommended to perform multiple visual checks where there are multiple visual states that need to be verified in a given scenario.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Group related tests into batches to enable easier management as well as automated maintenance of baselines<\/h2>\n

For related tests, it\u2019s recommended to use batches so that the tests display together on the Applitools Dashboard within an aggregated view.
\nRead more<\/a><\/p>\n

 <\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Utilize a wrapper class to encapsulate visual checks<\/h2>\n

It\u2019s a good practice to create a wrapper class that isolates the Eyes API calls so that if there are any changes to the API, the required changes to the test code will be limited to an individual class.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Use Layout mode for early unverified localization testing<\/h2>\n

Before your localized app has been verified by a specialist, use Layout mode. After you\u2019re sure the content is correct, use Strict mode.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Tag visual tests and execute them any time the UI is under test<\/h2>\n

Utilize tagging capabilities of your test runner to easily include and exclude visual tests when executing.
\nRead more<\/a><\/p>\n

 <\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Use the Ultrafast Grid to run cross-platform tests<\/h2>\n

Applitools\u2019 Ultrafast Grid enables you to execute your tests across many different browsers, viewports, and devices at a fraction of the time it takes with other solutions.
\nRead more<\/a><\/p>\n

 <\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Gate your builds with relevant visual checks<\/h2>\n

With the accuracy of Eyes API, we feel confident recommending that you have your visual tests follow the same gating strategy used for your functional tests.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Visual checks can be used when using feature files<\/h2>\n

When practicing BDD, continue to write feature files without implementation detail and use the step definition code to execute visual checks where necessary.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Use the Applitools dashboard to collaborate<\/h2>\n

The Applitools Dashboard should be used to annotate screenshots of failures with bug regions, assign failures to developers to review, and remark on questionable changes.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n

\n \"\"\n
\n

Integrate Applitools with Slack for faster collaboration on failed visual tests<\/h2>\n

If your team already uses Slack for collaboration, we recommend integrating Applitools with Slack, which makes sharing and resolving failures easier.
\nRead more<\/a><\/p>\n <\/div>\n <\/div>\n <\/div>\n <\/section>\n <\/div><\/div>\n\n\n\n

\n

What is Visual Testing<\/h2>\n\n\n\n

Automated visual testing<\/a> is a technique to verify that your application appears to the user as you intended. The Applitools Eyes API can be used to complement or encompass functional assertions<\/a> <\/strong>with less lines of code and more test coverage.<\/p>\n\n\n\n

Applitools Eyes is both flexible and powerful, and makes a wonderful addition to any UI or mobile test automation toolkit.<\/p>\n\n\n\n

Visual testing can be used whenever there\u2019s a need to verify the display of data on a web or mobile application.<\/p>\n\n\n\n

Functional assertion libraries verify information in the DOM<\/a>, but are incapable of verifying how that data is actually presented to the user.<\/p>\n\n\n\n

For example, here\u2019s a visual bug on Instagram. Functional assertions that check if the text exists would pass because yes, it does indeed exist in the DOM. However, it misses the fact that all of the text is overlapping and therefore is unreadable by the user.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

To avoid missing such bugs, Applitools should be used in your mobile and web UI tests.<\/p>\n<\/div><\/div>\n\n\n\n

\n

Where to Use Visual Testing<\/h2>\n\n\n\n

Our recommendation is to use Applitools Eyes as a superset for both visual and functional assertions.<\/p>\n\n\n\n

Just like functional assertions, visual assertions are used at the point of verification. An automation tool (e.g. Selenium, Cypress, Appium, etc) is used to interact with the application and then Applitools Eyes is used to visually verify the outcome.<\/p>\n\n\n\n

Also, just like with functional assertions, it\u2019s important to make the visual assertion only when the application is ready to be verified. For example, if your app needs to load data, your test code will need to wait until that data is loaded before verifying it. This is true of both functional and visual assertions.<\/p>\n\n\n\n

Some users create separate tests for their functional and visual checks. They utilize the functional tests to assert that all data is accurate, and they use the visual tests to assert that there are no visual bugs.<\/p>\n\n\n\n

While this is certainly a viable option, it is redundant. Not only is the visual check asserting that there are no visual bugs, but it is also asserting that everything on the page is correct, including the data.<\/p>\n\n\n\n

Therefore, visual checks are a superset of functional checks, and in many cases can do the job of both.<\/p>\n\n\n\n

For example, consider a test that adds an item to a shopping cart, and then wants to verify that the item was properly added.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

Including functional assertions for everything here is unnecessary if using a visual assertion. Not only will the visual assertion make sure all of the items are displayed properly, it is inherently also making sure that the product title, size, color, quantity, price, total, etc are also correct.<\/p>\n\n\n\n

An exception to this suggestion would be when it\u2019s necessary to verify things that are not visible on the UI, for example for backend databases or web service calls. In these cases, we recommend using visual assertions for the UI and coupling these with functional assertions for the backend.<\/p>\n<\/div><\/div>\n\n\n\n

\n

Visual Testing Techniques<\/h2>\n\n\n\n

Full Page<\/strong><\/h3>\n\n\n\n

While there are various options to visually verify your application with Applitools Eyes, we recommend verifying the entire page of your application to obtain the most coverage from your visual testing. This is the most common usage of visual testing.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

The benefit of using this approach is that everything on the entire screen will be captured and verified so you don\u2019t have to worry about missing key assertion points.<\/p>\n\n\n\n

For web tests, we recommend setting the viewport size when calling Eyes\u2019 open method and specifying a size that\u2019s at least 50 pixels below the maximum possible viewport size. This ensures that your test is run against the same window size each time. Otherwise, browser viewport variances can result in a new baseline being created.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

When using Applitools Eyes to check the entire page, we also recommend setting the API to capture a full page screenshot. This will ensure that the entire screen will be captured, even if Applitools has to scroll to capture it in its entirety. When requesting a full page screenshot, Eyes will capture a screenshot of the visible portion of the page, then scroll and capture a screenshot of the newly visible portion of the page. It will continue this until the entire page has been captured. Once done, Eyes will stitch together each of the individual images to make one single image of the entire page. In most cases, this is all that\u2019s needed. However, on applications with sticky headers (meaning the header of the page is visible even when scrolling), then you don\u2019t want to have the header duplicated in every portional screenshot and appearing multiple times in the stitched version of the full image. To avoid this, you can make a call to the Eyes command to set the stitch level <\/a><\/strong>to CSS.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

While validating the entire page is the most common and thorough approach, we do realize that in some cases, this may be too much for your needs.<\/p>\n\n\n\n

Examples:<\/p>\n\n\n\n