{"id":19990,"date":"2020-06-30T23:04:40","date_gmt":"2020-06-30T23:04:40","guid":{"rendered":"https:\/\/app14743.cloudwayssites.com\/?p=19990"},"modified":"2022-09-27T12:39:49","modified_gmt":"2022-09-27T19:39:49","slug":"end-smoke-sanity-regression","status":"publish","type":"post","link":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/","title":{"rendered":"The end of Smoke, Sanity and Regression"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\" id=\"h.c5hdgb8zz8oh\">Test Categories<\/h2>\n\n\n\n<p>When it comes to Test Strategy for any reasonable sized \/ complex product, one aspect would always be there \u2013 the categories of tests that would be created, and (hopefully) automated \u2013 ex: Smoke, Sanity, Regression, etc.<\/p>\n\n\n\n<p><strong>Have you ever wondered why these categories are required?<\/strong>&nbsp;Well, the answer is quite simple. We want to get quick feedback from our tests (re-executed by humans, or better yet \u2013 by machines in the form of automated tests). These categories offer a stepping-stone approach to getting feedback quickly.<\/p>\n\n\n\n<p><strong>But, have you wondered, why is your test feedback slow for functional automation?<\/strong>&nbsp;In most cases, the number of unit tests would be a decent multiple (10-1000x) of the number of automated functional tests. YET, I have never ever seen categories like smoke, sanity, regression being created for the unit tests. Why? Again, the answer is very simple \u2013 the unit tests run extremely fast and provide feedback almost immediately.<\/p>\n\n\n\n<p>The next question is obvious &#8211; why are the automated tests slow in giving running and providing feedback? There could be various reasons for this:<\/p>\n\n\n\n<ul><li>The functional tests have to launch the browser \/ native application before running the test<\/li><li>The test runs against fully \/ partially integrated system (as opposed to to a stubbed \/ mocked unit test)<\/li><\/ul>\n\n\n\n<p>However, there are also other factors that contribute to the slow functional tests:<\/p>\n\n\n\n<ul><li>Non-optimal tool sets used for automation<\/li><li>Skills \/ capabilities on the team do not match what is required to automate effectively<\/li><li>Test automation strategy \/ criteria is not well defined.<\/li><li>The Test Automation framework is not designed &amp; architected correctly, hence making it inefficient, slow, and gives inconsistent results.<\/li><li>Repeating the execution of the same set of Functional Automated Tests on variety of browsers, and viewport sizes<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h.fgdohsnfus9r\">Getting faster feedback from Automated Functional Tests<\/h2>\n\n\n\n<p>There are various techniques &amp; practices that can be used (appropriately, and relevant to the context of product-under-test) to get faster, reliable, consistent feedback from your Functional Tests.<\/p>\n\n\n\n<p>Some of these are:<\/p>\n\n\n\n<ul><li>Evolve your test strategy based on the concept of <a href=\"https:\/\/martinfowler.com\/articles\/practical-test-pyramid.html\" target=\"_blank\" rel=\"noopener noreferrer\">Test Automation Pyramid<\/a>&nbsp;<\/li><li>Design &amp; architect your Functional Test Automation Framework correctly. This post on \u201c<a href=\"https:\/\/www.infoq.com\/articles\/test-automation-ai-ml\/\" target=\"_blank\" rel=\"noopener noreferrer\">Test Automation in the World of AI &amp; ML<\/a>\u201d highlights various criteria to be considered to build a good Test Automation Framework.<\/li><li>Include Visual AI solution from Applitools that speeds up test implementation, reduces flakiness in your Functional Automation execution, includes AI-based Visual Testing and eliminates the need for Cross-browser testing &#8211; i.e. removes the need to execute the same functional tests in multiple browsers<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h.j7guj2mde4nn\">Optimizing Functional Tests using Visual AI<\/h2>\n\n\n\n<p>Let&#8217;s say I want to write tests to login to github.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"667\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0.png\" alt=\"github login screen\" class=\"wp-image-19978\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-300x125.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1024x427.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-768x320.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1536x640.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>Expected functionality is that if I click on \u201cSign In\u201d button without entering any credentials, I see the error message as below:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"667\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-6.png\" alt=\"github login screen with errors\" class=\"wp-image-19988\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-6.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-6-300x125.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-6-1024x427.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-6-768x320.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-6-1536x640.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>To implement a Selenium-based test for such validation, I would write it as below:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"764\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon2.png\" alt=\"code example\" class=\"wp-image-19984\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon2.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon2-300x143.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon2-1024x489.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon2-768x367.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon2-1536x733.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>Now, when I run this test against a new build, which may have added functionality, like shown below, lets see the status of this test result.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1344\" height=\"328\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon1.png\" alt=\"code example 2\" class=\"wp-image-19979\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon1.png 1344w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon1-300x73.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon1-1024x250.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon1-768x187.png 768w\" sizes=\"(max-width: 1344px) 100vw, 1344px\" \/><\/figure><\/div>\n\n\n<p>Sure enough, the test failed. If we investigate why, you will see that the functionality has evolved.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"668\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-4.png\" alt=\"github login with errors and labels\" class=\"wp-image-19985\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-4.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-4-300x125.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-4-1024x428.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-4-768x321.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-4-1536x641.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>However, our implemented test failed for the 1st error it found, i.e.<\/p>\n\n\n\n<p>The title of the page has changed from \u201cSignIn\u201d to \u201cSign in to Github\u201d<\/p>\n\n\n\n<p>I am sure you have also experienced these types of test results.<\/p>\n\n\n\n<p>The challenges I see with the above are:<\/p>\n\n\n\n<ul><li>The product-under-test is always going to evolve. That means your test is always going to report incorrect details<\/li><li>In this case, the test reported only the 1st failure it came across, i.e. the 1st assertion failure. The rest of the issues that could have been captured by the test were not even executed.<\/li><li>The test would not have been to capture the color changes<\/li><li>The new functionality did not have any relevance to the automated test<\/li><\/ul>\n\n\n\n<p>Is there a better way to do this?<\/p>\n\n\n\n<p>YES &#8211; there is! Use Applitools Visual AI!<\/p>\n\n\n\n<p>After <a href=\"https:\/\/auth.applitools.com\/users\/register\">signing up for a free Applitools account<\/a>, I integrated <a href=\"https:\/\/app14743.cloudwayssites.com\/tutorials\/selenium-java.html\">Applitools Selenium-Java SDK using the tutorial<\/a>&nbsp;into my test.<\/p>\n\n\n\n<p>My test now looks like this:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"461\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon3.png\" alt=\"code example 3\" class=\"wp-image-19987\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon3.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon3-300x86.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon3-1024x295.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon3-768x221.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/carbon3-1536x443.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>As you can see, my test code has the following changes:<\/p>\n\n\n\n<ul><li>There are no assertions that I had before<\/li><li>There are hence, fewer locators I need to worry about<\/li><li>Hence the test code is more stable, faster, cleaner &amp; simpler<\/li><\/ul>\n\n\n\n<p>The test still fails in this case as well, because of the new build. However, the reason for the failures is very interesting.<\/p>\n\n\n\n<p>When I look at the Applitools dashboard for these mismatches reported, I am able to see the details of what went wrong, functionally &amp; visually!<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"504\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-5.png\" alt=\"pasted image 0 5\" class=\"wp-image-19986\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-5.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-5-300x95.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-5-1024x323.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-5-768x242.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-5-1536x484.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>Here are the details of the errors<\/p>\n\n\n\n<p><strong>Screen 1 &#8211; before login<\/strong><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"866\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1.png\" alt=\"pasted image 0 1\" class=\"wp-image-19981\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-300x162.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-1024x554.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-768x416.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-1536x831.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p><strong>Screen 2 &#8211; after login with no credentials provided<\/strong><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"865\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-3.png\" alt=\"pasted image 0 3\" class=\"wp-image-19983\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-3.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-3-300x162.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-3-1024x554.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-3-768x415.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-3-1536x830.png 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>From this point, now, I can report the failures in functionality \/ user experience as a defect using the <a href=\"https:\/\/app14743.cloudwayssites.com\/docs\/topics\/test-manager\/viewers\/using-jira-with-eyes.html#Link\">Jira integration<\/a>, and accept the new functionality and <a href=\"https:\/\/app14743.cloudwayssites.com\/docs\/topics\/overview\/overview-reviewing-test-results.html?Highlight=update%20baseline\">update the baseline appropriately<\/a>, with simple clicks in the dashboard.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h.8o32008xincx\">Scaling the test execution<\/h2>\n\n\n\n<p>A typical way to scale the test is to set up your own infrastructure with different browser versions, or to use a cloud provider which will manage the infrastructure &amp; browsers. There are a lot of disadvantages in either approach &#8211; from a cost, maintenance, security &amp; compliance perspective.<\/p>\n\n\n\n<p>To use any of the above solutions, you first need to ensure that your tests can run successfully and deterministically against all the supported browsers. That is a substantial added effort.<\/p>\n\n\n\n<p>This approach of scaling seems flawed to me.<\/p>\n\n\n\n<p>If you think about it, where are the actual bugs coming from? In my opinion, the bugs are related to:<\/p>\n\n\n\n<ul><li>Server bugs which are device \/ browser independent<\/li><\/ul>\n\n\n\n<p>Ex: A broken DB query, logical error, backend performance issues, etc.<\/p>\n\n\n\n<ul><li>Functional bugs &#8211; which are 99% are device \/ browser agnostic. This is because:\n<ul>\n<li>Modern browsers conform to the W3C standard<\/li>\n<li>Logical bugs occur in all client environments.<\/li>\n<\/ul>\n<\/li><\/ul>\n\n\n\n<p>Examples: not validating an input field, reversed sorting order, etc.<\/p>\n\n\n\n<p>That said, though the browsers are W3C standard compliant, they still have their own implementation of the rendering engines, which means, <strong>t<\/strong><strong>he real value of running tests in different browsers and varying viewport sizes is in finding visual bugs!<\/strong><\/p>\n\n\n\n<p>By using Applitools, I get access to another awesome feature, due to which I can avoid running the same set of tests on multiple browsers &amp; viewport sizes. This ended up saving my test implementation, execution and maintenance time. That is the Applitools <a href=\"https:\/\/app14743.cloudwayssites.com\/ultrafast-grid\/\">Ultrafast Grid<\/a>.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"798\" height=\"444\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-7.png\" alt=\"pasted image 0 7\" class=\"wp-image-19989\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-7.png 798w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-7-300x167.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-7-768x427.png 768w\" sizes=\"(max-width: 798px) 100vw, 798px\" \/><\/figure><\/div>\n\n\n<p>See this quick video from about the <a href=\"https:\/\/www.youtube.com\/watch?v=AlVQ-P_vQN8\" target=\"_blank\" rel=\"noopener noreferrer\">Applitools Ultrafast Grid<\/a>.<\/p>\n\n\n\n<p>To enable the Applitools Ultrafast Grid, I just needed to add a few configuration details when instantiating Eyes. In my case, I added the below to my Eyes configuration.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"1544\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2.png\" alt=\"pasted image 0 2\" class=\"wp-image-19982\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2.png 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2-300x290.png 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2-1024x988.png 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2-768x741.png 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2-1536x1482.png 1536w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-2-30x30.png 30w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<p>When I ran my test again on my laptop, and checked the results in the Applitools dashboard, I saw the awesome power of the Ultrafast Grid.<\/p>\n\n\n\n<p>NOTE: The test ran just once on my laptop, however the Ultrafast Grid rendered the same screenshots by capturing the relevant page\u2019s DOM &amp; CSS in each of the browser &amp; viewport combinations I provided above, and then did a visual comparison. As a result, in a little more than regular test execution time, I got functional &amp; visual validation done for ALL my supported \/ provided configurations. Isn\u2019t that neat!<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"1020\" src=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1.jpg\" alt=\"pasted image 0 1\" class=\"wp-image-19980\" srcset=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1.jpg 1600w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-300x191.jpg 300w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-1024x653.jpg 1024w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-768x490.jpg 768w, https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/pasted-image-0-1-1536x979.jpg 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><\/figure><\/div>\n\n\n<h2 class=\"wp-block-heading\" id=\"h.z3qlyvy9u5ut\">Do we need Smoke, Sanity, Regression suites?<\/h2>\n\n\n\n<p>To summarize:<\/p>\n\n\n\n<ul><li>Do not blindly start with classifying your tests in different categories. Challenge yourself to do better!<\/li><li>Have a Test Automation strategy and know your test automation framework objective &amp; criteria (\u201c<a href=\"https:\/\/www.infoq.com\/articles\/test-automation-ai-ml\/\" target=\"_blank\" rel=\"noopener noreferrer\">Test Automation in the World of AI &amp; ML<\/a>\u201d highlights various criteria to be considered to build a good Test Automation Framework)<\/li><li>Choose the toolset wisely<\/li><li>After all the correct (subjective) approaches taken, if your test execution (in a single browser) is still taking more than say, 10 min for execution, then you can run your tests in parallel, and subsequently, split the test suite into smaller suites which can give you progressive indication of quality<\/li><li>Applitools with its AI-power algorithms can make your functional tests lean, simple, robust and includes UI \/ UX validation<\/li><li>Applitools Ultrafast Grid will remove the need for Cross-Browser testing, and instead with a single test execution run, validate functionality &amp; UI \/ Visual rendering for all supported Browsers &amp; Viewports<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Test Categories When it comes to Test Strategy for any reasonable sized \/ complex product, one aspect would always be there \u2013 the categories of tests that would be created,&#8230;<\/p>\n","protected":false},"author":37,"featured_media":19992,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[10004],"tags":[12830,10050,10844,12688,12690,10159,11051,12964,10192,12686],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v24.5 (Yoast SEO v24.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The end of Smoke, Sanity and Regression - AI-Powered End-to-End Testing | Applitools<\/title>\n<meta name=\"description\" content=\"Instead of creating smoke, sanity, or other category of &quot;fast&quot; end-to-end tests, use Visual AI as a capture engine for validating your test output.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The end of Smoke, Sanity and Regression\" \/>\n<meta property=\"og:description\" content=\"Test Categories When it comes to Test Strategy for any reasonable sized \/ complex product, one aspect would always be there \u2013 the categories of tests that\" \/>\n<meta property=\"og:url\" content=\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\" \/>\n<meta property=\"og:site_name\" content=\"AI-Powered End-to-End Testing | Applitools\" \/>\n<meta property=\"article:published_time\" content=\"2020-06-30T23:04:40+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-09-27T19:39:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"640\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Anand Bagmar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Anand Bagmar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\"},\"author\":{\"name\":\"Anand Bagmar\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/person\/fc39e08e8c24a4efcf2445d087febd1d\"},\"headline\":\"The end of Smoke, Sanity and Regression\",\"datePublished\":\"2020-06-30T23:04:40+00:00\",\"dateModified\":\"2022-09-27T19:39:49+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\"},\"wordCount\":1385,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png\",\"keywords\":[\"Code Snippets\",\"Cross-browser Testing\",\"Root Cause Analysis\",\"Technical Leaders\",\"Test Engineers\",\"Thought Leaders\",\"Ultrafast Grid\",\"User Stories\",\"Visual AI\",\"Visual Testing Strategies\"],\"articleSection\":[\"Advanced Topics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\",\"name\":\"The end of Smoke, Sanity and Regression - AI-Powered End-to-End Testing | Applitools\",\"isPartOf\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png\",\"datePublished\":\"2020-06-30T23:04:40+00:00\",\"dateModified\":\"2022-09-27T19:39:49+00:00\",\"description\":\"Instead of creating smoke, sanity, or other category of \\\"fast\\\" end-to-end tests, use Visual AI as a capture engine for validating your test output.\",\"breadcrumb\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png\",\"contentUrl\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png\",\"width\":1280,\"height\":640},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/app14743.cloudwayssites.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Advanced Topics\",\"item\":\"https:\/\/app14743.cloudwayssites.com\/blog\/category\/advanced-topics\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"The end of Smoke, Sanity and Regression\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#website\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/\",\"name\":\"Applitools Visual AI\",\"description\":\"Applitools delivers full end-to-end test automation with AI infused at every step.\",\"publisher\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/app14743.cloudwayssites.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#organization\",\"name\":\"Applitools\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png\",\"contentUrl\":\"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png\",\"width\":156,\"height\":28,\"caption\":\"Applitools\"},\"image\":{\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/person\/fc39e08e8c24a4efcf2445d087febd1d\",\"name\":\"Anand Bagmar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/app14743.cloudwayssites.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/6d6fe58f00f499285458d74a1b3e4a1a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/6d6fe58f00f499285458d74a1b3e4a1a?s=96&d=mm&r=g\",\"caption\":\"Anand Bagmar\"},\"url\":\"https:\/\/app14743.cloudwayssites.com\/blog\/author\/anandbagmar\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"The end of Smoke, Sanity and Regression - AI-Powered End-to-End Testing | Applitools","description":"Instead of creating smoke, sanity, or other category of \"fast\" end-to-end tests, use Visual AI as a capture engine for validating your test output.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/","og_locale":"en_US","og_type":"article","og_title":"The end of Smoke, Sanity and Regression","og_description":"Test Categories When it comes to Test Strategy for any reasonable sized \/ complex product, one aspect would always be there \u2013 the categories of tests that","og_url":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/","og_site_name":"AI-Powered End-to-End Testing | Applitools","article_published_time":"2020-06-30T23:04:40+00:00","article_modified_time":"2022-09-27T19:39:49+00:00","og_image":[{"width":1280,"height":640,"url":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png","type":"image\/png"}],"author":"Anand Bagmar","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Anand Bagmar","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#article","isPartOf":{"@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/"},"author":{"name":"Anand Bagmar","@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/person\/fc39e08e8c24a4efcf2445d087febd1d"},"headline":"The end of Smoke, Sanity and Regression","datePublished":"2020-06-30T23:04:40+00:00","dateModified":"2022-09-27T19:39:49+00:00","mainEntityOfPage":{"@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/"},"wordCount":1385,"commentCount":0,"publisher":{"@id":"https:\/\/app14743.cloudwayssites.com\/#organization"},"image":{"@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage"},"thumbnailUrl":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png","keywords":["Code Snippets","Cross-browser Testing","Root Cause Analysis","Technical Leaders","Test Engineers","Thought Leaders","Ultrafast Grid","User Stories","Visual AI","Visual Testing Strategies"],"articleSection":["Advanced Topics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/","url":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/","name":"The end of Smoke, Sanity and Regression - AI-Powered End-to-End Testing | Applitools","isPartOf":{"@id":"https:\/\/app14743.cloudwayssites.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage"},"image":{"@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage"},"thumbnailUrl":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png","datePublished":"2020-06-30T23:04:40+00:00","dateModified":"2022-09-27T19:39:49+00:00","description":"Instead of creating smoke, sanity, or other category of \"fast\" end-to-end tests, use Visual AI as a capture engine for validating your test output.","breadcrumb":{"@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#primaryimage","url":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png","contentUrl":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/06\/no-smoking-41752_1280.png","width":1280,"height":640},{"@type":"BreadcrumbList","@id":"https:\/\/app14743.cloudwayssites.com\/blog\/end-smoke-sanity-regression\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/app14743.cloudwayssites.com\/"},{"@type":"ListItem","position":2,"name":"Advanced Topics","item":"https:\/\/app14743.cloudwayssites.com\/blog\/category\/advanced-topics\/"},{"@type":"ListItem","position":3,"name":"The end of Smoke, Sanity and Regression"}]},{"@type":"WebSite","@id":"https:\/\/app14743.cloudwayssites.com\/#website","url":"https:\/\/app14743.cloudwayssites.com\/","name":"Applitools Visual AI","description":"Applitools delivers full end-to-end test automation with AI infused at every step.","publisher":{"@id":"https:\/\/app14743.cloudwayssites.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/app14743.cloudwayssites.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/app14743.cloudwayssites.com\/#organization","name":"Applitools","url":"https:\/\/app14743.cloudwayssites.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/","url":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png","contentUrl":"https:\/\/app14743.cloudwayssites.com\/wp-content\/uploads\/2020\/03\/applitools.png","width":156,"height":28,"caption":"Applitools"},"image":{"@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/person\/fc39e08e8c24a4efcf2445d087febd1d","name":"Anand Bagmar","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/app14743.cloudwayssites.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/6d6fe58f00f499285458d74a1b3e4a1a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/6d6fe58f00f499285458d74a1b3e4a1a?s=96&d=mm&r=g","caption":"Anand Bagmar"},"url":"https:\/\/app14743.cloudwayssites.com\/blog\/author\/anandbagmar\/"}]}},"_links":{"self":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/posts\/19990"}],"collection":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/comments?post=19990"}],"version-history":[{"count":0,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/posts\/19990\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/media\/19992"}],"wp:attachment":[{"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/media?parent=19990"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/categories?post=19990"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/app14743.cloudwayssites.com\/wp-json\/wp\/v2\/tags?post=19990"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}