{"id":45973,"date":"2023-01-17T15:42:06","date_gmt":"2023-01-17T23:42:06","guid":{"rendered":"https:\/\/app14743.cloudwayssites.com\/?p=45973"},"modified":"2023-01-24T08:49:53","modified_gmt":"2023-01-24T16:49:53","slug":"enhancing-ui-ux-testing-with-ai","status":"publish","type":"post","link":"https:\/\/app14743.cloudwayssites.com\/blog\/enhancing-ui-ux-testing-with-ai\/","title":{"rendered":"Enhancing UI\/UX Testing with AI"},"content":{"rendered":"\n
\n

This article is based on our recent webinar, <\/em>How to Enhance UI\/UX Testing by Leveraging AI<\/span><\/strong><\/em><\/a>, led by Chris Rolls from TTC and Andrew Knight from Applitools.<\/em> Editing by Marii Boyken.<\/em><\/p>\n\n\n\n

Last week, I hosted a webinar with Chris Rolls from TTC. In the webinar, Chris and I talked about the current state of software testing, where it\u2019s going, and how visual AI and automation will impact the future of software testing. In this article, I\u2019ll be recapping the insights shared from the webinar.<\/p>\n<\/div><\/div>\n\n\n\n

\n

The current state of testing<\/h2>\n\n\n\n

Software testing is often seen in businesses as a necessary evil at the end of the software development lifecycle to find issues before they reach production. Chris and I strongly agree that software quality is crucial to modern businesses to help to achieve modern businesses\u2019 goals and needs to be thought of throughout the software development lifecycle.<\/p>\n\n\n\n

Digital transformation and continuous delivery<\/h3>\n\n\n\n

The largest and most relevant companies today have embraced digital transformation and technology to run their businesses and meet their customers\u2019 needs. To keep up with digital transformation, you need modern software development practices like DevOps and continuous delivery. DevOps requires continuous testing to be successful, but we\u2019re seeing that reliance on manual testing is the biggest challenge that organizations face when adopting DevOps.<\/p>\n\n\n\n

Software testing and quality assurance<\/h3>\n\n\n\n

The software world is changing, so we need to change how we deliver technology. That requires modern software development approaches, which requires modern software testing and software quality approaches. Thankfully, testing and quality are far more top of mind now than in previous years.<\/p>\n\n\n

\n
\"Poll<\/figure><\/div>\n\n\n

From the audience poll results from our webinar, we see that continuous delivery is here to stay. When asked how often do you deploy changes to production<\/em>, over 40% of people stated that they deploy either daily or multiple times per day. This wasn\u2019t the case 10 to 20 years ago for most organizations unless technology was the product. Now, daily deployments are pretty common for most organizations.<\/p>\n\n\n\n

Low rates of test automation<\/h3>\n\n\n\n

However, we\u2019re seeing that getting high test automation coverage is still a huge challenge. In the survey, 55% of the respondents automate less than half of their testing.<\/p>\n\n\n

\n
\"Poll<\/figure><\/div>\n\n\n

The numbers may have a bit of sample bias because Applitools users actually automate on average 1.7 times higher than other respondents. The responses align with anecdotal experience that a lot of organizations are still in lower test automation coverage around 20% to 50%.<\/p>\n\n\n

\n
\"Poll<\/figure><\/div>\n\n\n

Testing complexity and the amount of testing needed are going up increasingly, and this shows in our survey as well. More than 50% of the respondents test two languages or more, three browsers or more, three devices or more, and two applications or more.<\/p>\n\n\n

\n
\"Poll<\/figure><\/div>\n\n\n

With two thirds of the respondents saying that one of the hardest parts of testing UIs is that they are constantly changing, traditional automation tools can’t handle testing at that speed scale and complexity.<\/p>\n\n\n\n

Using AI in testing<\/h3>\n\n\n\n

We know that there’s a lot of excitement around AI tools, and the survey shows that.<\/p>\n\n\n

\n
\"Poll<\/figure><\/div>\n\n\n

When asked What parts of the testing process are you supporting or planning on supporting with AI<\/em>, test offering, test prioritization, test execution, test management, and visual regression we all mentioned. The top two answers among respondents were test execution and visual regression.<\/p>\n\n\n\n

It\u2019s important to remember that continuous testing is about more than just test automation.<\/p>\n\n\n

\n
\"Poll<\/figure><\/div>\n\n\n

While test automation is key to the process, you still need to incorporate other software testing and software quality practices like load testing, security, user experience, and accessibility.<\/p>\n<\/div><\/div>\n\n\n\n

\n

The future of testing<\/h2>\n\n\n\n

What we\u2019re trying to achieve in the future of testing is to support modern software development practices. The best way we\u2019re seeing to do this is to have software testing and software quality more tightly integrated into the process. Let\u2019s talk about what a modern software approach looks like.<\/p>\n\n\n\n

Testing sooner and more frequently<\/h3>\n\n\n\n

To get tighter integration of quality into the process, testing can\u2019t just be an activity at the end of the development lifecycle. Testing has to happen continuously for teams to be able to provide fast feedback across disciplines and ensure a quality product. When this is done, we see increased speed not just of testing, but of overall software development and deployment. We also see reduced costs and increased quality.  More defects are found before production, and we see quicker responses when finding defects in production.<\/p>\n\n\n\n

Increasing how much testing is automated<\/h3>\n\n\n\n

Traditional testing approaches tend to be done mostly manually. Increasing test coverage doesn\u2019t mean manual testing goes away.<\/p>\n\n\n

\n
\"Graph<\/figure><\/div>\n\n\n

Automating your test cases frees up time to do more exploratory testing. Exploratory testing should be assisted by different tools, so AI has a good role to play here. Tools like ChatGPT are useful to brainstorm things like what to test next. Obviously we want to increase test automation coverage at all levels, including unit, API, and UI. Intelligent automated UI tests provide us more information than functional tests alone.<\/p>\n\n\n\n

Knowing when to use AI in testing<\/h3>\n\n\n\n

What does the future of testing with AI look like? It’s a combination of people, processes, and technology. Software testers need to be thinking about what skills we need to have to support these new ways of testing and delivering quality software.<\/p>\n\n\n\n

We need to uncover if a use case is better served by AI and machine learning than an algorithmic solution. To do this, we need to ask the following questions:<\/p>\n\n\n\n

    \n
  • What is the pain point that we are trying to solve?<\/li>\n\n\n\n
  • Are there good algorithmic or heuristic solutions available to address the pain point?<\/li>\n\n\n\n
  • Are there enterprise ready AI and machine learning solutions that help?\n
      \n
    • If there are solutions, what happens if the AI doesn\u2019t work?<\/li>\n\n\n\n
    • If there aren\u2019t solutions yet, how do we position ourselves to be ready for them?<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
      \n

      \u201cIt’s quite trendy to talk about artificial intelligence, but the reason why we’re partnered with Applitools is that they apply real machine learning and artificial intelligence to a problem that is not well solved by other types of solutions on the market.\u201d<\/p>\nChris Rolls<\/strong>, CEO, Americas, TTC<\/cite><\/blockquote>\n<\/div><\/div>\n\n\n\n

      \n

      Integrating visual AI into test automation<\/h2>\n\n\n\n

      Let’s talk about how we can integrate AI into our testing to get some of those advantages of increased speed and coverage discussed earlier.<\/p>\n\n\n\n

      What is visual AI?<\/h3>\n\n\n\n

      I like to explain visual AI visually. Do you remember those spot-the-difference pictures we had in our activity books from when we were kids?<\/p>\n\n\n

      \n
      \"Spot<\/figure><\/div>\n\n\n

      As humans, we could sit around and play with this to find the differences manually. But what we want is to be able to find these differences immediately. And that is what visual AI has the power to do.<\/p>\n\n\n

      \n
      \"Spot<\/figure><\/div>\n\n\n

      Even when the images are a little bit skewed or off by a couple pixels here or there, visual AI can pinpoint any differences between one view and another.<\/p>\n\n\n\n

      Now you might be thinking, Andy, that’s cute, but how’s this technology gonna help me in the real world? Is it just gonna solve little activity book problems? Well, think about all the apps that you would develop \u2013 whether web, mobile, desktop, whatever you have \u2013 and all the possible ways that you could have visual bugs in your apps.<\/p>\n\n\n\n

      Here we’ve got three different views from mobile apps. One for Chipotle, one for a bank, and another one for a healthcare provider.<\/p>\n\n\n\n

      \n
      \n
      \"Example<\/figure><\/div><\/div>\n\n\n\n
      \n
      \"Example<\/figure><\/div><\/div>\n\n\n\n
      \n
      \"Example<\/figure><\/div><\/div>\n<\/div>\n\n\n\n

      Visual AI test automation<\/h3>\n\n\n\n

      You can see that visual bugs are pervasive and they come in all different shapes and sizes. Sometimes the formatting is off, sometimes a particular word or phrase or title is just nulled out. What’s really pesky is that sometimes you might have overlapping text.<\/p>\n\n\n\n

      Traditional automation struggles to find these issues because traditional automation usually hinges on text content purely or on particular attributes of elements on a page. So as long as something appears and it’s enactable, most traditional scripts will pass. Even though we as humans visually can inspect and see when something is completely broken and unuseable.<\/p>\n\n\n\n

      This is where visual AI can help us, because what we can do is we can take snapshots of our app over time and use visual AI to detect when we have visual regressions. Because if, let’s say, one day the title went from being your bank’s name to null, it’ll pick it up right away in your continuous testing.<\/p>\n\n\n\n

      Visual AI in action<\/h3>\n\n\n\n

      In the webinar, I gave a live demo of automated visual testing using Applitools Eyes. In case you missed the demo, you can check it out here:<\/p>\n\n\n\n

      \n