Update & TL;DR

This post was written while Forrester’s research on agentic and autonomous testing was still emerging. Since publication, Applitools has been included in The Forrester Wave: Autonomous Testing Platforms, Q4 2025. The perspective outlined below reflects how this shift has since been validated and formalized by independent industry analysts.

• Agentic automation shifts testing from brittle, script-driven execution to intelligent systems that adapt based on change, risk, and context.
• AI augments human intent rather than replacing QA teams, enabling people to focus on quality strategy, governance, and risk decisions.
• This model is increasingly shaping how autonomous testing platforms are evaluated in the market.

Forrester, a leading global research and advisory firm, identified a major turning point in software testing in its Autonomous Testing Platforms Landscape, Q3 2025. The research describes a shift from traditional scripted automation to AI-augmented systems that can learn, adapt, and act under human guidance. This shift signals the rise of agentic automation: intelligent systems that create, run, and optimize tests within defined boundaries.

As delivery cycles compress and complexity grows, quality and engineering leaders are redefining what effective testing means in practice. Agentic automation bridges human intent with machine-driven precision—transforming testing from a reactive maintenance task into a proactive engine for reliability, speed, and continuous improvement.

From Automation to Intelligence

Traditional automation accelerated execution but left teams managing brittle scripts and endless maintenance. AI-augmented testing changes that dynamic. These systems:

• Learn continuously from results and application change.
• Adapt test scope and prioritization based on business risk.
• Optimize coverage while maintaining human oversight.

The result is testing that behaves less like a checklist and more like a self-improving quality partner, one that scales reliability across every release.

The Three Business Values Driving This Shift

Forrester highlights three outcomes motivating investment in more intelligent testing systems:

1. Accelerate Time to Value – AI-driven generation and self-healing shorten feedback loops and reduce maintenance.
2. Reduce Strategic Risk – Risk-based orchestration and built-in governance connect quality metrics directly to business priorities.
3. Democratize Testing – Low-code authoring and natural-language interaction let non-developers participate in quality, closing skill gaps.

Agentic automation brings these together: human-directed intent, machine-driven efficiency, and transparent oversight.

How AI-Augmented Systems Complement Human Expertise

AI in testing works best as augmentation, not replacement. By handling repetitive execution and maintenance, intelligent systems free QA professionals to focus on:

• Defining risk and coverage strategy.
• Establishing governance frameworks that maintain trust.
• Collaborating earlier with product and development teams.

Agentic automation shifts QA leadership from running tests to steering quality outcomes.

The Role of Visual and Experience Validation

Intelligent automation depends on reliable validation signals. Traditional assertions can’t always capture what matters to real users: layout, accessibility, and experience consistency. 

Visual and experience validation fill that gap, giving AI-augmented systems context they can trust. When machines validate what users actually experience, teams gain both speed and confidence—without rigid pixel-level comparison.

Building Toward AI-Augmented Readiness

Forrester describes this as a maturing market: organizations are blending traditional automation with AI capabilities to move toward greater autonomy over time. QA leaders can start by:

1. Stabilizing automation foundations and addressing flakiness.
2. Adopting AI-assisted detection of UI and data changes.
3. Integrating experience-level validation for richer feedback.
4. Connecting quality analytics to business metrics for continuous improvement.

Each step builds the trust and data maturity required for agentic automation to succeed under human orchestration. As adoption increases, these maturity steps align with how leaders in the market are being evaluated on autonomous capabilities.

What QA Leaders Can Do Next

Forward-looking teams are already experimenting with:

• Adaptive execution that prioritizes tests dynamically.
• Governance dashboards linking coverage, risk, and compliance.
• Visual AI that helps systems understand real user impact.

The goal isn’t full autonomy—it’s AI-augmented confidence: testing that’s faster, smarter, and more inclusive across roles. Read the full report now.

The post Agentic Automation: Preparing QA Leaders for the Next Leap in Testing appeared first on AI-Powered End-to-End Testing | Applitools.

AI is transforming software testing—but without clear context, even the smartest models can fall short. The new Model Context Protocol (MCP) aims to solve that problem, and it’s picking up momentum fast. Here’s what QA and development teams need to know—and why it matters right now. If you have questions about how we’re building for the future or how this fits into your testing strategy, let us know—we’d love to talk.

What Is MCP?

MCP, or Model Context Protocol, is an open standard designed to help applications provide AI models with structured context. Think of it as a standardized way for tools and systems to tell an AI assistant what’s going on—who the user is, what they’re doing, and what resources are available.

Anthropic introduced MCP in late 2024, and it’s already being adopted by major players like OpenAI, Microsoft, and testing leaders building next-generation AI workflows. Addy Osmani, an engineering leader at Google, calls MCP “the USB-C of AI integrations,” highlighting its potential to standardize the connection between tools and intelligent agents.

Why Context Matters in AI-Assisted Testing

Large language models (LLMs) are only as good as the context they receive. Without proper inputs, you get generic outputs—or worse, hallucinations. For QA teams using AI to generate tests, interpret failures, or automate user flows, missing context leads to fragile results and wasted time.

MCP helps solve this by passing structured information to the model: which test framework is in use, what files are open, what code just changed, and more. That means faster, more relevant AI assistance—and more accurate automation.

What MCP Enables in Testing Workflows

MCP makes it easier for tools and AI assistants to share structured context—like which framework is active, what code changed, or what the user is trying to do. That unlocks more accurate test generation, better debugging, and scalable, reusable automation.

It also supports dynamic discovery, so AI systems can find and connect with available tools at runtime—no brittle configs or manual setup required.

As testers ourselves, we take a measured approach to adopting new AI standards like MCP. That means vetting integrations for stability and reliability, so our customers can move fast without sacrificing trust.

Why It’s a Big Deal Now

There are two key reasons to pay attention to MCP today:

First, the standard is taking off. Thought leaders like Angie Jones, Filip Hric, Tariq King, and Addy Osmani are publishing real-world MCP demos and contributing open-source tools. It’s not theoretical anymore—it’s happening.

Second, the stakes are high. As more testing platforms integrate AI (including Applitools Autonomous), the ability to connect tools through open standards like MCP is becoming a competitive differentiator.

How Applitools Fits In

Applitools has long focused on intelligent automation—delivering AI-powered test creation, visual validation, and self-healing across platforms. As open standards like MCP emerge, we’re building on that foundation to extend context-sharing across tools, so teams can:

• Automatically create or update visual and functional tests based on code changes
• Route test context through the pipeline for faster root cause analysis
• Improve AI-generated tests with better accuracy and explainability
  

Security is also critical. As MCP evolves, host-mediated permissions and encrypted communication protocols are being considered by contributors to ensure context is shared safely and responsibly.

At Applitools, we’re building these principles directly into the future of Autonomous and Eyes—and we’d love to walk you through what’s on our roadmap. If you’re already an Applitools customer, reach out to your account team to schedule a preview conversation. If you’re not already using Applitools, schedule time with one of our testing specialists—we’re here to help.

The post MCP: What It Is and Why It Matters for AI in Software Testing appeared first on AI-Powered End-to-End Testing | Applitools.

Efficient software testing plays a crucial role in the digital financial industry, where customer trust relies on reliable systems and smooth user experiences. AI-powered tools are revolutionizing software testing for online banking. Learn how Applitools’ Autonomous Testing Platform, an AI-driven testing solution, can boost efficiency and ensure seamless functionality for digital banking platforms.

AI-Powered Testing for Visual Verification

Modern interfaces are becoming more dynamic, making software testing more challenging than ever. Applitools uses AI-powered end-to-end testing to deliver accurate, human-like visual validation. With this advanced approach, teams can easily detect visual bugs, handle dynamic content, and solve multi-device rendering issues. Compared to traditional testing tools, Applitools’ AI testing solution offers a faster, more reliable way to ensure seamless user experiences across all platforms.

Visual AI “sees” applications as a human would, identifying not just design inconsistencies, but functional issues within complex environments. This human-eye accuracy is especially critical for financial applications, where misdisplayed data or minor UI errors can compromise user trust.

Visual Verification with AI gives QA teams:

• Precise identification of visual bugs, such as misaligned content or missing elements.
• Compatibility across a range of devices, screen sizes, and browsers.
• Reduced false positives, ensuring only actionable defects are flagged.

Challenges in Automated Testing for Financial Apps

Some of the key challenges QA teams face in the financial services industry include testing data-heavy dashboards, personalized user experiences, and meeting strict compliance requirements. QA teams also navigate the complexity of testing constantly changing data, such as account balances and transactions, while ensuring seamless functionality across multiple devices and screen sizes. These unique testing challenges highlight the importance of effective QA processes in delivering reliable, user-friendly financial services.

Adding to these complexities are regulatory requirements, like accessibility compliance mandated by guidelines such as the European Accessibility Act. Traditional tools often fall short in dynamically adapting to such changes, leading to increased bottlenecks.

Common Obstacles Faced by QA Teams:

• Exponential growth in test scenarios due to dynamic UI states.
• Maintaining coverage amid rapid deployment cycles.
• Lacking tools to validate compliance and localization needs.

The Role of Visual AI and Autonomous in Testing

Applitools’ Visual AI is transforming the way teams approach UI testing. Developed with 11 years of research and development, this powerful technology compares both visual elements and DOM structure to deliver comprehensive UI validation. Key features like automated baselining and self-healing tests make maintaining tests easier, especially during frequent code updates or UI changes.

The Autonomous platform takes Visual AI testing to the next level by integrating it with end-to-end test workflows. The platform simplifies visual, functional, API, and accessibility testing. With Autonomous, teams can automate testing, schedule tests, validate results, and efficiently manage issues—all from a user-friendly interface.

Features of Visual AI & Testing with Autonomous:

• Self-healing capabilities to adapt tests to UI updates without manual adjustments.
• Natural language authoring that allows easy collaboration between technical and non-technical team members.
• Supports functional flows like login processes while ensuring personalized data fields meet validation patterns.

Efficiency and Coverage Improvements

A top US bank improved testing efficacy using Applitools Autonomous. By adopting Visual AI and scaling to test its entire ecosystem, the bank achieved:

• 5x Test Coverage Expansion across all devices and browsers.
• 35% More Defects Caught weekly within earlier production stages.
• Up to 999 Hours Saved Per Release by reducing manual test generation and maintenance efforts.

This expansive coverage allowed the team to uncover bugs that would otherwise have been missed, delivering higher-quality software quicker.

Visit the event archive to see more of the case study and watch the full session on-demand.

Security and Integration Features

Data security and adaptiveness in distinct environments are primary concerns in financial testing. The webinar addressed how Applitools ensures client protection with private cloud implementations and a secure architecture for test data management. The platform seamlessly integrates with firewalls and various development ecosystems, making it an ideal choice regardless of organizational infrastructure.

Ensuring Data Privacy and Flexibility:

• Test data secured using encryption and private cloud options.
• Compatible with diverse environments, both behind firewalls and in SaaS architectures.
• Allows scalability without compromising compliance adherence.

Strengthen Financial Application Testing

AI-assisted testing is helping overcome the hurdles posed by traditional testing. By integrating Visual AI with a broad end-to-end testing workflow, financial institutions can drastically enhance their QA practices. More importantly, these improvements do not require trade-offs between coverage, speed, and accuracy.

If you’re ready to elevate your testing game, particularly in high-demand sectors like online banking, platforms like Autonomous present an opportunity you can’t afford to miss.

Explore the full webinar on-demand or reach out to learn how Applitools Autonomous can streamline your QA efforts and ensure the flawless delivery of digital experiences.

The post End-to-End Testing Solutions for Online Banking Software Applications appeared first on AI-Powered End-to-End Testing | Applitools.

Forrester’s August 2023 report: The State Of TuringBots, 2023 examines the impact of AI on the Software Delivery Lifecycle (SDLC) and how organizations can effectively add this technology to their overall strategy. The findings were compelling and critical for modern businesses to understand so that they can adapt and stay ahead of the curve. Let’s take a look at some of the key points. 

Per the report, many organizations are grappling with the challenge of keeping up with business changes due to sluggish software development processes. The emergence of Generative AI has ushered in a new era of AI-assistive software, impacting industries irrespective of their SDLC maturity or existing AI utilization.

Forrester coined the name for this software as “TuringBots” which they now define as:

“AI-powered software that augments application development and infrastructure and operations (I&O) teams’ automation and semiautonomous capabilities to plan, analyze, design, code, test, deliver, and deploy while providing assistive intelligence on code, development processes, and applications.”

Forward-thinking organizations are embracing cutting-edge technologies like TuringBots to stay ahead. With GenAI revolutionizing numerous AI applications, the anticipated timeline for the development and impact of TuringBots has been accelerated to two to five years instead of a decade. This shift has brought about a deeper comprehension of the immense potential held by TuringBots.

Today, businesses rely on software as the backbone of digital operations, representing their strategies, processes, products, and services. However, many organizations need help in software development to match the rapid pace of business evolution and innovation. The report laid out common challenges like:

• Many developers still rely on manual testing despite automation advancements in the software development lifecycle. The lack of automation in various stages is attributed to tool complexity, skill gaps, and slow organizational modernization adoption.
• Lacking product management skills. The main hurdle in agile proficiency is the absence of business-led product ownership and management.
• IT that is resistant to change.

Per the report, academia and the tech industry have long aimed to streamline software development. With GenAI’s TuringBots, Forrester states that vendors accelerate product delivery by automating tasks and enhancing user experiences. The report recommends that it is time for tech leaders to empower their teams with TuringBots for maximum efficiency and take advantage of these benefits:

• TuringBots assist in development processes, though not fully mature for complete SDLC support.
• Easy access to project information is crucial for teams, covering project status, test completion, code check-ins, and more. Developers can save time by using Coder TuringBot plug-ins in IDEs like Tabnine or GitHub Copilot.
• These tools provide quick access to code snippets and information, helping to generate code efficiently through natural language chat.

The impact of TuringBots is significant across all industries, driving the transformation into digitally competent businesses. While the speed of adaptation varies, understanding and managing TuringBots is crucial for technology leaders. It is essential to swiftly grasp the key governance and best practices to mitigate risks related to insecure code, performance issues, and user experience.

The post Forrester Report Recap: The Future of Software Development appeared first on AI-Powered End-to-End Testing | Applitools.

What exactly does “platform” mean in today’s software world? At Applitools, it means more than just a tool – it’s a comprehensive solution enabling you to test like the best in the business. In the world of test automation, having a platform that can take your practice to the next level is crucial.

With more than a decade of experience serving top engineering teams and leading companies worldwide, we have taken a fresh approach to developing our platform. Our journey started with building the world’s first and best visual test automation solution, and we didn’t stop there. We continuously leveraged our insights to create more products that address the evolving needs of the industry. Now, companies using our platform can achieve higher quality and faster results than ever before, empowering developers to work smarter and push the boundaries of their test automation practice.

During our recent webinar, we unveiled our Intelligent Testing Platform and shared some key highlights, including:

• Easy one-click setup – Just one Autonomous at your website, and you are done. Everything you need is available out of the box.
• Automatic website/app discovery – Automatically create self-adjusting test suites that detect new, missing, changed, or faulty pages and components on every run.
• Natural language test builder – Describe complex end-to-end flows using nothing more than plain English. No coding or element-locating skills are required.
• Cross-device and browser testing – Test your public and internal apps on any device, browser, and OS using the world’s most modern test infrastructure available out of the box.
• Flexible test orchestration – Run tests on demand from your CI/CD or a webhook, or use our built-in test scheduler. No DevOps skills required.
• AI-assisted test maintenance – Self-heal broken locators, avoid repetitive maintenance activities, and group similar UI changes and issues together.

Ready to see more? Join Effortless Testing with Applitools Autonomous: A Hands-On Webinar. Dave Piacente will showcase more intricate use cases that demonstrate not just the platform’s technical prowess but its ability to transform your testing landscape with plain English.

With the Applitools Intelligent Testing Platform, you can reduce risk, enhance delivery velocity, and provide superior digital experiences that consistently exceed consumer expectations. It’s the ultimate tool to stay ahead in today’s ever-evolving landscape. The days of doing things the same old way are gone. It’s time to shake things up, test smarter, and embrace a new era of test automation.

Watch the webinar on-demand here, and join us as we continue to pave the way for the future of test automation!

The post Recap: A Test Automation Platform Designed for the Future appeared first on AI-Powered End-to-End Testing | Applitools.

In the ever-accelerating digital product landscape, the speed of development and deployment has become a critical factor for success. As businesses push for faster time-to-market, traditional QA and testing methodologies have increasingly become bottlenecks, unable to keep pace with the rapid development cycles. Enter Applitools Autonomous, a groundbreaking solution designed to transform the QA process and ensure that businesses can deliver flawless digital experiences faster and more efficiently than ever before.

Why Applitools Autonomous Matters

Accelerated Development, Uncompromised Quality

In the current competitive digital environment, the ability to quickly launch new products and features is a significant advantage. However, the necessity for thorough testing has traditionally slowed this process, creating a tension between the need for speed and the demand for quality. Applitools Autonomous addresses this issue head-on by leveraging AI to automate test creation, execution, maintenance, and reporting, significantly reducing the time and resources required for comprehensive testing.

Frontend Excellence as a Differentiator

Today’s consumers expect not just functionality but excellence in design and user experience. Visual defects or poor UI/UX can severely damage a brand’s reputation. Applitools Autonomous enhances collaboration among designers, developers, and product teams, enabling the seamless integration of tools like Figma and Storybook to elevate frontend experiences and ensure they meet the highest standards of quality and design.

The Problem with Traditional Testing

Businesses face significant challenges in ensuring their web applications perform correctly across various screens and devices. The dynamic nature of web content, frequent updates, and the vast array of devices make comprehensive testing a daunting task. Traditional testing tools, designed for a less complex web environment, fall short in providing the necessary coverage and efficiency, leading to bugs slipping into production, reduced brand integrity, and slow testing cycles.

The Solution: Applitools Autonomous

Applitools Autonomous revolutionizes QA by replicating the intelligence and accuracy of the best QA practitioners at scale. It automates the entire testing process, from test creation to maintenance, using AI. This AI-driven approach allows teams to generate test cases with a single click, create end-to-end tests in plain English, and utilize Visual AI to increase test coverage while reducing maintenance efforts. By integrating seamlessly into CI/CD pipelines, Autonomous enables continuous testing and monitoring, ensuring that any changes or new bugs are detected and addressed promptly.

Key Features of Applitools Autonomous

• Generative Testing: Automatically creates test cases for your site, improving test coverage instantly.
• Natural Language Test Builder: Allows for the creation of robust tests using plain English, making QA accessible to more teams.
• Contextual UI Testing: Enhances test reliability by leveraging contextual and semantic cues from the UI.
• Visual AI: Validates thousands of UI elements instantly, improving test coverage and reducing manual testing efforts.
• Intelligent Test Infrastructure: Features self-healing tests that adapt to UI changes, ensuring continuous operation.
• Flexible Execution: Supports on-demand testing, scheduled tests, and integration with CI/CD pipelines.

Ideal Customer Profile

Applitools Autonomous is particularly beneficial for large websites and applications that are content-rich or frequently updated. This includes e-commerce platforms, media and publishing houses, educational institutions, financial institutions, travel and hospitality companies, healthcare providers, and government and NGO websites. These organizations face unique challenges in maintaining quality and functionality due to the dynamic nature of their digital content, making Autonomous an ideal solution.

Transform Your QA with Applitools Autonomous

Applitools Autonomous is not just a tool; it’s a paradigm shift in digital quality assurance. By automating the testing process and leveraging AI, businesses can now ensure their digital experiences are flawless, without the traditional bottlenecks of QA. Embrace the future of testing with Applitools Autonomous and deliver superior digital products with confidence and speed.

The post The Rise of Generative QA appeared first on AI-Powered End-to-End Testing | Applitools.

We are thrilled to announce the launch of the Applitools Intelligent Testing Platform, a groundbreaking advancement in AI-powered test automation. As we step into a new era of quality assurance, Applitools is leading the charge with innovative solutions designed to revolutionize the way businesses approach testing across applications and documents. With the introduction of three powerful solutions—Autonomous, Eyes, and Preflight—our platform is redefining industry standards for flexibility, coverage, and ease of use.

Why Applitools? 

In the fast-paced world of digital innovation, ensuring the quality of web apps, mobile apps, and documents is mission-critical. Traditional testing tools led teams down a path of unsustainable quality. Where each unit of development required a unit or more of testing to validate the change. Applitools works differently at scale and makes validating your digital products remarkably intuitive and efficient, catering to a diverse range of testing requirements. Whether you’re a seasoned coder or someone with minimal testing experience, the platform empowers every team member to contribute to the quality assurance process.

Key Features of the Applitools Platform:

Dynamic Test Authoring: Say goodbye to the tedious aspects of test creation. Applitools allows for dynamic authoring of tests with AI, a codeless recorder, or your favorite framework. Integrate with popular tools like Selenium and Cypress to enable comprehensive ‘shift left’ testing directly from development.

Comprehensive Validation: With Visual AI, you can ensure your user interface works impeccably and looks exactly as intended. From functional and visual validation to accessibility and cross-browser testing, we cover every aspect to guarantee a seamless user experience.

Scalable Execution: Run your tests at an unprecedented scale with our cloud testing capabilities. Applitools’ self-healing locators and selectors correct tests on the fly, reducing maintenance and ensuring your tests evolve with your application.

Advanced Analysis & Maintenance: Dive deep into test analysis with automated grouping, root cause analysis, and powerful dashboards. Our predictive analytics help you stay one step ahead, ensuring that your testing strategy is as dynamic and innovative as your products.

Empowering Every Team Member 

One of the most significant advantages of the Applitools platform is its accessibility to a broad range of personas at your company. By reducing the reliance on coding expertise and offering various test creation and execution methods, we’re democratizing quality assurance. Now, everyone from QA professionals to digital marketers can efficiently build and maintain tests, contributing to a high-quality, reliable product.

By enabling product experts and other “users” of the application interface, we hope for tests to be more comprehensive, thoughtful, and robust. 

For the Future of Your Business

In the competitive landscape of digital products, the balance between speed, quality, and innovation is crucial. The Intelligent Testing Platform is more than a tool—it’s a strategic asset. For engineering and product teams, Applitools means reduced risk, improved delivery velocity, and superior digital experiences that align with consumer expectations and business goals.

As we launch the Applitools Intelligent Testing Platform, we invite you to join us in embracing the future of testing. With our commitment to innovation, community, and quality, we’re excited to partner with you in delivering excellence and driving success in your digital endeavors. Welcome to a new standard of testing—welcome to Applitools.

The post Introducing The Intelligent Testing Platform appeared first on AI-Powered End-to-End Testing | Applitools.

At the recent Future of Testing: AI in Automation event hosted by Applitools, I ran a session called ‘The Fear Factor’ where we safely and openly discussed some of our fears around the use of AI in test automation. At this event, we heard from many thought leaders and experts in this domain who shared their experiences and visions for the future. AI in test automation is already here, and its presence in test automation tooling will only increase in the very near future, but should we fear it or embrace it?

During my session, I asked the attendees three questions:

• Do you have any fears about the use of AI in testing?
• In one word, describe your feelings when you think about AI and testing.
• If you do have fears about the use of AI in testing, describe them.

Do you have any fears about the use of AI in testing?

Where do you sit?

I’m in the Yes camp, and let me try to explain why.

Fear can mean many things, but one of them is the threat of harm. It’s that which concerns me in the software testing space. But that harm will only happen if teams/companies believe that AI alone can do a good enough job. If we start to see companies blindly trusting AI tools for all their testing efforts, I believe we’ll see many critical issues in production. It’s not that I don’t believe AI is capable of doing great testing—it’s more the fact that many testers struggle to explain their testing, so to have good enough data to train such a model feels distant to me. Of course, not all testing is equal, and I fully expect to see many AI-based tools doing some of the low-hanging fruit testing for us.

In one word, describe your feelings when you think about AI and testing.

It’s hard to disagree with the results from this question—if I were to pick two myself, I would have gone with ‘excited and skeptical.’ I’m excited because we seem to be seeing new developments and tools each week. On top of that, though, we are starting to see developments in tooling using AI outside of the traditional automation space, and that really pleases me. Combine that with the developments we are seeing in the automation space, such as autonomous testing, and the future tooling for testing looks rather exciting.

That said, though, I’m a tester, so I’m skeptical of most things. I’ve seen several testing tools now that are making some big promises around the use of AI, and unfortunately, several that are talking about replacing or needing fewer testers. I’m very skeptical of such claims. If we pause and look across the whole of the technology industry, the most impactful use of AI thus far is in assisting people. Various GPTs help generate all sorts of artifacts, such as code, copy, and images. Sometimes, it’s good enough, but the majority of the time is helping a human be more efficient—this use of AI and such messaging, excites me.

If you do have fears about the use of AI in testing, describe them here.

We got lots of responses to this question, but I’m going to summarise and elaborate on four of them:

• Job security
• Learning curve
• Reliability & security
• How it looks

Job Security

Several attendees shared they were concerned about AI replacing their jobs. Personally, I can’t see this happening. We had the same concern with test automation, and that never really materialized. Those automated tests don’t maintain themselves, or write themselves, or share the results themselves. The direction shared by Angie Jones in her talk Where Is My Flying Car?! Test Automation in the Space Age, and Tariq King in his talk, Automating Quality: A Vision Beyond AI for Testing, is AI that assists the human, giving them superpowers. That’s the future I hope, and believe we’ll see, where we are able to do our testing a lot more efficiently by having AI assist us. Hopefully, this means we can release even quicker, with higher quality for our customers.

Another concern shared was about skills that we’ve spent years and a lot of effort learning, suddenly being replaced by AI. Or significantly easier with AI. I think this is a valid concern but also inevitable. We’ve already seen AI have a significant benefit to developers with tools like GitHub Copilot. However, I’ve got a lot of experience with Copilot, and it only really helps when you know what to ask for—this is the same with GPTs. Therefore, I think the core skills of a tester will be crucial, and I can’t see AI replacing those.

Learning Curve

If we are going to be adding all these fantastic AI tools into our tool belts, I feel it’s going to be important we all have a basic understanding of AI. This concern was shared by the attendees. For me, if I’m going to be trusting a tool to do testing for me or generating test artefacts for me, I definitely want that basic understanding. So, that poses the question, where are we going to get this knowledge from?

On the flip side of this, what if we become over-reliant on these new AI tools? A concern shared by attendees was that the next generation of testers might not have some of the core skills we consider important today. Testers are known for being excellent thinkers and practitioners of critical thinking. If the AI tools are doing all this thinking for us, we run the risk of those skills losing their focus and no longer being taught. This could lead to us being over-reliant on such tools, but also the tools biassing the testing that we do. But given that the community is focusing on this already, I feel it’s something we can plan to mitigate and ensure this doesn’t happen.

Reliability & Security

Data, data, data. A lot of fears were shared over the use and collection of data. The majority of us work on applications where data, security, and integrity are critical. I absolutely share this concern. I’m no AI expert, but the best AI tools I’ve used thus far are ones that are contextual to my domain/application, and to do that, we need to train it on our data. These could lead to data bleeding and private data, and that is a huge challenge I think the AI space has yet to solve.

One of the huge benefits of AI tooling is that it’s always learning and, hopefully, improving. But that brings a new challenge to testing. Usually, when we create an automated test, we are codifying knowledge and behavior, to create something that is deterministic, we want it to do the same thing over and over again. This provides consistent feedback. However, with an AI-based tool it won’t always do the same thing over and over again—it will try and apply its intelligence, and here’s where the reliability issues come in. What it tested last week may not be the same this week, but it may give us the same indicator. This, for me, emphasizes the importance of basic AI knowledge but also that we use these tools as an assistant to our human skills and judgment.

How It Looks

Several attendees shared concerns about how these AI tools are going to look. Are they going to a completely black box, where we enter a URL or upload an app and just click Go? Then the tool will tell us pass or fail, or perhaps it will just go and log the bugs for us. I don’t think so. As per Angie’s and Tariq’s talk I mentioned before, I think it’s more likely these tools will focus on assistance. 

These tools will be incredibly powerful and capable of doing a lot of testing very quickly. However, what they’ll struggle to do is to put all the information they find into context. That’s why I like the idea of assistance, a bunch of AI robots going off and collecting information for me. It’s then up to me to process all that information and put it into the context of the product. The best AI tool is going to be the one that makes it as easy as possible to process the masses of information these tools are going to return.

Imagine you point an AI bot at your website, and within minutes, it’s reporting accessibility issues to you, performance issues, broken links, broken buttons, layout issues, and much more. It’s going to be imperative that we can process that information as quickly as possible to ensure these tools continue to support us and don’t drown us in information.

Visit the Future of Testing: AI in Automation archive

In summary, AI is here, and more is coming. It’s very exciting times in the software testing tooling space, and I’m really looking forward to playing with more new tools. I think we need to be curious with these new tools, try them, and see what sticks. The more tools we have in our tool belts, the more options we have to solve our ever-increasing complex testing challenges. 

The post Should We Fear AI in Test Automation? appeared first on AI-Powered End-to-End Testing | Applitools.

The latest edition of the Future of Testing events, held on November 7, 2023, was nothing short of inspiring and thought-provoking! Focused on AI in Automation, attendees learned how to leverage AI in software testing with top industry leaders like Angie Jones, Tariq King, Simon Stewart, and many more. All of the sessions are available now on-demand, and below, we take a look back at these groundbreaking sessions to give you a sneak peek of what to expect before you watch.

Opening Remarks

Joe Colantonio from TestGuild and Dave Piacente from Applitools set the stage for a thought-provoking discussion on reimagining test automation with AI. As technology continues to evolve at a rapid pace, it’s important for software testing professionals to adapt and embrace new tools and techniques. Joe and Dave encouraged attendees to explore the potential of AI in test automation and how it can enhance their current processes. They also touch upon the challenges faced by traditional test automation methods and how AI-powered solutions can help overcome them.

Dave shared one of our latest updates – the integration of Applitools Eyes with Preflight! Learn more about Preflight.

Keynote—Reimagining Test Automation with AI by Anand Bagmar

In this opening session, Anand Bagmar explored how to reimagine your test automation strategies with AI at each stage of the test automation life cycle, including a live demo showcasing the power of AI in test automation with Applitools.

Anand first introduced the transition from Waterfall to Agile software delivery practices, and while we can’t imagine going back to a Waterfall way of working, he addressed the challenges Agile brings to the software testing life cycle. Each iteration brings more room for error across analysis, maintenance, and validation of tests. This is why testers should turn toward AI-powered test automation, with the help of tools like Applitools, to help ease the pain of Agile testing.

The session is aimed at helping testers understand the importance of leveraging AI technology for successful test automation, as well as empowering them to become more effective in their roles. Watch now.

From Technical Debt to Technical Capital by Denali Lumma

In this session, Denali Lumma from Modular dived into the concept of technical debt and proposed a new perspective on how we view it – technical capital. She walked attendees through key mathematical concepts that help calculate technical capital, as well as examples comparing Pytorch vs. TensorFlow, MySQL vs.Postgres, Frameworks vs. Code Editors, and more.

Attendees gained insights into calculating technical capital and how it can impact the valuation of a company. Watch now.

Automating Quality: A Vision Beyond AI for Testing by Tariq King

Tariq King of EPAM Systems took attendees on a journey through the evolution of software testing and how it has been impacted by generative AI. He shared his vision for the future of automated quality, one that looks beyond just AI to also prioritize creativity and experimentation. Tariq emphasized the need for quality and not just using AI to “go faster.” The more quality you have, the more productive you will be.

Tariq also dove into the ethical implications of using AI for testing and how it can be used for good or evil. Watch the full session.

Leveraging ChatGPT with Cypress for API Testing: Hands-On Techniques by Anna Patterson

In this session, Anna Patterson of EVERFI explored practical techniques and provided hands-on examples of how to harness the combined power of Cypress and ChatGPT to create robust API tests for your applications.

Anna guided us through writing descriptive and clear test prompts using HTML status codes, with a pet store website as an example. She showed in real-time how meaningful prompts in ChatGPT can help you create a solid API test suite, while also considering the security requirements of your company. Watch now.

PANEL—Testing in the AI Era: Opportunities, Hurdles, and the Evolving Role of Engineers

Joe Colantonio, Test Guild • Janna Loeffler, mParticle • Dave Piacente, Applitools • Stephen Williams, Accenture

As the use of AI in software development continues to grow, it is important for engineers and testers to stay ahead of the curve. In this panel discussion led by Joe Colantonio from Test Guild, Janna Loeffler from mParticle, Dave Piacente from Applitools, and Stephen Williams from Accenture came together to discuss the current state of AI implementation and its impact on testing.

They talked about how AI is still in its early stages of adoption and why there may always be some level of distrust in AI technology. The panel emphasized the importance of first understanding why you might implement AI in your testing strategy so that you can determine what the technology will help to solve vs. jumping in right away. Many more incredible takes and insights were shared in this interactive session! Watch now.

The Fear Factor with Richard Bradshaw

The Friendly Tester, Richard Bradshaw, addressed the common fears about AI and automation in testing. Attendees heard Richard’s open and honest discussion on the challenges and concerns surrounding AI and automation in testing. Ultimately, he calmed many fears around AI and gave attendees a better understanding of how they can begin to use it in their organization and to their own advantage. Watch now.

Tests Too Slow? Rethink CI! by Simon Stewart

Simon Stewart from the Selenium Project discussed the latest updates on how to speed up your testing process and improve the reliability of your CI runs. He shared insights into the challenges and tradeoffs involved in this process, as well as what is to come with Selenium and Bazel.
Attendees learned how to rethink their CI approach and use these tools to get faster feedback and more reliable testing results. Watch now.

Revolutionizing Testing: Empowering Manual Testers with AI-Driven Automation by Dmitry Vinnik

Dmitry Vinnik explored how AI-driven automation is revolutionizing the testing process for manual testers. He showed how Applitools’ Visual AI and Preflight help streamline test maintenance and reduce the need for coding.

Dmitry shared the importance of test maintenance, no code solutions for AI testing, and a first-hand look at Applitools Preflight. Watch this session to better understand how AI is transforming testing and empowering manual testers to become more effective in their roles. Watch the full session.

Keynote—Where Is My Flying Car?! Test Automation in the Space Age by Angie Jones

In her closing keynote, Angie Jones of Block took us on a trip into the future to see how science fiction has influenced the technology we have today. The Jetsons predicted many futuristic inventions such as robots, holograms, 3D printing, smart devices, and drones. We will explore these predictions and see how far we have come regarding automation and technology in the testing space.

As technology continues to evolve, it is important for testers to stay updated and adapt their strategies accordingly. Angie dove into the exciting world of tech innovation and imagined the future for test automation in the space age. Watch now.

Visit the full Future of Testing: AI in Automation on-demand archive to watch now and learn actionable steps to implement AI in your software testing strategy, key considerations before you start, other ideas around ethics and philosophical considerations, the importance of quality and security, and much more.

The post Future of Testing: AI in Automation Recap appeared first on AI-Powered End-to-End Testing | Applitools.

We have a lot of great webinars and virtual events here at Applitools. I’m hoping posts like this give you a high-level summary of the key points with plenty of room for you to form your own impressions.

Dave Piacente

Curious if the software robots are here to take our jobs? Or maybe you’re not a fan of the AI hype train? During a recent session, The Future of AI-Based Test Automation, CTO Adam Carmi discussed—in practical terms—the current and future state of AI-based test automation, why it matters, and what you can do today to level up your automation practice.

• He describes how AI can be used to overcome common everyday challenges in end-to-end test automation, how the need for skilled testers will only increase, and how AI-based tooling can help supercharge any automated testing practice.
• He also puts his money where his mouth is by demonstrating how the neverending maintenance overhead of tests can be mitigated using AI-driven tooling which already exists today using concrete examples (e.g., visual validation and self-healing locators).
• He also discusses the role that AI will play in the future, including the development of autonomous testing platforms. These platforms will be able to automatically explore applications, add validations, and fill gaps in test coverage. (Spoiler alert: Applitools is building one, and Adam shows a bit of a teaser for it using a real-time in-browser REPL to automate the browser which uses natural language similar to ChatGPT.)

You can watch the full recording and find the session materials here, and I’ve included a quick breakdown with timestamps for ease of reference.

• Challenges with automating end-to-end tests using traditional approaches (02:34-10:22)
• How AI can be used to overcome these challenges (10:23-44:56)
• The role of AI in the future of test automation (e.g., autonomous testing) (44:57-58:56)
• The role of testers in the future (58:57-1:01:47)
• Q&A session with the speaker (1:01:48-1:12:30)

Want to see more? Don’t miss Future of Testing: AI in Automation.

The post AI and The Future of Test Automation with Adam Carmi | A Dave-reloper’s Take appeared first on AI-Powered End-to-End Testing | Applitools.

The post Unlocking the Power of ChatGPT and AI in Test Automation Q&A appeared first on AI-Powered End-to-End Testing | Applitools.

The post Unlocking the Power of ChatGPT and AI in Test Automation Key Takeaways appeared first on AI-Powered End-to-End Testing | Applitools.