TL;DR

• Code-centric automation continues to slow teams down as UI changes multiply, making stability and evidence hard to maintain.
• AI code generators don’t solve the problem because they still produce brittle test code that requires constant oversight.
• Live LLM-driven execution introduces unpredictability. Regulated teams need deterministic runs, not improvisation
• A clearer path is intent-driven authoring paired with deterministic engines and Visual AI that detects visual drift and preserves audit-ready evidence.

Request our Governance Readiness Checklist

Teams in regulated environments face a familiar strain. Applications grow in complexity, expectations for fast releases keep rising, and every update requires clarity about what changed and whether required elements still appear as intended. Traditional automation wasn’t built for that pace or level of oversight, and the recent wave of AI coding tools hasn’t solved the core challenges.

A better model is emerging—one that uses AI to reduce the workload of authoring and maintaining tests while keeping execution deterministic, reviewable, and aligned with how people evaluate digital experiences.

This post breaks down why the legacy testing model is hitting its limits and how AI can support a more stable, more trustworthy approach.

Why traditional automation keeps slowing teams down

As digital experiences expand across pages, portals, member journeys, and product flows, test code becomes difficult to scale. Even minor UI changes break locators and assertions, creating unpredictable test runs, delayed reviews, and long maintenance cycles.

Developers are often asked to take on more of the testing responsibility. While this can improve feedback loops, it does not reduce the burden of maintaining code that reacts poorly to UI changes. And when teams already lack time, context switching between product development and test diagnostics becomes expensive.

The result is a predictable bottleneck: too many tests tied directly to implementation details and not enough stability across releases.

Why AI-generated test code hasn’t fixed the problem

The last few years have produced a surge of tools that promise to generate automation code automatically. But teams report the same issues repeating in a new form. LLMs can produce code quickly, yet the resulting output still inherits all the maintenance challenges of coded automation.

AI code generators also excel more at producing new code than updating existing flows. They struggle with assertions, hallucinate element behavior, and require human supervision to validate every step. For regulated teams that must show repeatability and generate evidence for every release, inconsistency becomes a risk rather than a convenience.

If the goal is to escape brittle code, producing more of it is not the answer.

Why live LLM-driven execution creates instability

Another idea gaining attention is allowing an LLM to operate the UI directly during test execution. In theory, this removes the need to write code. In practice, teams quickly run into new risks: undefined steps, inconsistent interactions, slow decision-making, and no reliable way to debug.

Execution in regulated environments must be predictable. It must be reviewable. And it must produce evidence that can be traced, explained, and defended. Live improvisation during a test run undermines each of these requirements.

Determinism matters more than novelty. A testing approach must produce the same result today, tomorrow, and during an audit review.

A clearer path forward: intent-driven authoring with deterministic execution

A more reliable model is emerging that uses AI to simplify authoring without relying on AI to make real-time decisions during execution.

Teams describe test intent in natural language. An AI system translates that intent into structured steps during authoring, where humans can review and adjust. Execution is then handled by deterministic engines and Visual AI that observe the rendered UI and detect visual changes, required-element presence, placement consistency, and contrast.

This separation delivers two advantages:

• People write and maintain far fewer lines of test code
• Test runs become stable, repeatable, and easier to verify

Visual AI provides a complete view of the screen state and compares each run against an approved baseline. When something changes, the system surfaces the difference, captures evidence, and supports reviewer approvals. When the change is expected, one acceptance updates the baseline and applies it across browsers and devices.

The outcome is a testing layer that is easier to maintain and easier to trust.

What this looks like in practice

Teams adopting this approach typically see changes across several parts of their workflow:

• Tests are written in plain language, without selectors or framework setup
• Visual AI validates full screens for layout, presence, placement, and readability
• Changes are highlighted automatically to reduce manual inspection
• Evidence is captured through screenshots, diffs, timestamps, and logs
• Debugging takes place in an environment where runs behave the same every time
• Reusable flows and data-driven steps integrate into the same natural-language format

Instead of managing a growing volume of fragile code, teams maintain intent-level descriptions supported by deterministic execution.

What this means for oversight and compliance

For teams in financial services, healthcare, insurance, or life sciences, the benefits go beyond efficiency.

A visually grounded testing model helps confirm that required notices, disclosures, language-access elements, and other regulated UI content remain present and placed as expected. It documents what changed and preserves evidence for review. It supports consistent experiences across browsers, devices, and PDFs without checking whether values, data, or regulatory text are correct.

Most importantly, it delivers predictable results.

Regulated environments depend on clarity and traceability. When every test run yields reviewable outputs, and every change is captured with context, teams can maintain confidence and release with speed.

If you’re assessing how well your testing workflow supports stability and audit readiness, request our Governance Readiness Checklist. We’ll share the version designed for your stage—whether you’re evaluating Applitools or optimizing an existing deployment.

The post AI Testing in Regulated Environments: Smarter Testing Starts With Stability, Not More Code appeared first on AI-Powered End-to-End Testing | Applitools.

TL;DR

• Autonomous testing is maturing quickly, but regulated organizations must evaluate platforms through the lens of traceability, auditability, and control.
• Forrester’s Autonomous Testing Platforms Landscape, Q3 2025 shows that the real differentiators now are explainability, risk-based orchestration, and AI governance—not just automation speed.
• Use this checklist to choose a platform that accelerates delivery while protecting oversight.

Download Forrester’s full report for detailed market insights

Rethinking Autonomy for Regulated Teams

With hundreds of tools now promising “AI-driven automation,” sorting true autonomy from clever scripting has become increasingly difficult. This matters even more for regulated teams planning their 2026 quality strategy. Speed is no longer the only concern. Proof, traceability, and controlled execution are now essential.

Forrester’s recent analysis highlights a market shifting from test automation to AI-augmented and agentic systems that generate, maintain, and execute tests under human supervision. The key question for regulated buyers is not whether autonomy will help, but whether the platform provides clear governance around how that autonomy operates.

Use this checklist to evaluate solutions with the guardrails required for safety-critical or compliance-heavy environments.

Core Capabilities Every Autonomous Testing Platform Should Provide

These capabilities form the baseline for operating safely and efficiently in regulated sectors.

Plain-language test authoring and execution
Non-technical reviewers should contribute without adding risk. Natural-language authoring and guardrails make collaboration safe and auditable.

Transparent AI actions
Every generated or changed step must be reviewable. No black-box maintenance. No silent updates.

Evidence management and auditability
Exportable logs, change histories, and evidence packs should support internal and external audits without manual rework.

Role-based control and gated approvals
Automation should accelerate work, but never bypass required compliance workflows.

Adaptive, governed maintenance
Self-healing is useful only when changes are traceable and reversible. Regulated teams need adaptive maintenance under human oversight.

If a platform lacks any of these essentials, it’s not built for environments where documentation and control are mandatory.

Where Advanced Platforms Differentiate

Once the fundamentals are covered, regulated organizations should look at the capabilities that separate mature autonomous solutions from those still catching up.

Intent-based visual and experience validation
Pixel comparison is brittle. Intent-driven validation ensures the interface appears correct, accessible, and compliant across devices and browsers.

Governance dashboards
AI actions, risk coverage, and test triggers should be visible and easy to trace for auditors and managers.

Actionable analytics and reporting
Evidence should turn into insights that support risk management, release approvals, and executive reporting.

Risk-based orchestration
Platforms should prioritize tests based on business criticality, change impact, and historical issues—not just run everything in bulk.

Applying Autonomous Testing in Regulated Workflows

Organizations across healthcare, life sciences, financial services, and other regulated industries are already adopting autonomous testing—but always with governance in place.

In the pharmaceutical sector, EVERSANA INTOUCH takes a hybrid approach, combining Applitools Eyes for Visual AI validation with Applitools Autonomous for intelligent test generation. This end-to-end strategy ensures quality products, supports compliance-ready evidence, reduces maintenance, and provides end-to-end coverage across complex workflows—all while keeping human reviewers in charge. Read the EVERSANA INTOUCH case study.

These hybrid models show how autonomy can increase coverage and speed without loosening control.

Applying the Checklist to Your Evaluation Process

Use this framework when comparing platforms side by side:

• Map your highest-risk business journeys. Focus on areas tied to compliance, customer safety, or financial impact.
• Prioritize transparency. Ensure the platform shows why AI takes each action and allows review before changes go live.
• Assess evidence and governance. Exportable results, audit-ready logs, and approval gates are non-negotiable.
• Evaluate adaptability. Autonomous maintenance should reduce manual effort but still operate inside defined boundaries.

• Reassess regularly. The market is moving fast. Capabilities that seem advanced today will become baseline expectations.

Choosing with Confidence

Autonomous testing is reaching maturity, but regulated organizations need more than speed—they need governance, visibility, and trust. Forrester’s research confirms that platforms built with explainability and risk alignment at the center are the ones best suited for compliance-driven teams.

Use Forrester’s analysis and this checklist to guide your next evaluation and choose an autonomous testing solution that accelerates both delivery and confidence. Download the Autonomous Testing Platforms Landscape, Q3 2025 report.

The post Buyer’s Checklist for Autonomous Testing in Regulated Environments appeared first on AI-Powered End-to-End Testing | Applitools.

Learn the differences between Selenium, Cypress, and Playwright when it comes to automated screenshot testing and screenshot quality.

The Issue with Testing Screenshots

Capturing screens is a fundamental piece of testing web assets and reporting defects. There are other use cases for these screenshots, too. I work as the Director of Quality Control for my company. We service the pharmaceutical industry. I was tasked with leveraging existing automation processes to deliver high quality images of client sites to submit for legal review more efficiently.

My journey to find the best way to take screenshots to fill this niche was filled with ups and downs. The journey started with Selenium WebDriver, which fell short of meeting the requirement. Next I moved to Cypress because it showed great promise. It also had limitations I could not work around. Finally, I landed on Playwright and was able to meet the desired output.

I will err on the side of caution and use a site that is completely unrelated to any of our client work. Snopes seems like a safe choice. Besides, we will not do any advanced or prolonged calls against their site. Be nice to your internet neighbors!

Screenshot Testing in Selenium WebDriver

Automating full page screenshots with high fidelity has been troublesome in the past. Selenium WebDriver, the de facto champion of browser automation, lacks a method to capture a full page. Having said that, Selenium drivers do have methods for screenshots. The sample code for taking a screenshot with Selenium WebDriver is straightforward:

takeSnopesScreenshot.js

let {Builder} = require('selenium-webdriver');
let fs = require('fs');

(async function checkSnopes() {
    let driver = await new Builder()
    .forBrowser('chrome')
    .build();

    await driver.get('https://www.snopes.com');
    // Returns base64 encoded string
    let encodedString = await driver.takeScreenshot();
    await fs.writeFileSync('./homepage.png', encodedString, 'base64');
    await driver.quit();
}())

Here are the steps we took to capture our image:

1. We start by bringing in our Builder class so we can create new webdriver instances.
2. We also need access to the file system so we can save our image.
3. Next, we instantiate a new webdriver for the Chrome browser.
4. Then we instruct the webdriver to open https://www.snopes.com. This should probably be put into a variable. We’ll leave it here since we are only going to one page on the site.
5. Next, we take a screenshot of the web page.
6. Then we save the image to disk by converting the base64 encoded string into a .png file.
7. Finally, we free the driver from service.

We get a screenshot like this after running the code:

The resulting image looks nice. A couple of things immediately stood out to me as problems here:

1. It is not a full-page image.
2. The scrollbar is displayed.

We attempt to fix these issues by setting the window size after initializing the webdriver:

takeSnopesScreenshot2.js

let {Builder} = require('selenium-webdriver');
let fs = require('fs');

(async function checkSnopes() {
    let driver = await new Builder()
    .forBrowser('chrome')
    .build();

    await driver.manage().window().setRect({ width: 1024, height: 2000 });

    await driver.get('https://www.snopes.com');
    // Returns base64 encoded string
    let encodedString = await driver.takeScreenshot();
    await fs.writeFileSync('./homepage.png', encodedString, 'base64');
    await driver.quit();
}())

Two things come to mind if this works the way I anticipate:

1. I may not know the dimensions of the longest page.
2. I do not want to add more code to maintain.

The resulting screenshot after resizing the window:

The second image is high quality. Obviously, the setRect() method did not force the full page image like I had hoped.

This works great for capturing what is in the current viewport, but not so well for a page where scrolling to see all the content is needed. Instead, we would need to “stitch” the images together to get a full-page screenshot. There is also a third-party plugin named aShot for the Java users of Selenium. Not much help in a .NET and JavaScript shop. No easy, consistent method to do full page captures is a deal breaker for my use case.

Screenshot Testing in Cypress

Speedy delivery, simplicity, and consistency are keys for a successful submission. The Champ (Selenium WebDriver) is too cumbersome. That leads to a scrapy newcomer, Cypress.io. Cypress is an amazing framework. Easy to use – check. Fast as lightning – double check. It even has a built-in Screenshot API that captures full page images – yes, please. Here is a quick example of what that looks like:

snopes.test.js

describe('submission screenshots', () => {
    it('takes screenshot of the home page', () => {     
        cy.visit("https://www.snopes.com");   
        cy.screenshot('/screenshots/cypress-homepage');
    });
});

Here’s the breakdown of what we did:

1. We set up our test suite using the built-in support from Mocha.
2. Next, we set up the test.
3. Then we visit snopes.com.
4. Finally, we take the screenshot and save it as cypress-homepage.png.

A few things to note here:

1. The code feels less intimidating and more accessible.
2. We did get a full-page image!
3. There is a problem, though. Can you see it? (click to view full size image)

Another great-ish image. Cypress, however, appears to suffer from some of the same stitching and consistency issues found in Selenium. Images were not consistent. Some were blurry. Others were missing content – usually near the bottom of the web page. This is not likely the end of the world for most use cases, but those are killer obstacles to overcome to create legal submission documents for a federally-regulated company.

Screenshot Testing in Playwright

I was quickly losing faith in being able to find a good solution to help with our automated screenshot efforts. Then, I found Playwright, Microsoft’s entry for automated UI testing. It is a fantastic framework that checks the boxes for being speedy and easy to use. It also has the capability to capture full page images out of the box. It looks promising. The only question left is the quality of images.

Spoiler alert – It works like a charm . Let’s take a look at some of the code:

snopes.test.js

const {test, expect} = require('@playwright/test');

// The site I will capture screens from.
const url = "https://www.snopes.com/";
const savePath = "./screenshots/";

// I have a wrapper function to beef up Playwright's screenshot method.
async function takeScreenshot(page, label) {   
    await page.screenshot({
      path: `${savePath}${label}.png`,
      fullPage: true,
    });
}

// Start the testing
test.describe('Submission Screenshots:', async () => {

    test('0.0 - Homepage', async({page}) => {
        await page.goto(url);    
        await takeScreenshot(page, "playwright-homepage");
    });

    test('1.0 - Menu', async({page}) => {
        await page.goto(url);
        await page.click("//html/body/header/div[1]/nav/div/div[3]/div/a[2]");
        await takeScreenshot(page, "playwright-menu");
    });

    test('2.0 - Search Results', async({page}) => {
        await page.goto(url);
        await page.fill('input[name="s"]', 'scam');
        await page.press('body', 'Enter');
        await takeScreenshot(page, "playwright-search-results");
    });
});

This snippet is a bit more polished than the others.

1. We moved the URL into a proper variable.
2. Next, we created a variable to hold the saved images path.
3. Next is a wrapper function to take care of the additional information (label) needed for each image.
4. Inside we call Playwright’s screenshot method.

From there the test suite and individual tests take on the Mocha format like Cypress.

These are the screenshots we get after running the code (click for full size):

Those images look amazing. Full page. Crisp. Consistent. The time taken to run through the screens is not bad for what we are doing, but there is considerable overhead when doing file I/O. We added 20 seconds by writing those large files to disk.

The chart below lists the times for capturing just the homepage image versus a visit to the page for each framework. Keep in mind that these times reflect visiting an external site on my personal internet connection. Times on a local build will be significantly faster.

Conclusion

These are simple examples of how to use the screenshot capabilities of three common automated testing frameworks. The differences of the resulting images were quite shocking. I presumed that a screenshot was a screenshot.

It is important to note that different here does not mean bad. You have to take your use case into account. Take the extra 20 seconds needed to write three full page screens to disk for example. That is not a typical use case, but it is perfect for the time savings we set out to recoup by automating submission screenshots. Is it perfect? No, but it gets closer with every site we automate.

There is room for enhancement. We can move from capturing and saving screenshots into automated visual testing with Applitools. Visual testing with Applitools is an enhancement worth making, especially for regulatory or compliance testing, because we will be alerted whenever a visual change is detected. This is done by swapping out the screenshot method calls with calls to Applitools Eyes.

The post Screenshot Testing with Selenium, Cypress and Playwright: 3 Popular Automation Tools Deliver Amazingly Different Screenshots appeared first on AI-Powered End-to-End Testing | Applitools.

The fourth industrial revolution – the digital revolution – has strong requirements for companies operating under strict business regulations.

Particularly, in the healthcare sector, companies must spend great efforts to survive “digital Darwinism”. The healthcare market is highly competitive and strongly regulated at the same time. Healthcare, pharmaceutical, and medical device companies invent new medicines and other products in a highly volatile business landscape. On one hand, they have to act as agile as possible, considering time-to-market delivery, on the other hand, they face strict compliance regulations, like FDA, HIPAA, etc.

The question is how healthcare companies can deliver new products and services at high speed while meeting their regulatory compliance obligations?

In this article, you will find the answer based on an example of the FDA-Requirement “Back-Box-Warning”.

The FDA (Food and Drug Administration) prescribes warnings and precautions on the package insert for certain prescription drugs. As these warnings are framed in a “Black Box” to catch the eye of the reader, they are also referred to as “Black Box Warnings” and can be found at the beginning of the package insert (see Picture 1) or in the drug description in the online store (see Picture 2).

If the FDA finds serious violations due to missing or unreadable Black Box Warnings, the FDA can take legal actions against a company.

Let me present an example showing how automated visual regression tests for web sites and PDF-documents can be implemented to automatically verifies regulatory requirements.  

The FDA’s General Principles of Software Validations recommends using visual regression testing for images and documents. The FDA makes this recommendation for companies using a software development lifecycle (SDLC) approach that integrates risk management strategies with validation and verification activities, including defect prevention practices.

What is visual regression testing?

Visual regression testing expands regression testing, where a program, or parts of it, repeatedly gets tested after each modification. To additionally avoid unintentional changes in design elements, positioning, and colors, QA-Teams use visual regression testing as part of their testing strategy and general quality assurance.

Visual regression testing can discover visual defects, obvious or not, due to modifications to the software. In practice, a baseline of original or reference images is stored. This “source of truth” can be compared after each program modification against a collection of “new” screenshots of a user interface. Each difference against the baseline will be highlighted and can serve as an alert.

Additionally, visual regression testing doesn’t only look at differences between the source and current status. It has the possibility to compare the source against any historical status on a UI level, independent of HTML, CSS, and JavaScript differences. It can also be used to highlight differences between documents, like PDFs, in the layout, or the content itself. For example, a missing black box warning in a package insert for a drug would be marked as a difference.

How to best use visual regression testing?

Many visual testing tools, such as Selenium, mark differences between screenshots or PDF-documents as passed or failed. With visual regression testing, you can choose which differences across multiple browsers and devices to accept or not. For example, a picture, which is displayed in a different resolution on a web-page after a program change may cause a problem in completing a user-action due to overlapping with an FDA-required text (see Picture 3). This can cause a reporting to the FDA by a competitor and a warning letter would be sent to the legal department of the healthcare company.

Visual regression testing tools and libraries, like Wraith, Gemini, and other Selenium-related frameworks, needs a deep knowledge by testers and high effort in installation and setup. The Applitools AI-Platform, where no installation-, setup- or coding-knowledge is required, could be a great alternative to start automated visual regression testing.

The Applitools Eyes cross-environment testing feature allows you to test your application on multiple platforms using a single, common baseline. The match level (Strict, Layout, Content, Exact) determines the way by which Eyes compares the checkpoint image with the baseline image.

Additionally, the Applitools PDF Tool allows you to easily run visual UI tests on a collection of PDF-files, by placing them inside a directory (see Picture 4). It runs as a standalone jar file and can be invoked as a process by any programming language and in your continuous delivery pipeline.

Summary

If you want to continuously deliver new products and services within a software development lifecycle, at high speed while considering regulatory compliance regulations, you should have an eye on visual regression testing. It can be used for automated testing by comparing hundreds or thousands of artifacts like images and PDF-Documents at very much speed. Therefore, it provides long term cost efficiency by avoiding extensive manual tests, especially when dealing with frequent changes on a UI or document base.

The post Continuous visual regression testing to enable regulatory compliance in the healthcare sector appeared first on AI-Powered End-to-End Testing | Applitools.