Eyes Archives - AI-Powered End-to-End Testing | Applitools https://app14743.cloudwayssites.com/blog/tag/eyes/ Applitools delivers full end-to-end test automation with AI infused at every step. Wed, 11 Mar 2026 19:01:49 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.8 Test Maintenance at Scale: How Visual AI Cuts Review Time and Flakiness https://app14743.cloudwayssites.com/blog/test-maintenance-at-scale-visual-ai/ Tue, 21 Oct 2025 20:22:00 +0000 https://app14743.cloudwayssites.com/?p=61615 Reduce flakiness, speed up reviews, and see how teams like Peloton cut maintenance time by 78% using Visual AI.

The post Test Maintenance at Scale: How Visual AI Cuts Review Time and Flakiness appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
smarter test maintenance at scale

Why Test Maintenance Breaks at Scale

Test maintenance at scale slows releases. Teams that rely on coded assertions spend more time updating tests than improving coverage. Brittle locators, environment drift, and false positives all add up—turning automation into a maintenance cycle.

Neglecting maintenance is like skipping car care: small issues snowball into costly downtime. A smarter approach replaces manual review and locator-based scripts with automated, visual validation that adapts as your UI evolves.

How Visual AI Delivers Test Maintenance at Scale

Visual AI replaces dozens of coded assertions with a single checkpoint that mimics how humans see. It validates full UI states, detecting layout shifts, missing elements, and text overlaps automatically.

By consolidating validations into one Visual AI check, teams cut review time, reduce false positives, and gain faster feedback cycles.

Scale Reviews with Ultrafast Grid and Grouping

Running tests one browser at a time no longer scales. The Applitools Ultrafast Grid executes a single test once, then validates results across every browser and device combination in parallel.

Batching and grouping features make reviews equally efficient—approve or reject similar changes across entire runs in just a few clicks.

How it works

  • Replace assertions with one visual checkpoint
  • Run once across all browsers and devices
  • Batch results for unified review
  • Approve or reject in bulk
  • Tune match levels for dynamic content

Together, these capabilities eliminate redundant effort and make large-scale testing faster to maintain.

Customer Results: 78% Less Maintenance

Teams that adopt this approach see measurable ROI. At Peloton, replacing a legacy visual testing tool with Applitools Visual AI produced a 78% reduction in maintenance time and saved about 130 hours per month.

With dynamic leaderboards, live data, and responsive layouts across web and native mobile, Peloton maintains quality at scale without expanding test overhead.

Three Features That Change Maintenance

Ultrafast Grid, Visual AI match levels, and bulk grouping—those three change the game.”

Mike Millgate, Smarter Test Maintenance at Scale

These three deliver flexible validation, fast execution, and effortless maintenance. Each removes manual steps and accelerates the feedback loop that keeps releases reliable.

Smarter Maintenance for Modern Teams

Smarter test maintenance isn’t about writing more code—it’s about automating intelligently. Visual AI reduces flakiness, speeds reviews, and scales across devices and environments.

To see what’s next, explore Applitools Eyes 10.22, featuring faster review cycles, new Storybook and Figma integrations, and even shorter feedback loops for test maintenance at scale.

Frequently Asked Questions

What is Visual AI testing?

Visual AI uses automated visual assertions to validate full UI states, catching layout and content changes that code-heavy checks miss.

How does Visual AI reduce test maintenance at scale?

One visual checkpoint replaces dozens of brittle assertions, while batching and grouping speed reviews across browsers and devices.

What’s the difference between Visual AI and visual regression testing?

Visual AI applies learned match levels and region logic to reduce false positives and handle dynamic content; classic visual diffing is more brittle. Learn more about Visual AI.

How do match levels help with dynamic content?

Layout, text, and color match levels tune sensitivity so teams can ignore cosmetic shifts while catching meaningful UI regressions.

Does Visual AI work with my framework (Selenium, Cypress, Playwright)?

Yes—Applitools has drop-in SDKs let you run your existing tests and add a single Visual AI checkpoint. Learn how to quickly integrate Applitools into your current tech stack.

The post Test Maintenance at Scale: How Visual AI Cuts Review Time and Flakiness appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Test Your Components Where You Build with the Applitools Storybook Addon https://app14743.cloudwayssites.com/blog/test-your-components-where-you-build-with-the-applitools-storybook-addon/ Fri, 17 Oct 2025 11:07:00 +0000 https://app14743.cloudwayssites.com/?p=61542 Test Storybook components with Visual AI inside Storybook. Catch UI bugs early, bulk-maintain baselines, and scale cross-browser coverage.

The post Test Your Components Where You Build with the Applitools Storybook Addon appeared first on AI-Powered End-to-End Testing | Applitools.

]]>

Local dev is where most UI changes happen (and where regressions sneak in). States drift, styles diverge, and tiny tweaks pile up until something breaks in CI. The Applitools Storybook Addon brings AI-powered visual testing straight into Storybook so you can catch issues as you code, approve the good changes quickly, and keep your CI/CD pipelines green.

AI-Powered Visual Testing Inside Storybook

Open your Storybook and run visual tests from an Applitools Eyes tab – no context switching. Results are grouped by component to mirror your Storybook structure, and a reporter widget highlights what needs attention first so you can review diffs in minutes, not hours. Learn more on our Storybook Component Testing with Applitools page.

  • Catch bugs where you build. Validate component states during local development and avoid surprises later.
  • Review faster with Visual AI. See only meaningful, human-perceptible UI changes without pixel-to-pixel noise. Tune sensitivity with AI match levels when you need to.
  • Scale coverage painlessly. Run once; render everywhere with Ultrafast Grid across browsers, devices, and viewports in parallel.

How to Use the Applitools Eyes Storybook Addon

Getting started takes just a couple of minutes.

  1. Install the SDK & Addon
    Add Applitools Eyes to your project and enable the Storybook addon (React, Vue, Angular supported). See the installation instructions in the Eyes Storybook Addon docs.
  2. Run Applitools Visual Tests in Storybook
    Open Storybook, switch to the Applitools Eyes tab, and trigger tests for a single story or an entire component. Results stream back in real time with automatic grouping by component.
  3. Review & Maintain
    Use Visual AI diffs, side-by-side views, and auto-maintenance to approve or reject changes in bulk. Prioritized sorting surfaces what needs attention first.
  4. Scale Across Browsers/Devices
    Turn on Ultrafast Grid to parallelize renders across Chrome, Firefox, Safari, Edge, and mobile sizes – without extra local setup.

Applitools Storybook Addon Use Case Playbook

Below are the three most common ways teams use the Eyes Storybook Addon each – with a quick, practical flow pulled right from the product.

Use Case: Guard Your Design System

As you refactor tokens or update themes, run visual tests on every component state. Spot unintended changes across the library instantly.

How to do it in Storybook

  1. Start Storybook and open your design‑system component in the Applitools Eyes tab.
  2. Click Run from the tab (or use Run in the left sidebar test module). The addon tests the stories and streams results inline for every browser/device in your applitools.config.js.
  3. In the sidebar, filter by Unresolved to zero in on changes across the library (Green = passed, Orange = unresolved, Red = failed).
  4. Open a story’s result and use Side‑by‑Side or the Slider to spot subtle spacing/typography diffs.
  5. Approve legit theme updates with Thumbs Up (or use ⋯ → Review actions to approve the whole story/batch). Reject regressions with Thumbs Down and fix.

Pro tip: Use the tab ⋯ → Configuration to confirm you’re validating the right browser matrix and server URL. See more options in the docs.

Use Case: Fix Fast During Local Dev

Working on a feature branch? Validate your component in Storybook before you commit.

How to do it in Storybook

  1. Open your feature’s stories, then hit Run in the Applitools tab for the component you’re touching.
  2. Watch statuses update inline; click the status buttons to filter to Unresolved so you only look at what changed.
  3. Click into any row to open compare tools: Diff Image, Actual Image, Expected Image, Side‑by‑Side, or Slider.
  4. If the change is intended, Thumbs Up to approve; otherwise Thumbs Down to flag and keep iterating.
  5. When you’re happy locally, push your branch. You can scale the same setup in CI using your existing Storybook build/preview URL.

Heads‑up: To view baselines or approve/reject, sign in to your Applitools account in the same browser that’s running Storybook (you’ll be prompted if not).

Use Case: Ship Multi‑Browser Confidence

One click, many targets. Validate layout and responsive behavior across browsers and viewports – early.

How to do it in Storybook

  1. In ⋯ → Configuration, verify your browsers/devices list (Chrome, Firefox, Safari, Edge; add viewports you care about).
  2. Hit Run for representative stories (states, theming, interactive). Results come back grouped by each browser/device so differences are obvious.
  3. Filter the sidebar by Unresolved and scan. Use Side‑by‑Side or Slider to compare layout at different sizes.
  4. Approve good changes in bulk (⋯ → Review actions) to keep maintenance low.
  5. For broader coverage, run the same setup in CI and expand the matrix.

Why Visual AI > Pixel Diffs for Storybook

Pixel-to-pixel tools are fragile with dynamic content and minor rendering differences. Applitools Visual AI mimics human vision to highlight only meaningful UI changes (structure, layout, content) while ignoring the noise. You can still dial sensitivity up or down with match levels whenever needed. Less flake, more signal.

Try AI-Powered Visual Testing in Storybook Today

Run your first component tests in minutes, review diffs right in Storybook, and expand coverage with Ultrafast Grid – without slowing delivery.

Frequently Asked Questions

What does the Applitools Storybook Addon do?

It runs Applitools visual tests from inside Storybook. You can trigger tests per story or component, then review results and diffs inline with automatic grouping that mirrors your Storybook tree.

Do I need to write tests with the Applitools Storybook Addon?

With the Applitools Storybook Addon, you existing stories become the tests.

How is the Applitools Storybook Addon different from Chromatic visual tests?

Applitools’ Visual AI detects signficant visual differences instead of only pixel-to-pixel comparisons. This means you see fewer false positives and spend less time on maintenance.

Appitools also lets you auto-maintain hundreds of tests at once (when you do need to perform test maintenance), run them across multiple browsers and devices instantly, and manage everything in the same platform that’s also running your Playwright and Cypress end-to-end test flows. See our Applitools vs. Chromatic comparison page for a deeper breakdown.

What about performance and CI stability?

Validate locally in Storybook to prevent CI failures. When you’re ready, run the same tests in CI and render broadly with Ultrafast Grid – fast and consistent.

Do I need an Applitools account to use the Storybook Addon?

Yes. You’ll need an active Applitools Eyes account and an API key to use the Applitools Storybook Addon.

The post Test Your Components Where You Build with the Applitools Storybook Addon appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Validate Your Figma Designs Before Code Ships with the Applitools Eyes Plugin https://app14743.cloudwayssites.com/blog/figma-design-testing-applitools-plugin/ Mon, 13 Oct 2025 22:00:00 +0000 https://app14743.cloudwayssites.com/?p=61531 Use the Applitools Eyes Figma Plugin to test and compare designs against your live app. Catch visual changes early to confirm UI accuracy.

The post Validate Your Figma Designs Before Code Ships with the Applitools Eyes Plugin appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Applitools Eyes Figma plugin on top of a blurry Figma frame

Even the best design systems can fall short when a layout moves from Figma to code. Fonts shift, buttons resize, and colors look a little off. These small issues result in visual drift and long review cycles between design, development, and QA.

Figma design testing with Applitools Eyes closes that gap. Export Figma frames directly to Eyes to compare what you designed with what you built using the same visual testing tools your QA teams already trust.

Design-to-Code Testing in One Place

The plugin lets you send Figma frames, including individual components, pages, or entire prototypes, straight into Applitools Eyes. Each exported frame becomes a visual baseline, the same kind used in automated tests.

Developers can run their regular visual tests against these baselines to confirm that what they’ve built matches the approved design. Meanwhile, Designers can export each new version of a design to see what changed between iterations. Everyone reviews results in the same Eyes dashboard, where visual differences appear side by side.

This shared view reduces guesswork and keeps teams aligned around what “correct” actually looks like.

How to Use the Applitools Eyes Figma Plugin

Getting started takes just a couple of minutes.

1. Install the Plugin

Open the plugin from the Figma Store, or open the Figma desktop app and select Plugins → Manage Plugins → Search “Applitools Eyes” → Install.

Applitools Eyes Figma plugin on top of a blurry Figma frame

2. Connect Your Applitools Account

Launch the plugin and enter your Applitools API key and server URL (default: https://eyes.applitools.com). These settings are saved for future use.

3. Select Figma Frames to Export

You can export a single frame, multiple frames, or a full design. The plugin automatically names them based on your Figma file, or you can customize names with dynamic parameters like {figma_filename}, {figma_page}, or {figma_frame}.

4. Adjust Settings

Optional configurations include:

  • Match level: strict, dynamic, layout, ignore colors, exact, or none
  • Contrast level: accessibility comparison thresholds
  • Auto-accept baselines: mark first exports as approved
  • And more…

5. Export and Review

Lastly, click Export to Eyes to send your selections to Applitools. Frames appear in the Eyes dashboard under the “Figma” environment. Designers and Devs can view differences directly and decide whether to accept or reject them.

Figma plugin overlayed on screenshot of Applitools Eyes comparing a Figma frame and visual test in Chrom

Three Use Cases for QA Teams

1. Design-to-Implementation Validation

Once designs are uploaded, developers can link automated tests to the same baseline using the “baseline environment name” provided by the plugin. When they run their tests, Eyes compares the live UI against the design reference.

Result: Teams catch spacing, text, or layout differences before they reach production.

2. Design-to-Design Version Comparison

Designers often revisit earlier layouts or explore small variations. Exporting both versions to Eyes highlights the exact visual differences, making it easy to review and choose the preferred version.

Result: Faster review cycles and fewer overlooked design changes.

3. Shared Visual Baselines for Collaboration

Designers, developers, and QA teams can all access the same Eyes dashboard. Instead of passing screenshots or notes, they can comment on the same visual checkpoints.

Result: Clearer handoffs and fewer miscommunications between design and engineering.

Why Visual Testing from Design to Code Matters

Designs are often reviewed visually, while code is tested functionally. However, the Figma plugin connects these two disciplines by giving both teams a consistent, visual source of truth.

For designers, it’s a way to confirm that their layouts are faithfully implemented without manually comparing screenshots. The plugin provides a reference that removes ambiguity about spacing, colors, or typography for developers. For QA teams, it introduces an additional layer of confidence that each release matches approved specifications.

This integration fits naturally into existing workflows: designs are exported once, developers test as usual, and visual checks happen automatically. What was once a manual review step becomes part of the team’s regular quality process.

Try Design-to-Code Testing for Yourself

The Applitools Eyes Figma Plugin brings visual testing into the design process, helping teams maintain consistency from mockup to release. It’s a straightforward way for design and development to share one accurate reference for how an interface should look in order to reduce manual review and give everyone confidence that what’s shipped matches what was designed.

Install the Applitools Eyes Figma Plugin and start validating your designs before code ships.

Frequently Asked Questions

What is the Applitools Eyes Figma Plugin?

The Applitools Eyes Figma Plugin lets you export frames from Figma into Applitools Eyes for visual testing. It helps teams compare their designs against live implementations or across design versions, ensuring the final product matches what was originally designed.

Why should I use the Applitools Eyes Figma Plugin?

The main reasons teams like using the plugin include:
– Detecting visual differences early in development
– Maintaining design consistency from mockup to production
– Reducing manual screenshot comparisons
– Providing a shared visual reference for design, QA, and development teams

How does Figma design testing work with Applitools?

Figma design testing with Applitools works by turning design frames into visual baselines inside the Eyes dashboard. Developers then run automated tests that capture the built UI and compare it to those baselines, highlighting any visual differences between design and implementation.

Can I compare two Figma designs using the plugin?

Yes. You can export two or more design versions to Applitools Eyes and compare them visually. The dashboard highlights differences such as layout changes, spacing updates, or color tweaks, making it easier to review design revisions before sign-off.

Do I need an Applitools account to use the Figma Plugin?

Yes. You’ll need an active Applitools Eyes account and an API key to export Figma frames to Eyes. Once connected, you can reuse your credentials for future exports.

The post Validate Your Figma Designs Before Code Ships with the Applitools Eyes Plugin appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Test Where You Build with Eyes 10.22: Visual AI for Storybook & Figma https://app14743.cloudwayssites.com/blog/visual-testing-for-storybook-and-figma/ Thu, 09 Oct 2025 17:53:32 +0000 https://app14743.cloudwayssites.com/?p=61397 Applitools Eyes 10.22 brings Visual AI testing directly into Storybook and Figma—helping teams test where they build, align design and development, and release with greater speed and confidence.

The post Test Where You Build with Eyes 10.22: Visual AI for Storybook & Figma appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Applitools Eyes Figma plugin on top of a blurry Figma frame

The new Applitools Eyes 10.22 release brings Visual AI testing directly to the tools teams already use to design and develop digital experiences. This update strengthens how teams build, validate, and release with three major enhancements:

  • Scale quality without slowing down delivery using the new Storybook Addon.
  • Align development and design with the new Figma Plugin, bridging intent and implementation.
  • Improve traceability and accountability with smarter Dashboard Optimizations for QA and leadership.

Together, these updates make visual testing faster, sharper, and more connected across the product pipeline, helping teams fully embrace shift-left testing and collaborate around a shared baseline for visual quality.

Scale Quality Without Slowing Down Delivery: Storybook Addon

Test Where You Build with the Storybook Addon

The new Storybook Addon for Applitools Eyes brings storybook visual testing directly into the development workflow. Developers can now validate UI components where they build them—no CI/CD wait times, no switching between tools.

Core Capabilities

  • Run and review visual tests directly inside your component workspace.
  • Approve or reject diffs inline and validate changes locally before merging.
  • Leverage higher concurrency for faster feedback.
  • Catch visual issues early and prevent CI failures that slow releases.

Use Case Example:
A frontend developer refactoring a shared button component runs Eyes tests directly in Storybook to verify that the update didn’t alter its appearance across the design system. By catching issues immediately, they avoid regressions on the main branch and prevent unnecessary rework later in the release cycle.

The Applitools Eyes Storybook Addon allows teams to scale quality by expanding coverage while maintaining release velocity. By embedding visual validation where developers work, organizations can expand test coverage and reduce release timelines without compromising accuracy.

See the newest features in action during the Platform Pulse webinar, coming October 22.

Align Development and Design: Figma Plugin

Bridge Design & Code with the Figma Plugin

The new Applitools Figma Plugin connects design intent with implementation through seamless Figma design testing and design-to-code comparison. Designers and developers can now validate visual consistency earlier—and with far less manual effort.

What You Can Do

  • Export Figma frames directly into Eyes for automated design-to-code validation.
  • Compare design-to-design versions to manage iteration cycles.
  • Maintain shared baselines so designers and developers work from the same source of truth.
  • Designers can validate implementations without writing code, while developers see exactly what “done” looks like.

Use Case Example:
A designer exports a component from Figma and compares it with the implemented version in Eyes to confirm that spacing, colors, and typography match the original design. Any differences appear immediately, allowing the designer and developer to resolve them early instead of during QA or after release.

This plugin bridges design and development, reducing handoff friction and review cycles. Teams gain a single source of truth for UI quality and deliver products that match design intent from concept to code.

Improve Traceability and Accountability: Dashboard Optimizations

Context-Rich Results with Dashboard Optimizations

Eyes 10.22 introduces dashboard enhancements that bring clarity and accountability to every test result. These updates help QA engineers, SDETs, and technical leaders focus on what matters most.

Key Highlights

  • Commit SHA and branch info appear directly in batch details for instant traceability.
  • “Required Attention First” sorting automatically surfaces unresolved and failed tests.
  • Auto-grouping mirrors your Storybook structure, making results easier to navigate.
  • Streamlined triage reduces noise from passed tests and accelerates review.

Use Case Example:
A QA engineer reviewing test results spots a visual regression and uses the Eyes dashboard to trace it back to the specific commit that introduced the change. With commit and branch details in context, the engineer can alert the right developer and close the feedback loop quickly.

These context-rich results strengthen accountability and compliance across teams. Organizations gain the visibility they need for auditability—particularly valuable for industries with governance or regulatory requirements.

Explore all the updates and improvements in the Eyes 10.22 Release Notes.

Broader Strategic Impact

Eyes 10.22 delivers improvements that go beyond new capabilities. Together, these updates create a foundation for stronger collaboration and faster releases across the organization:

  • Align Development & Design Around a Single Source of Truth – With the Storybook addon and Figma plugin, teams collaborate around shared visual baselines.
  • Scale Quality Without Slowing Delivery – Native testing inside development tools ensures visual coverage grows with velocity.
  • Improve Traceability & Accountability – Git-linked context and prioritized review queues make it easier to understand, resolve, and communicate changes.
  • Drive Shift-Left Adoption Across the Organization – With validation embedded where work happens, teams can catch issues earlier and release with confidence.

Quality That Scales: Visual AI for Every Team

With Applitools Eyes 10.22, teams can test where they build—bringing Visual AI testing directly into their design and development workflows. By embedding validation inside Storybook, Figma, and the Eyes dashboard, organizations can scale software quality without adding maintenance or slowing delivery. Developers, designers, and QA now share one intelligent workflow powered by Visual AI—delivering expanded test coverage, faster feedback, and less maintenance across every stage of the testing lifecycle. Learn more about Applitools Eyes.

The post Test Where You Build with Eyes 10.22: Visual AI for Storybook & Figma appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Applitools Autonomous and Eyes: New AI Features, Better Execution, and What’s Next https://app14743.cloudwayssites.com/blog/applitools-autonomous-eyes-ai-testing-updates/ Thu, 07 Aug 2025 12:27:00 +0000 https://app14743.cloudwayssites.com/?p=61068 The newest updates to Applitools Autonomous and Eyes introduce AI-assisted test creation, built-in API and data support, and previews of upcoming MCP and mobile features.

The post Applitools Autonomous and Eyes: New AI Features, Better Execution, and What’s Next appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Screenshot showing variables created by random test data generation

Test automation is essential but often time-consuming. Writing and maintaining tests, generating reliable data, switching tools for API calls, and keeping everything aligned across environments can slow down even the best teams.

The latest updates to Applitools Autonomous, part of the broader Applitools Intelligent Testing Platform, introduce features that significantly reduce this overhead. From natural language test authoring to integrated API testing and deterministic execution, these additions help teams move faster with fewer manual steps. Alongside ongoing improvements to Applitools Eyes, the platform continues to evolve to support modern testing workflows at scale.

New Applitools Autonomous Highlights at a Glance:

  • Natural language test creation powered by LLMs
  • On-the-fly test data generation
  • Enhanced API testing with visual builder
  • Deterministic test execution (no LLMs at runtime)
  • Upcoming support for mobile apps, IDE integration, and Storybook workflows

Natural Language Test Creation, Powered by LLMs

Instead of writing test steps manually or wrestling with locators, you can now describe your intent in plain English. Applitools Autonomous converts your input into executable test steps.

Autonomous interprets the instruction and adapts to your application’s context. Tests can be created by typing, recording interactions, or letting the system generate steps automatically. This approach makes test authoring more accessible, easier to maintain, and more readable across teams.

On-the-Fly Test Data Generation in Context

Need a specific persona, value range, or edge case? Autonomous now includes built-in test data generation. No external tools required.

Just describe what you need, a French fashion designer or a prime number over 1,000, and the platform generates valid, realistic data at runtime. Datasets are generated ahead of time, so test execution remains fast and predictable.

Enhanced API Testing in the Same Flow

You can now send and validate API requests directly within your test flow, using a Postman-style interface.

Author steps in several ways:

  • Describe them in plain English
  • Use raw HTTP or cURL
  • Use the interactive UI builder

Once executed, responses can be inspected, variables extracted, and values asserted. UI, API, and visual checks all operate in a single environment—no tool switching needed.

Deterministic Execution Model for Reliable AI-Powered Tests

A standout feature of this release is the deterministic execution engine behind every test.

“You don’t need to be a prompt engineer or even a developer to scale automation. But when your test runs, it executes with the speed and reliability of code.”
Adam Carmi, CTO and Co-founder of Applitools

Unlike some platforms that rely on live LLMs during execution—an approach that can be slow or unpredictable—Applitools separates test creation from test execution.

  • LLMs assist during authoring and data generation.
  • Test runs are powered by a proprietary deterministic model that ensures speed, stability, and consistent behavior.

This offers the flexibility of AI and the dependability of code, without trade-offs.

What’s Coming Next

Applitools continues to invest in both Autonomous and Eyes, with upcoming features focused on deepening cross-functional collaboration, improving performance, and expanding platform coverage.

For Applitools Autonomous:

  • Native mobile app testing: Author and execute tests across devices and operating systems.
  • Autonomous MCP server: Translate high-level test cases or BDD scenarios into full test flows.

For Applitools Eyes:

  • Eyes MCP server: Move Visual AI directly into your workflow. Maintain, review, and run tests directly from your preferred IDE.
  • Visual testing in Storybook: Approve changes directly where components are built.
  • Performance improvements for component tests: Shorter pipelines and faster feedback loops.
  • Figma collaboration enhancements: Sync designs and visual testing for consistent results.

Where Things Stand Now

Whether you’re building automation for the first time or looking to reduce the overhead of test maintenance, this release meets teams where they are. With natural language authoring, integrated data and API support, and a deterministic execution engine, Applitools helps teams reduce manual effort and work more confidently.

If you’re already using Applitools, now’s a great time to explore the latest features. If you’re just getting started, we invite you to see what’s possible with a free trial for you and your team.


Quick Answers

What new capabilities were added in the latest Applitools updates?

Applitools expanded AI-assisted authoring and integrated API/data support in Applitools Autonomous while keeping fast, deterministic execution for stability.

How does Applitools keep AI authoring reliable at run time?

By separating natural-language test authoring from deterministic execution, test runs remain fast and consistent in Applitools Autonomous (https://app14743.cloudwayssites.com/platform/autonomous/) even as teams scale.

How do these updates reduce flakiness and speed feedback loops?

Visual validation focuses on what users actually see and runs in parallel across browsers/devices with the Ultrafast Grid (https://app14743.cloudwayssites.com/ultrafast-grid), so teams get fewer false positives and faster results with Visual AI (https://app14743.cloudwayssites.com/visual-ai).

The post Applitools Autonomous and Eyes: New AI Features, Better Execution, and What’s Next appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Handling Animations and Loading Artifacts in Visual Testing https://app14743.cloudwayssites.com/blog/handling-animations-and-loading-artifacts-in-visual-testing/ Mon, 21 Jul 2025 18:12:29 +0000 https://app14743.cloudwayssites.com/?p=61002 Master dynamic content visual testing with our hands-on tutorial. Learn to capture rich UI experiences effectively.

The post Handling Animations and Loading Artifacts in Visual Testing appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Stylized screenshot with half greyed out and other half colorized to highlight dynamic content

Have you ever encountered a situation where you try to take a screenshot, but instead of the beautifully well-crafted UI, all you’ve got is an image of a spinner/skeleton/loading screen? Handling animations and loading artifacts in visual testing can be daunting.

Don’t worry – it can happen to anyone, and we’re not here to judge you 😉

One of the SDK engineers here at Applitools, Noam, breaks this down into a hands-on tutorial, hoping it will help you get a better understanding of the industry’s best practices around visual testing of rich and dynamic UI experiences.

Let’s dive right in!

Framework Native Solutions

Most frameworks already have different mechanisms to handle animations and loading artifacts. Keeping things simple is often the best way to achieve code stability and maintainability. Using your framework’s built-in tools would most often be the best approach.

For example:

Playwright JS

// Playwright: wait for spinner to be removed
await page.waitForSelector('.spinner', { state: 'detached' });
await eyes.check()

Cypress

// Cypress: wait for spinner to not exist
cy.get('.spinner').should('not.exist'); 
cy.eyesCheck()

Selenium JS

// Selenium: wait for spinner to be invisible
const spinnerElements = await driver.findElements(By.css('.spinner'));
if (spinnerElements.length) {
  await driver.wait(until.elementIsNotVisible(spinnerElements[0]), 5000);
}
await eyes.check()

A Common Pitfall

Even if the UI appears visually unchanged, frontend frameworks like React, Vue, and Angular may re-render elements under the hood. This can lead to stale element references, especially when capturing regions right after a DOM change.

Consider the following example:

cy.get('.main').then($el => {
  cy.get('.spinner').should('not.exist'); // spinner disappears after main was located

  cy.eyesCheckWindow({
    tag: 'main',
    target: 'region',
    element: $el, // stale reference if .main was replaced
  });
});
  1. First, Cypress locates .main
  2. Then, Cypress waits for the spinner to disappear
  3. This example would fail (even if the new element has the exact same properties) if the main element is replaced by another element

How to avoid that?

When possible (e.g., Playwright), it’s preferred that you use locators instead of selectors. If you can’t, it’s better if you use selectors instead of DOM references (element: ‘.main’).

Videos, CSS Animations, GIFs

There are many techniques to eliminate other types of dynamic behaviour in web pages. Playwright, for example, provides a Clock API that allows pausing JavaScript time-related events (including JS-driven animations). It’s also possible to install custom CSS snippets to pause and reset CSS-related animations. Other JS specialized crafted snippets would be required for resetting GIFs, videos, and so on – you get the idea.

This never-ending cat-and-mouse game can be prevented by using Applitools Ultrafast Grid (UFG). Instead of rendering web pages on locally executed browsers, the UFG team maintains specialized logics and fine-tuned commands that ensure a stable and consistent rendering experience. While UFG offers more than just rendering stability, it’s worth noting that classic screenshots can still achieve stable results. UFG just makes it easier!

Algo-Based Solutions

If you intentionally want to capture dynamic content (e.g., animations, changes), a smarter strategy is to embrace that variability and use smart matching algorithms to compare just what you need, like those found in Applitools Eyes.

Any match level can be used for the entire screenshot or specific regions of the screen. Read more in the Match Level Best Practices tutorial. For example, algorithms like the Layout match level can drastically improve your experience with localization testing.

The waitBeforeCapture Setting

Performing wait operations can become more complicated when:

  1. Testing with no-code visual testing SDKs (e.g eyes-storybook)
  2. Testing with advanced Eyes features like lazyLoading and layoutBreakpoints

The waitBeforeCapture setting was invented for these types of use cases (and a few others).

This setting can receive three types of arguments:

  1. Milliseconds – the simplest approach. While it’s not always the most innovative or sophisticated pattern, in many cases, it “does the trick.” In general, waiting for explicit timeouts during tests is not recommended. However, when compared to clock manipulations and code injections, sometimes the simplicity and stability is worth the longer run-time.
  2. Selector – when we’re waiting for something to appear, most SDKs support passing a selector, and Applitools Eyes will automatically wait for an element that matches this selector to appear in the web page.
  3. Custom function – see code example
// eyes-storybook
  waitBeforeCapture: async () => {
    while (document.querySelector('.spinner')) {
      await new Promise((resolve) => setTimeout(resolve, 100))
    }
    return true;
  }

// eyes-playwright
await eyes.check({
  name: 'my-step',
  async waitBeforeCapture() {
    await page.locator('.spinner').waitFor({state: 'hidden'})
  },
})

The waitBeforeCapture setting can be defined in your applitools.config file, in your eyes.check settings, using the Target settings builder, and in other similar places. Please refer to the documentation of the specific SDK you’re using for concrete examples.

Storybook Play Functions

A nice eyes-storybook-specific trick to achieve a desired rendering state would be a Storybook Play Function.

Applitools Eyes will run your play functions and wait for them to finish before capturing anything on the screen. Use Play Functions to navigate the story to an interesting state and wait for the story to be stable inside the play function to help Eyes understand what the best time is to capture the screenshot.

Applitools is Here to Help

We hope you’ve found this article interesting, and maybe it solved some of the most common visual testing issues you may have encountered. Go ahead and try these examples out for free with Applitools Eyes.

However, if something isn’t clear or if you’d like advice regarding the best way to incorporate visual testing into your organization, please don’t hesitate to reach out to our experts! Testing is our passion, and we’re here to help.

Quick Answers

What are “loading artifacts” in visual testing, and how do I avoid flaky tests?

Loading artifacts are transient UI elements, like spinners, skeleton cards, GIFs, that appear while data is fetched. If a screenshot is captured before they disappear, your baseline image won’t match future renders, causing false failures (flaky tests).

Why do I get “stale element reference” errors after a React/Vue/Angular re‑render?

Modern frameworks often replace DOM nodes even when the UI looks identical. If you save a DOM reference (e.g., cy.get('.main')) before waiting for the spinner to vanish, that reference may point to a removed element, causing stale errors. Capture by selector or locator, not by saved element handles, to avoid this.

What is the waitBeforeCapture setting in Applitools, and what values can it accept?

waitBeforeCapture delays the screenshot after the DOM is stable. It accepts:
Milliseconds (e.g., 500)
CSS selector to wait for element presence/absence
Custom async function for complex logic (e.g., loop until .spinner hidden)

Can I use Storybook Play Functions to control the render state before visual testing?

Yes. In eyes‑storybook, Applitools runs each story’s Play Function and waits for it to finish—perfect for clicking buttons, filling forms, or pausing animations before the snapshot.


Is it better to fast-forward the JavaScript clock or add explicit waits for CSS animations?

Fast-forwarding the JS clock (e.g., page.clock.fastForward(1000) in Playwright) is usually more reliable and efficient than using hard timeouts. It advances timers without waiting in real time, making tests faster. However, it won’t affect CSS-driven animations since those still require CSS overrides to pause or skip transitions. For full stability, combine clock control with style injections or use Applitools Ultrafast Grid, which auto-handles CSS animations under the hood.

The post Handling Animations and Loading Artifacts in Visual Testing appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Leveraging Applitools for Seamless Visual Testing in Playwright https://app14743.cloudwayssites.com/blog/leveraging-applitools-for-seamless-visual-testing-in-playwright/ Fri, 31 Jan 2025 21:29:23 +0000 https://app14743.cloudwayssites.com/?p=59583 As applications become more complex and UI consistency becomes critical, ensuring that the user interface appears as expected across multiple environments is key. Applitools Eyes, when integrated with the Playwright...

The post Leveraging Applitools for Seamless Visual Testing in Playwright appeared first on AI-Powered End-to-End Testing | Applitools.

]]>

As applications become more complex and UI consistency becomes critical, ensuring that the user interface appears as expected across multiple environments is key. Applitools Eyes, when integrated with the Playwright SDK, provides a powerful, efficient, and streamlined approach to visual testing in your Playwright tests. The combination of Playwright, a popular end-to-end testing framework, and Applitools Eyes, a visual AI-powered testing tool, makes visual validation easier, faster, and more scalable.

Applitools Eyes is a visual AI-powered testing tool that helps address these UI consistency challenges. It uses advanced AI-driven image comparison to detect visual differences. Unlike traditional visual testing tools that perform pixel-by-pixel image comparisons, Applitools Eyes uses AI algorithms to determine if the visual differences are actual bugs.

Applitools Playwright SDK introduces several improvements designed to streamline the process of setting up and running visual tests, making the experience more efficient and user-friendly.

In this blog, we will discuss when it is appropriate to use visual testing with Playwright and which cases will be suitable for using Applitools.

Overview of Visual Testing

Visual testing is a technique performed to ensure the visual appearance of a given website or application matches the provided design and layout . This includes the comparison of the actual UI against a standard image or a reference UI image. This type of testing primarily focuses on detecting any visual anomalies, which include alignment issues , font problems, color discrepancies , images, or structural shifts that can disrupt the user experience.

How Visual Testing Differs from Functional Testing

Visual testing and functional testing are both crucial if the goal is to deliver a high-quality application; visual testing checks the visual appeal of the application, while functional testing verifies how the application functions.

Let’s see some major differences between visual and functional testing.

AspectVisual TestingFunctional Testing
PurposeEnsures the layout and design of the UI look as expected.Verifies that the application’s functions work as intended.
ScopeFocuses on UI elements’ appearance (position, dimensions, colors, fonts, etc.).Focuses on the application’s behavior, verifying logic, workflows, and system responses.
ApproachUses image comparison tools to detect visual discrepancies.Uses test scripts or user simulations to validate functionality.
TechniquesRelies on image or screenshot comparison against baseline images.Involves input/output verification, user interaction simulation, and data validation.
ToolsTools like Playwright, Cypress, and other functional testing frameworks.Tools like Playwright,Cypress, and other functional testing frameworks.
Issues DetectedIdentifies layout errors, pixel misalignments, broken images, wrong colours, or responsive design bugs.Identifies logical errors, broken workflows, malfunctioning features, or incorrect data processing.
Use CaseSuitable for detecting unintended UI changes during development.Suitable for validating features and ensuring correct application logic.

Before delving into how we can perform visual testing using Playwright and Applitools, let’s review some of the new improvements that Applitools has introduced .

What’s New In Applitools Playwright SDK?

The Applitools Playwright SDK is a library that integrates Applitools’ visual testing capabilities with the Playwright automation framework.

Let’s explore the improvements done by Applitools to streamline the process of setting up and running visual tests, making the experience more efficient and user-friendly. 

Here’s a detailed breakdown of what’s new in the updated SDK:

Test Fixtures:

  • Test fixtures Centralize and reuse setup code across multiple tests, making the setup for visual tests more consistent and reducing redundant configuration.
  • This feature is particularly valuable when running tests on multiple pages or scenarios, as it simplifies the initialization of Applitools Eyes in each test.

CLI Onboarding:

  • The Command-Line Interface (CLI) simplifies the initial setup process by automating configuration steps, helping users quickly integrate Applitools Eyes with Playwright. 
  • This is especially helpful for new users, reducing the complexity of the setup and enabling faster onboarding into visual test creation.

Config Object Setup:

  • The configuration object setup automates the insertion of Applitools Eyes settings into the Playwright configuration file (e.g., playwright.config.js or playwright.config.ts). 
  • This feature eliminates manual setup, reducing the risk of errors, ensuring accurate configuration from the outset, and saving developers valuable time.

Custom HTML Reporter:

  • The custom HTML reporter enhances Playwright’s default test report by integrating visual test results from Applitools Eyes. 
  • This allows developers to view a direct comparison between the baseline image and the current test screenshot, helping identify visual regressions more effectively. 
  • By adding visual results to the standard Playwright report, it simplifies the review process, offering a comprehensive overview of the test outcomes.

Before we deep dive into how Applitools helps in visual testing, let’s first see how we can perform visual testing using Playwright.

Visual Testing Using Playwright

Playwright comes equipped with tools to make visual testing effortless, allowing developers to take and compare screenshots of web pages or specific web page elements. 

Here’s how visual testing works in Playwright:

  1. First Run: Playwright captures and saves a screenshot of the page or specific UI elements. This is known as the base image.
  2. Subsequent Runs: In the next run, Playwright takes a new screenshot and compares it against the base image.
    • If there are no differences, the test passes.
    • If differences exist, the test fails, flagging the areas where changes occurred.

Prerequisites

Ensure the following tools are installed:

Node.js: Download and install from Node.js.

Visual Studio Code (Optional): Recommended IDE for coding.

Install Playwright:

npm i @playwright/test

Write the Simple Visual Regression Test Using Playwright
Create a new JavaScript file in your test folder, e.g., demo.spec.js.

Use the page.screenshot() method to capture the screenshot and the expect.toMatchSnapshot() assertion from the @playwright/test module to compare images.

const { test, expect } = require('@playwright/test'); 
test('Visual Regression Test Example', async ({ page }) => { 
   // Navigate to the website 
   await page.goto('https://playwright.dev'); 
   // Capture a screenshot of the page 
   const screenshot = await page.screenshot(); 
   expect(screenshot).toMatchSnapshot(); 
}); 

When we execute the above code for the first time, the test case fails with the below error, and Playwright captures and saves a screenshot of the page or specific UI elements. This is known as the base image.

When we execute the same test case again, it will pass.

Visual Regression Test with maxDiffPixels option 

There might be the case when there are some minor differences in the images and the test case starts failing; to handle this situation we have an option, the ‘maxDiffPixels option,’ that we can pass. It allows you to specify the maximum number of pixels that can differ between two images for the comparison to still be considered successful.

const { test, expect } = require('@playwright/test');
test('Visual Regression Test Example', async ({ page }) => {
  // Navigate to the website
  await page.goto('https://playwright.dev');
  // Capture a screenshot of the page
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchSnapshot({ maxDiffPixels: 100 });
}); 
const { test, expect } = require('@playwright/test');
test('Visual Regression Test Example', async ({ page }) => {
  // Navigate to the website
  await page.goto('https://playwright.dev');
  // Capture a screenshot of the page
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchSnapshot({ threshold:0.5} );
}); 

Visual Regression Test with Threshold option

Passing ‘threshold’ as an option is another option to avoid failing the test case. This is particularly useful for handling slight differences that occur due to rendering variations.

const { test, expect } = require('@playwright/test');
test('Visual Regression Test Example', async ({ page }) => {
  // Navigate to the website
  await page.goto('https://playwright.dev');
  // Capture a screenshot of the page
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchSnapshot({ threshold:0.5} );
}); 

So far we have seen how we can automate the visual UI using Playwright. In the next section you will see how we can use Applitools Eyes and Playwright SDK together to automate visual testing.

Visual Testing using Applitools Eyes and Playwright SDK

Visual testing is a crucial aspect of modern software quality assurance. It ensures that a web application’s UI renders correctly across different devices, screen sizes, and browsers. Tools like Applitools Eyes and frameworks like Playwright SDK simplify this process by providing robust solutions for automated visual regression testing.

Below are the steps to set up Applitools Eyes and Playwright SDK.

Install Playwright

npm init playwright@latest

Install Applitools Eyes SDK 

npm install -D @applitools/eyes-playwright

Add the below line in .spec.js file 

To configure your project for Applitools Eyes, we have to add the below line.

import { test } from '@applitools/eyes-playwright/fixture';

In the next section you will see examples to explain how we can use SDK with Playwright tests. 

Before moving in the detail of various types of matchLeavel, let see one basic example to understand how we do the visual testing in Applitool,  and the method used in Applitools, to capture and validate a checkpoint in your application’s UI.

Below is code that integrates Applitools Eyes with Playwright to perform a visual test.

import { test } from '@applitools/eyes-playwright/fixture';
test('My first visual test Using Applitools with matchLevel: Dynamic', async ({ page, eyes }) => {
await page.goto('https://app14743.cloudwayssites.com/helloworld/');
  // Visual check
  await eyes.check('Landing Page', {
    fully: true,
    matchLevel: 'Dynamic',
  });
});

In the above code we mainly use two things: one is the method eyes. check() and two options: {fully: true}, and {matchLevel: ‘Dynamic’}

eyes.check(): Performs a visual snapshot of the page or a specific element and compares it with the baseline image stored in Applitools.

Options:

  • fully: true:
    • Captures the entire page, including scrollable sections.
    • Ensures that all content on the page is included in the visual test.
  • matchLevel: ‘Dynamic’:
    • A matching algorithm that focuses on the content while ignoring dynamic changes, such as text or layout differences.
    • Useful for pages with frequently changing content, like user-generated text or dynamic data.

Below are the options that we can pass in our test case. These options provide flexibility and precision, enabling robust visual testing tailored to different scenarios and challenges.

This table summarizes each configuration, making it easier to understand their purposes.

OptionPurposeExample Use Case
matchLevel: ‘Dynamic’Ignores minor layout or content changes (e.g., dynamic text, animations).Testing pages with frequent dynamic content (e.g., dates, real-time updates) without raising false alarms.
matchLevel: ‘Strict’Detects even small pixel or layout changes for precise visual validation.Validating critical UI components or designs where every pixel matters (e.g., brand logos, product pages).
region: componentFocuses the visual check on a specific component or region of the page.Testing a new or modified UI component in isolation (e.g., buttons, headers, or form fields).
fully: trueCaptures and validates the entire page, including content outside the viewport.Ensuring the layout and content of a long webpage or scrolling section is consistent.
ignoreRegions: [dynamicContent]Excludes specific regions with dynamic content from the validation.Ignoring areas like ads, live feed sections, or widgets with fluctuating content (e.g., real-time graphs).

Examples with different matchLevel

MatchLevel determines how the captured visual output is compared with the baseline image.Let’s see the examples with different options.

1. Visual Testing with matchLevel: Strict

This ensures pixel-perfect comparison between the baseline and current screenshot, highlighting any visual difference, no matter how small.

test('My first visual test Using Applitools with matchLevel: Strict', async ({ page, eyes }) => {
  await page.goto('https://app14743.cloudwayssites.com/helloworld/');
  await eyes.check('Landing Page', {
    fully: true,
    matchLevel: 'Strict',
  });
});
  • Purpose: Validates the entire page with Strict match level.
  • Strict Match Level: Identifies even small differences, such as minor pixel or layout changes, ensuring high precision in visual validation.

2. Visual Testing with matchLevel: Dynamic

Focuses on content consistency by ignoring minor layout differences, such as text movement, while ensuring key visual elements remain unchanged.

In the below example you can see even if the text color is changed after clicking on link ‘?diff2’ test case still works perfectly fine because we have set matchLevel: Dynamic. This ensures the visual test focuses on the content and structure of the text rather than superficial changes like color.

test('My first visual test Using Applitools with matchLevel: Dynamic', async ({ page, eyes }) => {
 await page.goto('https://app14743.cloudwayssites.com/helloworld/');
 await page.getByRole('link', { name: '?diff2' }).click();
 await eyes.check('Homepage', {
   fully: true,
   matchLevel: 'Dynamic',
 });
});
  • Purpose: Validates the entire page with Dynamic match level.
  • Dynamic Match Level: Ignores minor content/layout differences (e.g., text changes, animations) and focuses on high-level structural comparisons.
  • fully: true: Captures a screenshot of the full page, not just the visible viewport.

3. Visual Testing for a Specific Region/Component

Limits visual comparisons to a particular area or component of the application.

test('My first visual test Using Applitools with particular region/Component', async ({ page, eyes }) => {
  await page.goto('https://app14743.cloudwayssites.com/helloworld/');
  const component = page.locator('.fancy.title.primary');
  await eyes.check('My Component', {
    region: component,
  });
});
  • Purpose: Tests only a specific component or region on the page.
  • region: Focuses the validation on the area defined by the component locator (.fancy.title.primary), instead of the entire page.

4. Visual Testing with Ignored Regions

Excludes specific areas from comparison, preventing false positives caused by expected dynamic changes in those regions.

test('Visual test Using Applitools with ignoring the region', async ({ page, eyes }) => {
  await page.goto('https://app14743.cloudwayssites.com/helloworld/');
  const dynamicContent = page.locator('.fancy.title.primary');
  await eyes.check('Homepage', {
    fully: true,
    matchLevel: 'Strict',
    ignoreRegions: [dynamicContent],
  });
});
  • Purpose: Tests the page while ignoring specific dynamic regions.
  • ignoreRegions: Excludes certain areas from the visual validation (e.g., areas with dynamic content like dates, ads, or animations).
  • fully: true: Captures the entire page for validation.

About dynamic match level

Dynamic match level is a feature in Applitools Eyes that verifies text by matching it against predefined or custom patterns, ensuring the content adheres to a specific format. It focuses on format validation rather than content changes, making it ideal for dynamic text like dates, emails, or numbers.

Type of dynamic match level

Dynamic match levels allow for flexible verification of text by matching it against predefined or custom patterns. The default types include:

  • TextField: Text inside input fields.
  • Number: Numeric values like ZIP codes or phone numbers.
  • Date: Validates text as a date in proper format.
  • Link: Hyperlinks or URLs.
  • Email: Checks for a valid email address format.
  • Currency: Recognizes monetary values.

These types ensure accurate validation by focusing on the format rather than the content changes, such as dynamic updates to dates or numbers.

Custom dynamic pattern

To create a custom dynamic pattern we have to follow below steps in Applitools dashboard 

Here are the steps to create a custom dynamic pattern:

  1. Open the Settings Window: In the Page Navigator, select Apps & Tests.

In the list of applications on the left, hover over an application and click > Settings.

  1. Add a Custom Type: Click Add custom type.

  1. Define the Custom Pattern:

Enter a name and a Regex pattern.

  1. Save the Custom Pattern:

In the below screenshot you can see we have set a custom pattern for Zip Code. Once the custom pattern is created, its name can be used in the code to reference the pattern.

Click Add will save the entered pattern. 

  1. Apply the Settings:

Once all required patterns have been selected, click Apply settings.

Applitools Dashboard: Execute visual testing using the Applitools Playwright SDK

Applitools’ Dashboard is a powerful interface designed to manage and analyze test results efficiently, especially for visual testing and monitoring user interfaces.

Below are the steps that we normally have in the Applitools Dashboard when we execute any visual test case.

Precondition : Run the below command to export the key 

export APPLITOOLS_API_KEY='your_api_key_here'

Once the key is exported, the next step is to execute the test cases using the below command.

npx playwright test --ui tests/visual.spec.js

First Run: Playwright captures and saves a screenshot of the page or specific UI elements. This is known as the base image.

Subsequent Runs: In the next run, Playwright takes a new screenshot and compares it against the base image.

  • If there are no differences, the test passes.
  • If differences exist, the test fails, flagging the areas where changes occurred.

In the below screenshot in the Applitools Dashboard, you can see all the above four test cases are executed successfully.

Applitools Playwright SDK with Example in Detail

Let’s take an example of a particular component in the page. In the below screenshot, the component focuses on a specific component or region of the page. You can see only the ’Hello World’ part is tested. In case there is any change in this component, the test case will fail.

In the below example we have updated the component with the text ‘Happy World’ 

test('Visual test Using Applitools with Updating Component UI', async ({ page, eyes }) => {
await page.goto('https://app14743.cloudwayssites.com/helloworld/');
await page.getByRole('link', { name: '?diff2' }).click();
await eyes.check('Homepage', {
  fully: true,
  matchLevel: 'Strict',
 });
});

When we execute the above code, it will not execute successfully because the UI of the component is changed after clicking on the link ‘diff2.’

To fix the above problem, we have marked it resolved from the Applitools Dashboard. Once we accept the change, it becomes the base image.

Now when we execute the test case again, it gets executed successfully with the updated UI.

When Playwright’s Visual Testing Features Are Useful 

Playwright’s built-in visual testing features are well-suited for scenarios where:

Simple Visual Comparisons:

  • Comparing entire pages or specific UI components with baseline images for detecting pixel-level differences.
  • Example: Ensuring the homepage layout hasn’t shifted after a CSS update.

Static UIs:

  • Testing applications with minimal dynamic or animated content, as these are less prone to false positives caused by animations or transient elements.

Fast Feedback:

  • When quick pixel-by-pixel comparisons in CI pipelines are sufficient without advanced AI-powered analysis.
  • Example: Smoke testing in a fast-moving development cycle.

Budget-Friendly:

  • Playwright’s native screenshot and comparison capabilities don’t require additional tools or subscriptions.

When Playwright’s Visual Testing May Fall Short

Dynamic Content:

  • UIs with dynamic or frequently changing elements (e.g., time, randomized content, or animations) can cause false positives due to exact pixel mismatches.
  • Example: A dashboard with live-updating graphs.

Complex Visual Changes:

  • Playwright’s threshold-based comparison may miss subtle or semantic changes that don’t exceed the defined pixel difference threshold.
  • Example: Slightly altered typography or colour changes.

Ignored Regions:

  • While Playwright allows element-specific testing, ignoring specific dynamic regions within a larger page requires custom logic.
  • Example: Excluding advertisements or live widgets from visual comparison.

How Applitools Handles Situations Better

Applitools offers superior visual testing capabilities compared to Playwright, especially when handling dynamic content, ensuring pixel-perfect validation, focusing on specific components, and reducing false positives. 

Dynamic Content Handling:

  • Match Levels: Applitools’ AI-powered matchLevel options (e.g., Strict, Layout, Content) allow intelligent comparison by focusing on structure and ignoring irrelevant pixel differences.
  • Example: Testing a real-time dashboard with dynamic data but a consistent layout.

Ignored Regions:

  • Applitools allows you to define ignored regions to exclude specific parts of a page from comparison (e.g., ads or timestamps).
  • Example: Excluding a “current time” widget from visual testing.

Advanced Visual Validation:

  • Features like dynamic regions and AI-based semantic analysis help detect visual regressions that go beyond pixel-by-pixel differences.
  • Example: Identifying a button misalignment that might not be detected by Playwright’s threshold settings.

Visual Testing Across Components:

  • Allows you to target specific regions or components for testing without manually cropping or capturing screenshots.
  • Example: Testing the styling of a navigation bar or modal dialog.

Conclusion

Playwright’s built-in visual testing features are suitable for simple cases where you need pixel precision in image comparisons of static UIs. However, it may struggle with complicated designs, whether they involve minor changes within the site or more intricate design requirements .

Applitools Eyes, which employs artificial intelligence, is more suitable for overcoming these difficulties. It performs extraordinarily in cases with dynamic content; it easily identifies subtle visual regressions and is equipped with extra functionalities such as ignored areas and cross-browser testing. For teams with significant testing and enhancement requirements, where the built-in visual testing capabilities of Playwright may fall short for complex and large-scale application testing, Applitools serves as an invaluable complement. It ensures and elevates the visual quality of dynamic applications, making it an ideal choice for robust testing and quality assurance.

About the Author

Kailash Pathak (Applitools Ambassador | Cypress Ambassador)

Senior QA Lead Manager with over 15 years of experience in QA engineering and automation. Kailash holds certifications including PMI-ACP®, ITIL®, PRINCE2 Practitioner®, ISTQB, and AWS (CFL).

As an active speaker and workshop conductor, Kailash shares his expertise through blogs on platforms like Medium, Dzone, Talent500, The Test Tribe, and his personal site https://qaautomationlabs.com/


Quick Answers

How do I add Applitools Eyes to an existing Playwright project?

Follow the Playwright Quick Start to install the SDK, run CLI setup, and execute your first visual test (https://app14743.cloudwayssites.com/tutorials/playwright).

What’s the difference between Playwright functional checks and visual testing?

Functional asserts confirm behavior (clicks, responses, state), while visual testing validates the rendered UI—catching regressions in layout, fonts, colors, and dynamic content that functional checks overlook.

How do I run cross-browser visual tests quickly in CI?

Use the Ultrafast Grid to fan out visual checkpoints across real browsers/devices in parallel without maintaining an in-house grid (https://app14743.cloudwayssites.com/ultrafast-grid).

How do I reduce flakiness in Playwright tests?

Favor stable visual checkpoints over brittle locator assertions, use consistent viewports and network conditions, and rely on Visual AI to ignore non-material diffs (https://app14743.cloudwayssites.com/platform/validate/visual-ai/).

The post Leveraging Applitools for Seamless Visual Testing in Playwright appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
How to Use the Updated Applitools’ Playwright SDK for Efficient Visual Testing https://app14743.cloudwayssites.com/blog/how-to-use-the-updated-applitools-playwright-sdk-for-efficient-visual-testing/ Tue, 12 Nov 2024 21:12:13 +0000 https://app14743.cloudwayssites.com/?p=58838 Efficient, reliable visual testing is essential for teams working with Playwright. Applitools’ updated Playwright SDK simplifies integration, reduces setup time, and offers more powerful reporting. Let’s check out the new...

The post How to Use the Updated Applitools’ Playwright SDK for Efficient Visual Testing appeared first on AI-Powered End-to-End Testing | Applitools.

]]>

Efficient, reliable visual testing is essential for teams working with Playwright. Applitools’ updated Playwright SDK simplifies integration, reduces setup time, and offers more powerful reporting. Let’s check out the new features and how they streamline your testing.

What’s New in Applitools Playwright SDK?

The updated SDK introduces improvements that let you set up and run visual tests more efficiently:

  • Test Fixtures: Minimizes repetitive setup code
  • CLI Onboarding: A new CLI that speeds up initial setup, and an example Visual AI test
  • Config Object Setup: Auto-inserts Eyes configuration in playwright.config.js/ts
  • Custom HTML Reporter: Extends Playwright’s report to include Eyes checkpoint images

Join the Playwright Visual Testing Best Practices webinar led by software architect and international speaker Cory House. Discover best practices for visual testing in Playwright to enhance UI consistency, swiftly resolve issues, and ensure seamless user experiences across platforms.

Using Test Fixtures for Simplified Visual Testing

Test fixtures streamline visual testing by automating many of the functions necessary for setup. The SDK now automatically handles Eyes.open() and Eyes.close() functions as well as the collection of all test results. With these automatic integrations, you can focus directly on creating test cases without the need for redundant code.

Playwright SDK test fixtures screenshot

Enhanced Reporting with the Custom HTML Reporter

The HTML reporter is designed to make analyzing test results more intuitive. It integrates Eyes checkpoint images directly into Playwright’s report. You can approve or reject baselines all within the report without having to navigate to the Eyes dashboard. This enhanced reporting setup provides clear visual insights that make it easy to understand test outcomes without extra steps.

Playwright HTML report with side by side results screenshot

Advanced API Usage for Custom Testing

The SDK also supports advanced API usage for complex testing needs. This lets you create custom configurations and expand test cases for specific UI components. As your project grows, this flexibility enables you to scale visual tests seamlessly and ensure thorough visual validation across different scenarios.

Backward Compatibility and Migration Tips

The SDK also maintains backward compatibility, making it easy to transition from the previous SDK while retaining existing configurations. You are welcome to start running a few tests in both SDKs to verify functionality and gradually implement the new SDK on simpler tests before migrating critical ones.

Getting Started with the Playwright SDK

Getting started with the SDK is straightforward. First, install the SDK with npm install @applitools/eyes-playwright. Next, use the CLI tool to enter your API key with npx eyes-playwright setup, automatically updating the environment. The SDK sets up configurations and imports for you, even adding a Visual AI demo test in Applitools Eyes that you can execute right away.

The latest Applitools Playwright SDK provides a faster, simpler, and more insightful visual testing experience. Download the SDK, try the demo test, and see firsthand how it can enhance your Playwright testing workflow.

The post How to Use the Updated Applitools’ Playwright SDK for Efficient Visual Testing appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Easier Test Maintenance for Dynamic Content with Applitools Eyes 10.20 https://app14743.cloudwayssites.com/blog/easier-test-maintenance-for-dynamic-content-with-applitools-eyes-10-20/ Mon, 21 Oct 2024 16:08:16 +0000 https://app14743.cloudwayssites.com/?p=58066 Applitools knows that you know how crucial it is to deliver flawless user experiences without spending time managing unnecessary test failures or constant updates.  The latest Applitools Eyes release introduces...

The post Easier Test Maintenance for Dynamic Content with Applitools Eyes 10.20 appeared first on AI-Powered End-to-End Testing | Applitools.

]]>
Eyes Dynamic Match Level

Applitools knows that you know how crucial it is to deliver flawless user experiences without spending time managing unnecessary test failures or constant updates. 

The latest Applitools Eyes release introduces new features designed to make your visual testing more efficient and faster. These features will make it easier to manage your tests, reduce manual intervention, and improve your test coverage–whether you’re part of a QA team, a developer, or a test engineer. 

Let’s dive into the top three features of Applitools Eyes 10.20.

Dynamic Match Level

Smart Handling of Variations

With Applitools Eyes 10.20, managing visual test results just got easier thanks to the new dynamic match level. This feature allows users to specify certain patterns—like dates, numbers, or email addresses—that can change without triggering test failures. It identifies specific text patterns that commonly vary, such as timestamps or transaction IDs, and ignores them during test execution.

Eyes 10.20 Dynamic Match Level

By default, the system provides patterns like dates, phone numbers, and currency values, but users can also create custom patterns using RegEx. This level of flexibility is especially beneficial for those of you dealing with extremely dynamic content, such as real-time stock market data or dynamic ad targeting, where some elements should not cause a test to fail.

Example Use Case: Insurance policy numbers often follow specific formats but vary between users or transactions. During visual testing, these dynamically generated numbers can trigger false positives. With the new dynamic match level, you can now ignore the variability in policy numbers while still validating the rest of the content and UI.

For instance, a health insurance company has an online portal where customers can view their policies. Each policy is identified by a unique policy number formatted as “POL-XX-YYY” where “XX” and “YYY” are randomly generated for each customer. Testing tools that cannot handle identifying content patterns would cause visual tests to fail unnecessarily every time the policy number changes. However, using Dynamic Match Level lets this health insurance company’s visual testing ignore the “POL-XX-YYY” format in tests ensuring the system validates only significant UI changes.

Dynamic Region Selection

Precision with Less Effort

In addition to manually drawing regions, visual validation can now be anchored directly to DOM elements. As the page content changes now, Applitools Eyes adapts the validation region, ensuring tests don’t break due to layout shifts or content variations. Dynamic region selection allows you to set visual test regions by simply selecting elements on a web page instead of manually drawing regions, saving you time and improving accuracy.

The new dynamic method uses the underlying DOM structure, allowing for automatic region adjustments as the page content changes. If the layout or size of the elements shift, Applitools Eyes automatically adapts to the change without breaking the tests.

Why it matters: This feature is particularly useful for those of you who have apps with extremely dynamic content, where visual elements often change. Enterprise SaaS to media outlets will see the benefits of using dynamic region selection since they no longer have to worry about small shifts in UI elements, reducing the need for constant manual updates and cutting down test maintenance time significantly.

Insights Dashboards

Quantify Your Testing Success

One of the more analytical features added to Applitools Eyes 10.20 is the new Insights Dashboards for testing ROI, which provide insights into the effectiveness of your automated maintenance efforts. These dashboards display critical data on how much time you’ve saved using Applitools’ automated visual testing and maintenance tools.

The dashboards break down the number of visual validation results automatically accepted by Applitools and track the time saved through automated maintenance. This is based on the assumption that manually reviewing a page takes about two and a half minutes. By quantifying this, the dashboard helps you understand the real-world impact of automating your visual testing workflow.

Eyes 10.20 Insights Dashboard

Pro tip: With Insights Dashboards, you can highlight specific wins, such as reducing manual test reviews or cutting test maintenance time. If you notice test maintenance time decreasing steadily, it’s a good sign your visual testing strategy is becoming more efficient. This data can be shared during quarterly business reviews with leadership or when advocating for further automation resources.

Additional Improvements

The 10.20 release also introduces faster load times on the “Tests Requiring Attention” page—especially beneficial for teams working with large test suites. The enhanced GitHub integration means you can track baseline merges directly within the baseline history page, minimizing manual effort and improving workflow efficiency. Visit the Applitools Eyes 10.20 release notes to see everything that’s included.

To Wrap Up…

One of the biggest hurdles in the day-to-day work for anyone involved in QA is dealing with flaky tests caused by constantly changing text, like dates or order numbers, and shifting UI elements, like buttons that move slightly depending on screen size. Your users expect flawless experiences. 

The Applitools Eyes 10.20 release is packed with features that simplify and streamline the visual testing process, from dynamic region selection to enhanced match levels and improved performance. These updates will help your team reduce manual test maintenance, increase test coverage, and measure the ROI of your testing efforts more effectively. Whether you’re managing complex web applications or handling high volumes of test data, these features are designed to make your workflow faster and more efficient.

The post Easier Test Maintenance for Dynamic Content with Applitools Eyes 10.20 appeared first on AI-Powered End-to-End Testing | Applitools.

]]>