Learn the best way to scale mobile automation testing today. We’ll look at a history of traditional mobile automation testing strategies and discuss why each one is and isn’t effective, and then see why a new tool, the Native Mobile Library, is different.

In today’s mobile testing world there are many different approaches to scaling our test automation for native mobile applications. Your options range from running locally with virtual devices (simulators/emulators) or real devices, to a local mobile grid/lab, to docker containers/virtual machines, to remote cloud test services. 

As you’re probably aware, testing native mobile applications can sometimes be a difficult endeavor. There are a lot of moving parts and many points of failure involved. To successfully execute, everything needs to work in complete harmony.  

For example, just executing a single Appium test you need:

• An Appium server & all required dependencies installed
• A mobile device or emulator/simulator
• Valid test code logic
• A compiled mobile application 
• Application web service APIs running and stable (if applicable)

Now let’s say you want to scale your tests across multiple devices for your cross-device validation needs. We’re now introducing more points of failure for each device that is tested. A test on one device may execute just fine but on another, it may fail for various unknown reasons. We then often have to spend a vast amount of time investigating and debugging these failures to find the root cause.

Let’s also consider that because we’re adding more devices we will likely need to add more conditional logic to our test code to accommodate these different devices. We may need to add conditionals for different device resolutions/screen sizes, OS versions, orientations, scroll views, locators/selectors, or maybe gestures for specific devices. This all adds more coded logic to our test suite or framework we need to maintain and thus refactor in the future when our application changes. 

Because of these reasons and/or perhaps others, many people often don’t scale their mobile test coverage across different devices. Either it might introduce more test maintenance, more test flakiness, a longer test execution time, or access to different devices is not possible. Crossing fingers and hoping for the best has been the approach for many…

Scaling Mobile Automation Testing

Now let’s cover some common mobile test scaling approaches. I’ll go over the benefits and drawbacks of each of these approaches and finally introduce you to a new modern approach and technology, the Applitools Native Mobile Library.

Sequential Execution

The simplest but also the most inefficient and slowest approach. The example diagram below shows executing two tests (Test A & Test B) executing across three different mobile devices.

As stated before, this is the simplest approach but very inefficient and slow. Perhaps some people aren’t aware of different or better approaches. Or maybe some bad test practices are being used such as one test (Test B) is dependent on (Test A) to complete in order to proceed. Generally, a good test practice is to never have any tests dependent on another to execute. Test suites/frameworks should be architected in a way to run any test in any order by utilizing test hooks/annotations, database seeding, mocking, or API test data injection to set up each test independently. 

For example, say I have an app that has a shopping cart and I have a test “Add items to shopping cart” which failed due to a bug in the app. If I relied on that test to then run my “shopping cart” specific tests, I couldn’t. That is where good test architecture comes into play by setting up each test independently by some of the methods mentioned above or other means.

Parallel Device Execution

We live in a fast-paced CI/CD world these days, and feedback loops that are as fast as possible are crucial. So what can you do to help get this test feedback ASAP? Parallelization! 

In this scenario, we are parallelizing the devices needed for cross-device validation. It’s however still inefficient since the tests themselves are running sequentially. But still much better than the former sequential approach.

Parallel Test Execution

This approach to mobile automation testing is a bit more efficient as it’s parallelizing the tests in your test suite. Ultimately it should reduce the execution time and also promote good test practices keeping tests independent from one another to execute. However, the inefficiency now is that each device is sequentially tested. 

Parallel Test & Parallel Device Execution

This is the most efficient and fastest traditional approach as it parallelizes both your test suite and devices for cross-device validations. However, it comes at a cost of machine resources, cloud concurrency or minutes usage on license limits (if applicable), and added complexity in your test framework having to manage different sets of parallelization. Some frameworks have this architecture “baked” into them but for many others, it’s up to the developer/tester (or whomever) to implement this logic themselves.

With that said, all of the approaches above come with but are not limited to some of the challenges below:

• Susceptible to continual dependency and versioning conflicts. 
• Any single device (or devices) either locally or on a cloud test service can have issues at any given time resulting in test flakiness.
• Network or latency timeout issues can occur.
• Service crashes or issues occurring on any running parallel process or thread.
• Added test code complexity for parallelization.
• Added test code conditionals to accommodate different device form factors/resolutions and application layouts. 

Applitools Native Mobile Library

Now that we’ve talked about the typical traditional approaches to mobile automation testing, let’s talk about the next generation of mobile cross-device testing using the Applitiools Native Mobile Library! Our Native Mobile Library (NML) uses a new technological approach to asynchronously validate your native mobile application in parallel and easily across many different devices in a single execution. 

What this means is the parallelization of devices is handled on the Applitools NML. Since it’s asynchronous you are not waiting on the device to connect or the test results, which frees your tests to execute as fast as possible! 

Some key benefits:

• Execute your mobile tests on just one device (emulator, simulator, or a real device)!
  • Supports Appium as well as XCUI, and Espresso frameworks for iOS and Android native applications. 
• Simplistic test authoring! ​​
  • Create tests with only one device and form factor/app layout in mind! You won’t need to add additional logic in your code for different devices or resolutions.
• Less code maintenance.
  • No extra coded conditionals for specific devices or supporting multiple execution environments (Local and Cloud). Tests only need to work on any single device and the environment you execute in.
  • Visually Perfect with Applitools Visual AI to perform full-page mobile UI validations. No more coded UI assertion logic!
• Fast and easy cross-device scaling without complicated multi-parallelization logic and achieve 10x faster executions compared to traditional approaches.
• Fewer points of failure.
  • No longer is there a need to execute the same test redundantly across different devices increasing test flakiness and execution time. Instead, run once on any real device or simulator/emulator and get validations across many devices.
• Fast CI/CD feedback loop.
  • Release faster with an assurance of cross-device coverage and your application UI is visually perfect. 
• No Appium version pinning.
  • Use any version of Appium that works for you and your tests.
• Secure!
  • No need to upload an application to 3rd party cloud test vendors and risk exposing sensitive proprietary data and/or information.  
  • Tests can run locally on your network without opening firewall access or proxies.

Now let’s look at some example execution architectures using the Applitools Native Mobile Grid! 

Asynchronous Parallel Device Execution

This next diagram is similar to the traditional Parallel Device Execution we talked about above. However, the parallelization is offloaded to the NMG to handle and no additional logic for parallel threads or processes is required. This approach is still not the most efficient since each test is running sequentially but it’s lightyears faster than the traditional approach.

Asynchronous Parallel Test Execution

Out of all the approaches for mobile test automation we’ve discussed thus far, this next example is by far the most superior, efficient, and fastest approach possible to scaling your native mobile test coverage! The added benefit of this approach is your entire test suite execution time will now only take as long as it takes to run your slowest test in your suite! 

This example is similar to the traditional Parallel Test & Parallel Device Execution above but the device parallelization is now offloaded to the NMG and handled asynchronously.

Code Example using Mobile Native Library for Mobile Test Automation

The below code is an example using Appium Java, testing an iOS native application. For those of you familiar with the Applitools Ultrafast Test Cloud for desktop and mobile web applications, this should look very similar. All we need to do in the test code is define the devices we want to validate for our cross-device coverage needs. In this case, we’ve specified 15 iOS devices we’ll validate in the same execution time as it takes to run the tests on just one device! Additional settings such as setting the OS version and orientation per device can be specified (not shown).

public class iOSNativeUFGTest {
   private WebDriverWait wait;
   private IOSDriver driver;
   private VisualGridRunner runner;
   private Eyes eyes;

   private static IOSDriver startApp() throws Exception {
       DesiredCapabilities capabilities = new DesiredCapabilities();
       capabilities.setCapability("platformName", "iOS");
       capabilities.setCapability("automationName", "XCUITest");
       capabilities.setCapability("deviceName", "iPhone 12 Pro Max"); //Launch a single local simulator or real device
       capabilities.setCapability("platformVersion", "15.4");
       capabilities.setCapability("app", "/Users/justin/Desktop/MyApp.app");
       return new IOSDriver<>(new URL("http://localhost:4723/wd/hub"), capabilities);
   }

   @Before
   public void setup() throws Exception {
       runner = new VisualGridRunner(new RunnerOptions().testConcurrency(15));
       eyes = new Eyes(runner);
       eyes.setApiKey(System.getenv("APPLITOOLS_API_KEY"));;

       Configuration conf = eyes.getConfiguration(); //Configure the 15 devices we want to validate asynchronously
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_11));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_11_Pro));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_11_Pro_Max));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_12));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_12_Pro));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_12_Pro_Max));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_12_mini));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_13_Pro));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_13_Pro_Max));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_XS));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_X));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_XR));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_11));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_8));
       conf.addMobileDevice(new IosDeviceInfo(IosDeviceName.iPhone_7));

       eyes.setConfiguration(conf);

       driver = startApp();
   }

   @Test
   public void iOSNativeTest() throws Exception {
       eyes.open(driver, "My Native App", "Login View"); //Start a visual test
       eyes.check("Login", Target.window().fully(true)); //Capture the mobile UI view
       eyes.closeAsync(); //End the visual test
   }

   @After
   public void tearDownTest(){
       /The /results object contains validation results on all 15 devices which can then be asserted for visual and accessibility checks
       TestResultsSummary results = runner.getAllTestResults(false); 
       eyes.abortAsync();
       driver.quit();
   }
}

Conclusion

Now for the first time ever, native mobile developers can perform continuous testing, running their entire test suites on every pull request or code push across many different devices to get immediate quality feedback at once. Just like web application developers have for many years now! 

Hopefully, this article illustrated the benefits and superiority of the Applitools Native Mobile Library to other traditional approaches for mobile automation testing. As you now can see how simple and efficient it is to expand your cross-device coverage with the Native Mobile Library there are no more excuses not to. So what’s stopping you?! 

How do you get started with the Native Mobile Library you may ask? To start using the Native Mobile Library, simply sign up at the link below to request access. You can read more about the Applitools Native Mobile Grid on our website.

 Happy testing!

The post How to Scale Mobile Automation Testing Effectively appeared first on AI-Powered End-to-End Testing | Applitools.

For anyone trying to automate repetitive and time-consuming tasks, Microsoft has just quietly released a free tool that will make your life easier. Power Automate Desktop is a codeless tool that arose out of their Softomotive acquisition a year ago. Now fully integrated into Windows, it is available for free to all Windows 10 users.

In this step-by-step tutorial, we’ll learn how to use Microsoft Power Automate Desktop to automate an application without code. We’ll even learn how to codelessly test it visually too.

Automating with Power Automate Desktop

So what can be done with Power Automate Desktop? A better question might be, what can’t you do with it? Power Automate Desktop provides many already pre-built “Actions” to automate many different processes or applications out of  the box. For example, you may want to automate sending an email triggered by a particular event. Or you may want to extract values from an Excel spreadsheet and store them somewhere else like a database on Azure. If so, Power Automate Desktop has you covered along with hundreds of other similar actions. If an action you need doesn’t exist, no problem either. It’s easy to create one – so let’s go ahead and do so! 

A Codeless Example of Power Automate Desktop

For the context of this example. Let’s say you’re a tester and you’ve been tasked to automate your Windows desktop application. Where do you start!? You could go down the path of using a framework such as WinAppDriver, which is an Appium driver built and maintained by Microsoft to automate desktop Windows applications. However, with this approach you will need some coding experience and some knowledge of the framework to really provide a viable solution. There are other enterprise options such as Micro Focus’s UFT One, formally named QTP/UFT. But this too requires some coding knowledge to be successful, along with a hefty paid subscription. 

The beauty of Power Automate Desktop is that, not only is it completely free, but it can be 100% codeless and no coding knowledge or background is required. A Web and Desktop recorder is built in to record any UI elements or views you need to interact with for your automation, which can easily be plugged into your “Flow.” A Flow in Power Automate Desktop is essentially the same thing as “automation script” in the coding world. 

For this example we are going to use a very simple application: the Calculator app. 

Creating Your First Flow

1. If you haven’t done so already, download and install Power Automate Desktop.
2. Open Power Automate Desktop and create a New Flow.

   

3. This will open up the Action flow builder screen.
4. Let’s choose for our first action to “Run application” and set it to open the Calculator app.

   

5. We want to give it some time to wait for the Calculator application to start. There are many different wait actions we can use. I unfortunately didn’t have a lot of luck using some explicit waits so I used an implicit 5 second wait instead that gave me the desired result.

   

6. Next, let’s take a screenshot of the calculator application and store it in a folder in PNG format. We will use the “Take screenshot” action.
   

   Tip! Whenever you screenshot your application, make sure the application window is isolated (preferably with a dark background) and not overlapping any desktop icons or other windows. This will provide a much better image result.

   

7. We’ll now click on an element on the Calculator application. To do this we can use the action “Click UI element in window” and the built-in recorder by clicking the “Add a new UI element” button to detect an element, in this case the 8 button, to click. When hovering over the 8 button, you can use Ctrl+left click to store the element in your list. Set the Click type to Left click.

   

   

8. Go ahead and repeat the step above by adding 3 more “Click UI element in window” actions to multiply 8×8=64.

   

9. Add one more screenshot action (like in step 6 above). Give the screenshot filename something different than you did in step 6 and store it in the same folder. 
10. Finally, use the action “Close window” to close the Calculator application.

    

11. Now go ahead and execute your Flow by clicking the Run button! If all goes to plan, you should see the Calculator application launch, take a screenshot, multiply 8×8, take another screenshot and finally close the Calculator application.
12. Your two screenshot images should look similar to this:

    

Visually Validating Your Desktop App

So now let’s say you want to visually validate your desktop application with Applitools using our AI image comparison algorithms. But why would you want to visually validate your application? 

Why Visually Validate Your Application? 

Well the old saying goes “A picture is worth a thousand words.” That cannot be any truer than when it comes to testing. Factor in that in today’s software development world, the pace of releasing software updates is drastically increasing. A visual snapshot of your application can uncover bugs, visual discrepancies or layout issues you never knew existed and also ensure your Application’s UI still works and is functional. 

Like in this example, if 8×8 didn’t equal 64 we’d know right away something was wrong because the screenshot would look different. With an image, every visual attribute (text, color, location, etc..) of every element is automatically captured. 

Visual Testing with Power Automate Desktop

So lets add an action to upload our screenshots we took above to Applitools!

1. For this next step we have a few different options. Applitools provides over 50+ SDKs to do visual testing. You can integrate these SDKs directly into your automation framework or use one of our standalone codeless tools. Power Automate Desktop does provide actions to execute programmable scripts such as Python. However, for this example we are going to use our ImageTester CLI codeless tool (also known as the Screenshots CLI) to upload these screenshots from the command line. 
2. If you haven’t done so already, download the ImageTester and place it in a folder of your choosing on your PC. 
3. Next add your Applitools API Key to your System Environment Variables as APPLITOOLS_API_KEY.
4. Now we want to add another action called “Get environment variable” and place it after the “Close window” action.

   

5. Finally, let’s add one more action, “Run DOS command,” to execute the ImageTester CLI we downloaded above. This is where we’re going to use the API Key we are storing in the EnvironmentVariableValue above.

   

6. Your final Flow should look similar to this.

   

7. Now let’s run our Flow again! After this execution we should see our images in Applitools as new baselines.

   

Testing that is Automated and Visually Perfect

And that’s it! You now have stored baseline images of your application. So whenever your development team makes changes to your application in the future you can run this Power Automate Desktop Flow again to ensure your application still functionally works the same but is also visually perfect!

You can see the full Flow in action in the video below:

Applitools comes with many amazing features such as CI/CD integrations, bug tracking and accessibility testing, just to name a few. Visual testing with Applitools will help you improve your overall software quality, test smarter and more efficiently, and release application updates faster without sacrificing quality. Applitools is free so join Applitools today and see for yourself. 

And don’t forget to download Power Automate Desktop and play around with it now that it’s available. Happy testing!

The post Tutorial: How to Automate with Power Automate Desktop appeared first on AI-Powered End-to-End Testing | Applitools.

Some questions I get asked a lot from customers are: Can I automate visual testing of videos? What are some techniques for automated video testing? What software testing tools should I use to test video? 

They’re asking because manual video testing can consume a lot of time — hours, if not days. It can be a bottleneck in your testing efforts and slow down release cycles. 

My answer: you CAN and should automate video testing!  In this post, I’d like to show you how.

Video testing tip #2: Ensure correct conversion

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post How to Automate Your Video Testing using Selenium appeared first on AI-Powered End-to-End Testing | Applitools.