March 1, 2023
As teams are moving to quicker release cycles, the need for Automated UI testing has become critical for deploying sooner and more often while maintaining a high level of quality. In order to obtain a high coverage of automated regression tests, the tests must be able to be run consistently and cost effectively and the results need to be able to be analyzed effectively.
This article highlights how bringing together the process and tools can create a platform for consistently running automated tests and managing results to identify new bugs and maintain existing tests. Automated UI Functional Testing
Teams can begin introducing Automated UI testing into any project at any stage of its life. We have found different approaches work best depending on if it is a new project versus a mature product. New projects can begin in-sprint automated testing while mature products should approach the testing of existing functionality from a top down approach focusing on end to end scenarios first.
To support automated testing it is important that your testing approach follow a consistent approach across testers and teams. Acceptance Test Driven Development and BDD provide a consistent language between business, QA, and developers for defining executable scenarios that make excellent candidates for automation.
Automated testing is a challenge for a lot of organizations. Automation is often perceived as expensive, fragile, and hard to maintain. Keyword based testing provides an abstraction layer from the complexities of automation code to allow QA professionals with no coding experience to write and maintain automated tests.
TestArchitect by LogiGear is a tool for creating automated tests using keyword based testing with an Excel-like interface that is comfortable to use and provides self-documenting test cases that can be understood by business, QA, and developers.
TestArchitect implementation addresses a number of the key challenges with automated testing.
Interfaces - Interfaces in TestArchitect provide an abstraction layer between the automated tests and the screens/pages/controls of the application under test. If a page is changed, only a single interface that could be referenced by thousands of tests would need to be updated. This addresses concerns that automated tests are fragile and require a lot of rework if the interface changes.
Actions - Actions in TestArchitect encapsulate interface level activities and steps into business level actions that can be reused across multiple test cases and parameterized for maximum reuse.
DevOps / Development Lifecycle Integration - Supports Visual Studio Online and TFS as Test Case and results repositories to integrate seamlessly into the application delivery and DevOps practices.
Automated regression tests require a platform for running and managing results. Microsoft Azure provides the platform for QA teams to self-service testing environments to run tests across operating systems and web browsers while only paying for the computing that you need and to easily scale as the testing needs grow.
Visual Studio Online provides the test case repository that provides traceability from User Stories to code deployments. With a recent release of VSO, the testing results can now be analyzed directly from a web interface in addition to Microsoft Test Manager that has been available for years.
Microsoft Azure contains all of the components required for the test platform. Hybrid configurations can be utilized to support any number of existing infrastructure configurations including on-prem TFS and/or build servers.
TestArchitect provides the run time engine for running the automated tests from the Test Agents, capturing detailed results available within Visual Studio Online.
Automated UI regression tests should be set up to run on a regular basis (usually nightly) in the testing platform on Windows Azure using Visual Studio Online and TestArchitect to produce consistent testing results. The results can quickly be analyzed to determine if there are any new bugs or changes in requirements.
In a recent update to Visual Studio Online, the TEST hub now contains the results of the test runs. The test run list shows which ones have completed successfully, need investigation, or completed the investigation. This is a typical starting point for analyzing the results.
Clicking on a test run will display the Run summary page for that run with visualizations to understand the outcome of the test run.
The Test results tab displays the results of the individual tests. Each failing test can be investigated by clicking on it.
The Result summary displays the details and provides thefunctionality to analyze the results and create a bug and/or update the analysis to fill in thefailure type and resolution. When using Test Architect for Visual Studio, the results from the test are also attached so you can see the results from individual steps.
The TestArchitect results show the details of each step so you can quickly see what step / action failed and the details of the error message.
This article showed you how you can use Visual Studio Online to manage the test results of automated tests including TestArchitect Automated UI tests being run in an Azure platform.
If you're looking for ways to help improve your product delivery pipeline including automated testing and deployments, contact us at Lunavi to find out how we can help you achieve higher quality and efficiency in your DevOps practices.
.png){kind=logo}
We're here to help you tackle what's next in your digital journey.
