Our client is the market-leading data authority for the hybrid cloud. They provide a full range of hybrid cloud data services that simplify application management and data across cloud and on-site environments to accelerate digital transformation. Their solution empowers global organizations to unleash the full potential of their data to expand customer engagement, foster greater innovation, and optimize their operations.

The Challange – Achieve CI/CD

to achieve CI/CD. Their application was complex and included many user flows, requiring the creation of thousands of functional tests with the goal of shifting left. In addition, they needed to test as close to development as possible.

Selenium was their first choice and a team of over a dozen testers was assembled to begin the task. After a few months, it became apparent that it was going to take more time and manpower to achieve their goal of CI/CD. The tests were complex and took three days to author. To make things worse, the tests would often break, leading the team to spend extra time maintaining and fixing the tests. Shifting left would require teaching the developers Selenium as well.

The lack of tools to troubleshoot a failed test (e.g screenshots to compare, details error messages pointing to the right step, test history or the parameters over the flows) led to long time-to-resolution involving a number of team members. A lot of time was wasted, not only trying to figure out why the test failed but to also explain the discoveries to the developers.

Maintaining tests took a lot of my time. When developers run tests that fail it becomes more of a distraction than confidence in bug prevention. Both groups had to stop what they are doing to figure out if its the functionality or if it’s the test. We found ourselves spending more time trying to stabilize the tests than actually testing.

COMPANY QA MANAGER

The Solution

Testim’s solution that uses machine learning for the authoring, execution, and maintenance of automated test cases was implemented. With Testim they were able to capture the scenarios in minutes, as well as complement that with Javascript syntaxes. The team was able to spend more time creating new tests, and validating the status of their application

Today we have hundreds of tests running on every pull, giving the developers feedback to their code within minutes. The developers themselves easily update test scenarios so QA can focus on increasing coverage. We also significantly reduced our cost of quality: The rich information we get allows us to reduce the time to troubleshoot by 80%.

The Results

As a rapidly growing enterprise, they needed a way to optimize processes through automation. This plays a significant role in their move to agile development. Within a couple of months of using Testim, The team was able to create hundreds of their UI test scenarios. Today, they are proud to say that they are authoring tests in under an hour compared to the three days it was originally taking.

Before Testim it would take 3 days to author a single test in Selenium, now even for the less experienced tester, writing tests takes under an hour, developers can update tests on the fly and figure out where the tests are failing without any additional help.

Now, troubleshooting a failed test takes a fraction of the time. There is an indication on which step failed, including JS syntaxes, screenshots comparing prior runs, and access to the DOM with detailed error messages.

However, the biggest impact was the reduction on maintenance. Tests are stable and trustworthy so when a test fails the user knows it is either due to a bug or the test requires a change in the flow. The team focused most of its time on increasing coverage knowing that the tests adapt to UI changes by the development team.

Today, we are proud to say that they are fully CI/CD, testing on every code push, and running thousands of tests every day.