Testim Development Kit Open Beta Overview Register Today

MakerSights increases end-to-end test coverage to ensure enterprise quality

MakerSights Helping leading apparel, footwear and homegoods brands inform decision-making throughout the product creation go-to-market

MakerSights supports retail organizations with a system of learning to assess trends, gain market feedback and bridge the gap between what brands think consumers want and what consumers value. Brand product teams use MakerSights to support informed decision-making at every stage of the go-to-market processes, to reduce risk and improve product-market fit.

With MakerSights, customers can quickly learn from their most important consumers to gather insights on future product designs. Customers evaluate product insights to better predict new market entry costs and risks. MakerSights customers walk into important meetings empowered with data, not just opinions.

Challenges

As a fast-moving SaaS startup that sells to enterprises, MakerSights’ business depends on delivering a consistent and stable platform. Each brand’s business and products are unique. Whether it’s the design of a watch face, the cut of jeans, or the desired colors for winter sweaters, each customer has a different set of products that they want to evaluate.

Mike Mullins, Director of Engineering, has been with MakerSights since its inception and experienced the effects of rapid customer growth first hand. Each new customer added variability in how they deployed different feature sets and the type of offerings they tested. Given this, MakerSights needed to increase end-to-end functional testing to ensure quality remained high while undergoing continual innovation.

“We could test that an API would return the correct value, but that’s not enough. If the display isn’t right, our customers aren’t going to take ‘the API was right’ as an excuse.

 

We knew full end-to-end integration was the direction we needed to go and committed as a team to that route.

The False Start

They decided to try an open-source approach to test automation and selected Selenium. Even though they were a team of highly-skilled developers, they found Selenium to be a challenge.

Creating tests, maintaining tests and running a selenium environment, all proved to be time-consuming, stealing precious time away from new feature development.

Mike emphasized,

 

We had a cheat sheet of 10 ways to enter text into Selenium. It was frail, fickle, and always breaking.

Mike and his team spent months implementing Selenium, because of these challenges the project was put on hold, and the team went back to manual testing to meet their product deadlines. They spent several additional months trying to get Selenium to fully functional and finally decided to pursue other automation solutions.

MakerSights looked at other tools that use AI to automate testing. However, they found the learning curve to be too steep, requiring their developers to write out test cases using specific terminology which felt unnatural. In addition, they didn’t have confidence that other solutions would be able to handle the variability their customers create.

Solution

MakerSights knew that they needed a solution that would allow them to quickly create stable tests, without hiring specialized skills.

Mullins and the team found Testim.

MakerSights started using Testim, and after just 20 calendar days, they got far as they had with Selenium, with only a fraction of the team resourcing and time.

Testim solved for the Selenium problem of brittle selectors that could break a test with every code change. “Being able to record user flows and customize the selectors makes test creation much easier,” said Mullins.

MakerSights has integrated Testim into its developer flows. Each GitHub pull request kicks off a build in TravisCI and tests the “happy path” in Testim. Their goal is to shorten release cycles to one day. They haven’t transitioned to a fully continuous delivery organization yet, but Testim’s automation helps them get closer to their goals by enabling them to focus on other business-critical initiatives.

Benefits achieved:

  • Significantly faster test creation (in as little as 5 minutes) driving improved test coverage

  • Lower maintenance and better stability because of Smart Locators

  • Postponed hiring an engineer just to manage testing

  • Eliminated third party grid licensing by using Testim’s grid

  • Moved closer to their goals of achieving continuous delivery