No company would release a piece of software without testing it. But there are many ways of testing software. End to end testing is one way. It might even be the most important way, though often the most time- and resource-intensive one. Luckily, there are ways to automate end to end tests.
In today’s post, we’ll clarify not only what end to end testing is, but how it fits into an overall testing strategy that can benefit every organization.
What Is End to End Testing?
In theory, end to end testing is the process of testing a piece of software from start to finish as it will be used by the actual users. For a web application, you would start a browser, navigate to the correct URL, use the application as intended, and verify the behavior. In the case of desktop application, you would start up the application, use it, and verify its behavior too. If you’re testing an API, you would make calls just like the real clients would do.
Whatever your piece of software, the idea is to run the application with all of its dependencies: database, external services, metrics, logging, etc. This can still be in a test environment, separate from the production environment. But ideally, this test environment represents the production environment as closely as possible.
You may have noticed that I started this section with the words “in theory.” This is because it might not always be possible to run the application without real side effects, even in an isolated testing environment. This can be the case when external dependencies only exist in the production environment.
For example, in an industrial setting, it’s not uncommon to have only production PLCs (computers that drive industrial machines). You don’t want your end to end tests influencing the machines in the production environment. So in this case, you’ll have to be creative and stop the commands from going to the PLCs, while still verifying their correctness.
There are many situations where you might want to “fake” certain components of your overall application for your end to end tests. But in end to end tests, you want to reduce these fake components to a minimum. Read on to find out why.
Why Do We Need End to End Tests?
Software can and should be tested on many levels. At the lowest level, we have the unit test. This is where programmers write a small routine to test the routine they are actually writing. All these small routines together will form your application. But it’s useful for the programmers to write these small test routines so they can have fast feedback on whether or not they are implementing a feature correctly.
However, just verifying that all those small pieces work isn’t enough. It doesn’t ensure that they work correctly together. This is where integration tests come into play. In integration tests, two or more components are used together in a test and the result is verified.
But even if all these tests run perfectly fine, you could still have bugs or unmet requirements.
First of all, if you have a UI, unit and integration tests don’t cover the UI. Is the UI structured correctly? Is it a pleasant experience? Are all the necessary elements in place? Can the user achieve what they want to achieve? These types of questions can only be answered by an end to end test.
A second reason why end to end tests are necessary is that there are so many moving pieces in modern software: servers can cause applications to behave differently, configuration must be set up correctly, and external dependencies influence our application. Testing separate components will not cover all this.
Does this mean unit and integration tests don’t have value? Definitely not. Unit, integration, and end to end tests all play an important role in an overall testing strategy.
The Overall Testing Strategy
Testing software is an activity that happens on multiple levels, by multiple people in different roles. Developers test their code with unit and integration tests, while testers work together with developers to define test scenarios that they’ll run manually or automatically.
End to end testing is the category of tests that’s drawn at the top of the testing pyramid. The testing pyramid gives us a guideline of how many types of tests we should have. Because they are easy to write and maintain, and fast to run, it’s valuable to have many unit tests. As we move up the pyramid, our tests start using more pieces of our application. These tests become slower to execute and harder to maintain.
That’s why there are usually fewer end to end tests than there are unit tests. But the end to end tests are still featured in the testing pyramid. In the section above, I’ve explained why they’re still valuable.
The different categories of tests each have their unique value in the overall strategy of testing your software. But many organizations still perform the end to end testing manually—even when there are tools that can automate end to end tests. With automated end to end tests, you can add and maintain more tests in less time.
Using End to End Tests
It isn’t always clear how to start with end to end tests. Many companies manually test new features and leave it at that. But this allows bugs in features that previously worked to go unnoticed.
The first step is to identify test scenarios. Get together with business owners, developers, and testers and think about some scenarios that you want to test before every release. This doesn’t have to be an exhaustive list. It’s better to have a minimal set running first, and then expand on it later.
Using this list, you can now map out the different steps in each scenario. Write down the steps that need to be performed and the expected results.
Notice that we still haven’t automated anything yet. You can already use these scenarios to perform manual end to end tests. Of course, the real value lies in automating these tests. But if, for some reason, it can’t be automated (yet), at least you have some end to end tests to get started with.
However, a next step will be to automate these tests. An application without a UI like a REST API can be tested by executing a series of calls to the API and checking the results. Tools like Postman or a Cucumber library are useful here.
UIs are traditionally harder to run in automated tests. Tools like Selenium can run automated UI tests, but easily lead to fragile tests. They also require a decent amount of work to build and maintain. With Testim, you can create tests just by using your application. Testim records your actions and can then run those actions as an end to end test. Testim is unique because it will learn about your application. If you change your UI, Testim is able to notice this and adapt the test. This leads to less fragile tests, allowing you to focus more on improving your product.
Once you have a few end to end tests set up, make sure they run in your CI process. This provides valuable feedback to developers so they know if they broke anything with a code change. As you experience the value of end to end tests, you can then repeat the process: identify test scenarios, list the steps, and automate the test.
Test Your Software
This article aimed to clarify not only what end to end testing is, but also why it’s necessary. End to end testing accompanies other types of tests like unit and integration tests. It fits in an overall testing strategy that every organization will benefit from. This rings especially true if you automate your end to end tests.
Author bio: This post was written by Peter Morlion. Peter is a passionate programmer that helps people and companies improve the quality of their code, especially in legacy codebases. He firmly believes that industry best practices are invaluable when working towards this goal, and his specialties include TDD, DI, and SOLID principles.