What Are Testing Frameworks?
Before we dive into our list of frameworks, it’s important to understand what a testing framework is.
Put simply, a testing framework is a tool—or set of tools—that helps you perform unit testing. A testing framework should provide you with at least
- Rules or conventions for writing test cases and organizing them in test suites.
- Means to automatically execute and self-grade the tests.
- Results of the tests that are displayed and formatted.
By now you should have a firm grasp on what testing frameworks are and why we need them. With that out of our way, it’s time to tackle our list of tools, starting with Mocha.
This framework makes asynchronous testing simple by running tests serially. That enables accurate reporting and, at the same time, mapping of uncaught exceptions to the correct test cases. The Mocha framework has an extremely flexible configuration. It allows you to pair it with the libraries you particularly need for functionalities such as assertions, mocking, and so on.
The downside of such flexibility is that Mocha requires more work for its initial setup. Also, while other frameworks already come with assertions and mocking functionalities, Mocha requires additional tools for that.
Puppeteer is a Node.js library that allows users to control a headless Chrome or Chromium browser. That way it’s possible to automate tests that you’d typically perform manually using a regular browser. The project’s page lists some examples of tests you can perform:
- Generate screenshots and PDFs of pages.
- Crawl an SPA (single-page application) and generate pre-rendered content (i.e., “SSR” [server-side rendering]).
- Automate form submission, UI testing, keyboard input, etc.
- Capture a timeline trace of your site to help diagnose performance issues.
- Test Chrome extensions.
- Testim released a free recorder for Puppeteer. Check it out and use it as often as you like.
This framework already comes with pretty much everything you need to start testing. It has built-in assertions and test doubles with spies, unlike, say, Mocha, which requires you to install third-party tools for those functionalities.
However, Jasmine is also flexible and can integrate seamlessly with a large number of additional testing tools. You can easily use Karma as a test runner or employ Sinon.JS for your mocking needs.
QUnit was originally part of the jQuery programming library. John Resig created it for testing jQuery, jQuery UI, and jQuery Mobile. In 2008, QUnit was extracted from the jQuery codebase and released as a standalone framework, but it would still require jQuery to work. In 2009, they fixed that with a rewrite, making QUnit finally independent from jQuery.
If you take a look at the Jest “getting started” page, you’ll see that you can install it, write your first test, and run it in a matter of a few minutes, without complicated setups or additional installs. Another advantage of Jest is its impressive performance.
Jest is primarily used for testing React applications, but that doesn’t mean you’re restricted to using it to test only apps written with the framework. On the contrary: you can use it to test Node, Angular, and Vue applications as well, just to name a few.
Put simply, we wanted to keep the list as concise as possible. To do that, we’ve decided not to include on the list testing tools that aren’t frameworks, strictly speaking. So, that’s the reason a super popular tool such as Karma didn’t make it: it’s a test runner, not a framework. The same reasoning applies to Sinon (a mocking tool) and Chai (an assertion library).
You might argue that Puppeteer doesn’t quite fit into the list and, to some extent, I’d agree. That is to say, all of the other tools are frameworks more suited to performing unit testing–style tests, while Puppeteer is more of a browser automation tool. It is still a testing framework, though, in the sense that it allows you to design automated tests that are later run without human intervention.
Evaluating the Frameworks
The first criterion is certainly the easiest one for us to evaluate. All of the frameworks we’ve covered here are open-source projects, so you can download and use them for free (as in beer). So, we have no clear winner here. Let’s move on.
Installing all of the tools above is fairly easy. After all, you can install all of them by using npm (or, alternatively, Yarn.) However, some of the tools offer additional installation options.
For instance, besides npm and Yarn, you can install QUnit as a standalone download. But that would require you to store and manage this asset. Instead, you can use the version hosted at the jQuery CDN. Jasmine also offers a standalone version that doesn’t depend on Node and allows you to run specs using your browser.
So, do we have a winner here? We think so. If we were to pick based only on ease of installation, we would have another draw. But, as we’ve seen, QUnit and Jasmine both offer more options, since you can use them as standalone libraries. Is this additional flexibility useful? Your mileage may vary on this, but for us, QUnit and Jasmine are the winners here.
Winners: QUnit and Jasmine
Ease of Setup
Now, let’s look at the ease of setup. When it comes to the initial setup, we have a clear winner here: Jest. The framework promises zero-config usage and delivers on that promise. Sure, it allows you to do optional configurations, but it does feel amazing to install a testing framework and have a real test running in about two minutes, without any configuration or installation of further tools.
Finally, the learning curve. We’ll have to pick Jest again here. Since it comes with a wide API and doesn’t require as many third-party tools as the other frameworks, Jest inevitably presents a less steep learning curve.
Here’s the Winner
So, since it was the winner in two of the four criteria we used, we’re going to pick Jest as our winner. That doesn’t mean you shouldn’t try the other tools. On the contrary: experiment and practice as much as you can since that’s the best way to learn.