Is the test pyramid a bad sign?

The testing pyramid is a well-known pattern for testing strategy which is praised in most companies practicing tests.

However, this is not a magic solution, and in your case it might not be the right testing strategy but instead the main reason why your tests are ineffective.

The value goes down the pyramid

A bug is an abnormal behavior visible to the end user, and it is important to note that incoherent states can exist within the application without leading to a bug.

That is why it is important to focus on bug instead of trying to find all incoherent states while test so we can focus our efforts on tests that are valuable.

The problem with the pyramid in that situation is that it will make you focus your effort on what is at the bottom of it, unit tests.

These tests have a major issue as they are the lowest level of granularity, they will make you check that each class behaves or functions as expected.

However, these abnormal behaviors don’t always transform into a bug, and you will end up creating tests will low value.

A refactoring issue

Going down the pyramid means also going down the abstraction coupling more and more our tests to the implementation details while creating a new feature this might not seem like such an issue.

However, as soon as refactoring, the code kicks in, all the issues reveal themselves as changing the simplest of things can cause a cascade of tests to fail.

That bottleneck can make developers fear refactoring their code and instead only focus on making quick fixes.

That over time leading to the technical debt increasing and making it harder to maintain the code base, pushing some code which could have been kept up for years using multiple small refactorings into a costly total rewriting of that part of your application.

You are not Google

All big companies are using the testing pyramid, yes, but also all of them are also using microservices, mono repositories and so on.

What you need to understand is that you are not Google and you have a really different context for your engineers to work on.

Due to that, some solutions that might be great for big five companies are not for smaller ones like yours, and the testing pyramid is one of them.

The diamond

If the testing pyramid is not a proper solution, then what is an alternative solution to this?

First, I would say that if you start from no tests at all them, the best would be to build the pyramid by starting by the top and going down gradually.

This is due to multiple factors.

Value

When starting writing tests from the bottom of the pyramid, you will start with the tests that have the least value at a business level and are extremely fragile.

Due to that you will not get the maximum value from theses tests and you are in fact reducing your chances to create higher level tests.

If your manager or boss is not seeing value from you being testing but rather a way to slow down your development workflow, he or she won’t let you go further and this is why it is important to start by what can achieve the maximum value to convince them to go that way.

Understanding

The more you go down the pyramid, the more expectations you will have about the application will be abstract and the more changes you will have to write tests that are off.

Remember that it is important to test expectation you have about the application, not your implementation, and starting from an end user perspective is the best approach to make a clear distinction between both of them.

That way you will start creating some scenarios or acceptance criteria that will be transformed into recipes for manual tests.

Once this is done, then you will invest some time into acceptance testing to start automating the process while still basing yourself on acceptance criteria and slowly moving downward the pyramid.

Never go too low

As long as you as you remain a relatively small company, then you should never fall into the trap of implementing the full pyramid.

Instead, you should more aim for anti-fragile tests.

For that you should work with black box tests that are using acceptance as base.

That way you should be able to write a test once and don’t have to write it again except if the expectations concerning that scenario changed as it is not linked to the implementation from the application.

Join my newsletter

Never miss a single article and receive a free ebook “Bug hunting for small companies”.


Leave a Reply

Your email address will not be published. Required fields are marked *