This is the first in a series of posts about testing backend services and some principles that we follow at letgo. In this post, we’ll cover some core testing concepts that we need to understand in order to improve our test suite.
Anatomy of a backend service:
We try to follow some DDD and Hexagonal Architecture principles when designing our backend services. We also try to apply CQRS using Command and Query buses whenever a project’s complexity demands it. The following design is a simplified version of the anatomy of one of our use cases without buses and a few more layers that we use to implement.
Controller:
A controller handles HTTP requests and responses, translating requests coming from clients to something understandable by our system. That’s done by validating the input and wrapping it into Domain Objects, or failing with an HTTP error. The validation itself occurs in the Command Handlers but we will omit them for simplicity’s sake.
Service:
All the logic that our application does should go here. All dependencies that imply IO will be injected here.
Repository:
A repository provides access to data. It’s usually a database but it can also be an external service. The repository’s interface belongs to Domain. Its implementation belongs to infrastructure. We should try not to put too much logic in repositories. Ideally, repositories should have an API similar to an array: add, search by id, update and delete.
Types of test
Out of all the types of tests, we’re going to focus on three.
Unit
This kind of test checks that our domain logic works as expected. These tests should be really fast, isolated and repeatable. That creates a fast feedback loop that maximizes value while developing. You invoke services mocking infrastructure (or using in-memory implementations) and assert the response that it returns or side effects in case of a command. You should try to test all possible errors, not just the happy path. What you’re testing here is the application’s behavior.
Integration
This type of test is used to check that the implementation against an external service works. They’re typically used to check that your repository works as expected against a database or external service. These tests are slower and add less value than unit tests, since they have some overlap with acceptance tests.
Acceptance
Taking into account the test pyramid, the main purpose of this test is to check that all pieces that have already been tested in isolation work well with each other. It can also be used to check HTTP status or your data validation. We don’t test our controllers in isolation, because we try to keep them very small, so this test also tests the logic placed in them. This kind of tests are usually really slow but they maximize value from a business point of view because it’s the only test that ensures the whole system works as expected.
Some other testing concepts…
Determinism:
Determinism is a concept that relates to repeatability. A system is deterministic if a specific input will always have the same output. A system can be a whole service, a single class or a bunch of classes that collaborate with each other.
Let’s look at an example:
class ExampleService {
def myMethod(value: Double): Double =
value + System.currentTimeMillis()
}
We can easily see that code is non-deterministic. The result of this function depends on when you call it. You can see a deterministic system as a mathematical function f(x) = y. In an ideal world, all non-deterministic effects should be in the infrastructure level.
In the following posts we’ll explore techniques to help us isolate non-deterministic effects and make code like this easily testable.
Mutable State:
In an application, a mutable state is the data that belongs to it and could mutate as time passes when an event happens in the system. Almost all useful modern software applications have state. We usually store the state of an application in some kind of database but code can also have state (through mutable variables). As an example of code with state we can take a look at this piece of code:
class ExampleService {
var statefulVariable: Double = 0
def myMethod(value: Double): Double =
value + statefulVariable
}
In this example a mutable variable is defined in the class scope. This variable leads this class to a non-deterministic behaviour, the value of this variable affects the result of myMethod. If we decide to have stateful code we need to handle a new bunch of problems like concurrency and non-deterministic behaviours. As a rule, we should try to isolate state to infrastructure or an external system but sometimes performance requirements don’t allow it. In following blog posts, we’ll explore some strategies to isolate state. Spoiler: There are tools like actors or channels that allow us to handle stateful code in a safer way.
Coverage
If we visualize our domain layer as a graph, the more paths our domain logic has, the harder it is to test. You need more tests to cover all possible paths. In general, if we do abstractions, complexity increases and we need more tests to cover all paths. This is why we only abstract when we’re very sure that we’re creating a lot of unnecessary boilerplate and we aren’t afraid of some duplication. We prefer simplicity over easiness.
Testing all paths in acceptance tests is way harder than testing them with unit tests because they have more moving parts to set up. It’s also more expensive because acceptance tests are the slowest. When we develop we want fast feedback and when we run our test pipeline we want it to be as fast as possible because we like continuous integration and deployment. We’re not obsessed with code coverage percentage- we see as more of a consequence than an objective of a good test suite.
Some key points of a good test suite:
- Non-fragile tests: We don’t want to break a lot of tests when we refactor something, and we don’t want false positives either. So we try to test our SUT from outside without knowing how it’s implemented internally. We try to test use cases instead of classes.
- Fast: Some of letgo services have hundreds of tests. We’ve set the max time at 10 minutes. If our test pipeline takes more time than that, we should look into what’s happening and how we can speed it up.
- Confidence: Our build always needs to be green on the master branch. If something fails in master we should stop what we’re doing and fix it.
- Cheap: We try to be pragmatic. The theory is nice but we also have deadlines so sometimes we need to choose the cheapest time option and that means that some services might only have some acceptance tests. We’ll talk about applying software economics to testing in further posts.
- Readability: We try to not abstract too much in our test code. We prefer to have two 50-line tests than two 5-line tests with multiple (and often rigid) abstractions or helpers. We want to read a test 6 months after coding it and understand what it does.
When we run our tests
At letgo we work with git branches and Github pull requests. We try to keep it as small as possible and when merged into master we want to deploy it to production. Every developer usually runs at least unit tests on their local machine, but it’s not guaranteed. Before a pull request is eligible to be merged all tests should be passing in the team CI server and should be reviewed by other devs. These tests are run against a virtual merge of the branch with master. After merging we usually run our tests again, but this time we also run other processes like code quality code static analysis tools, code coverage, etc. Once this build has passed it’s deployed to staging and then to production.
