What makes good tests?

Let’s add another “what makes good tests” article to the world! Just kidding, but I do think this is a topic that is always worth discussing. People are well aware by now that writing tests is good because they not only validate that your code works but also validate that your contracts, assumptions, and dependencies work well together.

I talk about testing quite a bit in my book but all that maters is that tests are readable and that they fail when you break your code.

Here are a few guidelines to go by when thinking about your tests.

Single tests

Tests should always run and fail in isolation. You never want test setup to be dependent on other tests, or rely on any particular order of tests. Each tests should effectively work on a blank slate of the world. That either means its testing pure stateless code or if its interacting with the local word (via sqlite or docker or something) then the dependencies should be wiped between tests and set-up fresh for each test. Alternatively you can namespace and create new scoped spaces for long running dependencies - a new schema in sqlite, a new bucket in localstack s3, etc. The underlying context stays up (db/docker) but the namespace/scoping changes per test.

Setup should be clear

Test setup should be clear so that you can tell what the test needs.

Here’s an example of a bad test:

it "validates context" do 
  expect(call).to eq(result)
end

This is a bad test because

• It’s unclear what setup is happening (the setup happens outside the scope of the test)

• It’s unclear where the result value is, or why it’s set up this way? Again the scope is set up outside the test

A better version of this would be

it "validates context" do
  sample = 123

  insert_data(sample)

  expect(call).to eq(sample)
end

Regardless of what may or may not be happening here you can clearly see that we set something up by inserting our data as part of the test, then validating that we got something back related to the insertion.

The example here is silly but these kinds of tests happen all the time.

Test setup in the real world can be very non-trivial, so ensure you wrap test setup in classes or utilities that make it easier to re-use. These test setup utilities can and should be tested themselves too! If you need to set up a complicated API interaction wrap it up! If you need to generate sample data and insert it into in memory db wrap it up! The more you create your own internal testing API the more you can re-use this and then run faster, have stronger tests, and the tests themselves stay short but clear - the entire setup, call, validation is obvious.

I’ve talked about internal APIs before and test setup tooling falls into that same bucket.

Make sure it breaks

Not a lot to say here other than make sure if you write tests make sure you break the code under test and validate that the test itself breaks! You might be surprised to find that sometimes your tests pass and even if you break your code… they still pass. That’s not a good test.

This is especially important when over-using mocks. I encourage teams I mentor to use as much real resources as possible. Actually talk to a database, actually make an in memory http call to your API, etc. When you over-use mocks you might find that you are only testing what the mock says is doing, and that nowhere is the actual interaction between logical components is tested.

Avoid cargo culting

This brings to me another point to not fixate on cargo cult definitions. It doesn’t matter if something is an integration test or a unit test, its just a test. Some tests are small and stateless, other tests require more setup. If the test provides value then its a valuable test!

This is technically an integration test because it talks to a database, but … who cares?

Notice that it

• Sets is data up

• Is isolated per test

• Clearly asserts the contract under tests

• Is obvious what the expectation

This a recipe for a successful test.

Conclusion

Tests are first class code. Treat them as such. Writing a good test is just like writing good non-tests - prefer readability, keep it simple, re-use and componetize shared dependencies, and don’t be afraid to refactor!