Note: Cross posted from The Typemock Insider Blog.
Permalink
Sam Newman asks three questions that can help you decide whether to write tests for a feature.
- How easy is the feature to test?
- What is the likelihood of the feature breaking?
- What is the impact of the break?
Let’s start with the 2nd one. That’s the most important. Because if you don’t have any logic in your code, like wrapper properties, there’s not much point in putting a test around it. Also, if all code A does is call another piece of code B that is surrounded by tests, the likelihood of the coda A breaking is low, and therefore you might decide you don’t need a unit test for it. (An integration test that covers both is another story). In all other cases it makes sense to write a test.
A feature is sometimes hard to test if it requires a whole setup of servers and databases. However, with Isolator, you can, and should, write tests for the different components. If you decide it needs a test (according to the former paragraph), it is easy to unit test with Isolator, and therefore you should write the test. For integration tests it’s just as important: If a system is too complex to test, what do you think the probability is for having bugs there? Exactly. Q1 is irrelevant.
Q3 starts off with good intent. It analyzes the severity of a possible bug, and based on it directs you to either write the test or not. However, it does not take into account that a future code change might affect the current behavior of the feature. We write tests not just to make sure our code works now, but also as a safety net that accompanies the code through its lifetime. If we don’t have a test in place when we are changing our code, we don’t know we’ve broken something.
To counter that let’s take a look a different example: Let’s say I have a web page, with a yellow background. Now, I’m sure it never happened to you, but with every visit from a marketing person, the color changes. Should I have a test in place that checks that the background color is the correct one, then maintain the test with every change? Obviously not. And this is where Q3 comes from. If the impact is just changing the color identifier, I will not write a test.
So when do I really write a test? Where there is logic. It is logic that causes the state to change, and it’s logic that interacts with other components in your system. Where there’s logic there’s a chance for a bug (Q2). And bugs in logic have the biggest cost associated with them (Q3). That’s why there’s no need to test properties, if they have no logic. And where there’s a bug, write a test for it as well.
Integration tests are not as different, but they do cost you more to write, because you can write a whole lot of them, based on the system complexity. So what should you do? We write tests to lower the maintenance cost of our code. If you have a very good suite of unit tests, you can minimize the number of integration tests around the well tested code. This is an analysis driven decision, unlike the unit test question.
What’s your guidelines for writing tests?