Writing end-to-end automated tests.

They are important, but I don’t like to write them. Just thinking about setting up a system all ready for testing just a single test case, then cleaning up after it. Trying to think about all the things that can go wrong and can stop the build, because some environmental god will decide to change the conditions, and won’t let me in on that in advance.

This story is about one test I decided to tackle recently. There were some unit tests for that code, so I did have  something to start with. To the feature I added, I needed a couple more, and it was easy to start from the existing ones to cover the new stuff. The problem was that these tests were not valuable enough for the feature: while they checked some logic in it, the feature was about deploying and cleaning subsystems. Isolated tests help, but you don’t get enough confidence that the feature actually works.

This component has been around for years without end-to-end tests, and was manually tested years ago. We get some support questions on it from time to time, and I decided to at least created tests for the functionality I’ve added.

Funny side note: As we were getting close to making the first test pass, we were reminded that we have similar tests for that! Turns out this was half right – there were tests, but they were not testing actual success scenario. While I was looking at these stupid tests, I remembered writing them a few years ago. Guess who was writing that feature then. You can laugh now.

In the end, we’ve got it right and got the tests into the build. It was green and everything was good.

Not exactly.

Here’s the interesting part: adding the new functionality took an hour, which included manually testing it about 15 minutes.  Writing two tests for it took almost a day (that’s an exaggeration: the 2nd test was quickly deleted as we understood there was no actual use case for it. So that’s basically one day for one test) .

Do you know why so few developers write tests? Because this is what they bump into when they first try, and first impressions count.

Testing is still hard, regardless of tools. The complexity of the systems we’re dealing with makes any prescribed test (manual or automated) susceptible to changes. It’s still hard to understand the whole software, remember if there’s any code already doing what you’re working on, and explain what you’re doing to an unsuspecting pair. (did I mention it was a work in  a pair?).

The more I do this, I’m amazed at how many organizations take testing so lightly. How testing is not being taught as a standard curriculum part of CS. Testing skills, can and should be developed in anyone who touches the software: developers, testers, business analysts. Without the ability to test properly, quality and delivery capabilities suffer.

When first-time developer write tests, who were not educated properly, smash into real code, usually the code wins.

And we all lose.

(Image source)