Test failure scenarios first
September 21, 2009 - 0 Comments - tdd
Lately in test-driven development, I’ve noticed that I tend to write tests against invalid input before I get to tests against valid input. This is a subtle technique which I think merits some discussion.
One of the big wins with TDD is the way that it forces you to think about the problem in terms of the API; this pushes the engineer to design to an interface, not an implementation, which is one of the major takeaways from the original Gang of Four book on design patterns. And I think the higher-level concern that designing to an interface facilitates is the process of knowing the problem domain, which leads organically into the process of dividing up responsibilities between discrete classes and modules.
In many ways, a problem domain can be defined not only by what it’s supposed to do, but also by its constraints, the cases in which it should fail. Thinking about these cases upfront, testing against invalid or nonsensical inputs, leads to software which is more resilient to misuse and unexpected scenarios.
I find that defining these constraints up front is a great way not only to get the ball rolling. It also ensures that you’re intentionally thinking about constraints as you write your tests. For example, it’s very easy to write simple tests with assert_raise blocks and then have the implementation raise the appropriate exceptions for those scenarios. Just like that, you’ve got passing tests around some basic constraints, such as parameter requirements. Once you have a good set of constraints defined and tests for them are passing, you can start writing tests against the cases which are supposed to work.
If you read my Shoulda tests, you will often see a context start with a bunch of tests that look like should 'fail if X' do ... end. After that, you start to see the tests which test the real meat of the feature.
Besides that it lubricates the initial friction of writing tests, I think this approach leads to solid, secure, and fault-tolerant code. The client code can catch the exceptions as appropriate and react accordingly. Whatever happens, you can rest easy that the API won’t let the caller do things it shouldn’t because you thought of these things up front and wrote tests against those cases first thing. Of course, human nature is that we can always miss cases that we hadn’t considered, but thinking about them from the get-go is much better than trying to rig it in after you already have an otherwise complete implementation.
This is just something which I’ve noticed emerging in my own testing style, which is of course in a continuous state of evolution. Does anybody else have thoughts on testing around constraints? Do you tend to write them first or save them for later? Do you notice any patterns in your testing style which you’d like to share?
