code.isdangero.us

My test-driven Ruby setup

May 12, 2009 - 8 Comments - ruby tdd tools

I am a relatively recent convert to the mantra of TDD:

1. Write a failing test
2. Make the test pass
3. Refactor

It took my involvement in writing an analytics API for the iPhone to help me see the light. This API was receiving millions of requests per day from hundreds of thousands of unique users, so every deployment of new code had to be rock-solid.

Of course, back then I was just using vanilla Test::Unit; we didn’t know any better. Since then, as I urged my colleagues to give TDD a shot, I have researched a number of libraries which take a lot of the pain out of writing tests. Here follows a description of my testing stack, along with brief explanations of why I went with them over the alternatives.

Shoulda

I love Shoulda. Stringified test names + nestable contexts + easy macros makes for well-organized, concise tests. The readable output it generates is also a great feature; consider the test below:

When this is run and it invariably fails, we get the following output:

Failure has never before read so well.

So Shoulda provides some obvious advantages over vanilla Test::Unit, but why not go with RSpec like so many cool kids are doing?

For me, it came down to a question of Occam’s Razor. It seems from my research that RSpec’s goal is to help people understand the value of testing by making test code read more like English. I can see how that might be effective in producing the “Ah ha!” that so many developers still need before they see the value in writing tests, but I personally don’t need any further convincing, so RSpec’s fancy syntax seemed extraneous and just another thing I would have to learn when the typical assert methods already work just fine. A report of strange issues with RSpec which drove a switch to Shoulda probably also influenced my choice.

Shoulda makes me happy; I shoulda been using it since a long time ago.

Machinist

Have you ever actually written test data using Rails’ built-in fixtures feature? It’s a nightmare. Even after the introduction of named fixtures, it was still a disorienting mental dance trying to remember all of them when editing them or using them in your tests. Fortunately, alternatives came along which take a more generative approach, allowing you to create valid objects as your tests need for them, specifying only the attributes which need to differ from the default blueprint.

This is definitely the right way to go for test data. Two solutions available at this time are Factory Girl and Machinist. Lots of people seem to like Factory Girl, and it’s a great solution and if I recall correctly it came around first, but given the choice I went with Machinist for its more concise blueprint DSL.

Compare the following example from Factory Girl:

with the equivalent blueprint for Machinist:

Same number of lines, but Machinist’s lines are consistently shorter, which means fewer keystrokes to define a blueprint. It’s also more readable in my opinion.

As far as I can tell, there is just one thing that Factory Girl has which Machinist lacks, and that is the ability to define a new factory which inherits from another factory. This would be useful if you have a basic User factory followed by more specific types of Users. So far, I have found that I can get by just fine without this feature, but it would still be a fine addition to Machinist.

Autotest

If you do TDD with Ruby, you should absolutely be using autotest. I like to just keep autotest running in a second monitor while I edit code in the other. It’s very intelligent about keeping things speedy by following you around your code, running only the tests you need to be focusing on, the tests which last failed, until you get them passing again, at which point it runs the whole suite again to ensure that no other regressions occurred as a result of your fixes.

What I’m trying to convey is that autotest does more than merely save you the effort of running your tests manually. It makes testing utterly integral to your coding experience while leading you in the right direction within the context of TDD. It helps you focus into a Zen-like state of flow. It makes TDD worth pursuing.

The available plugins for autotest make things all the sweeter. I personally just stick with Red/Green to make the test output easier to grok, but there’s also Growl integration for those of you who would be into that. All good stuff.

Update: Since I wrote this post, a small extension for Mac users of autotest has been released, called autotest-fsevent. This extension reduces the CPU and disk demand of autotest by using the Leopard’s FSEvent to receive notifications about modified files, instead of polling the filesystem constantly. If you ever find yourself running autotest from your laptop, this will also save you some battery time. As FSEvent is part of Leopard, autotest-fsevent requires Mac OS X 10.5.

Mocha

Mock objects are sometimes necessary, although I’m beginning to discover that they can actually do wonders for your tests if used appropriately, a realization which makes me want to go back and do some refactoring on my current project. I will perhaps cover this in a later article.

I currently use Mocha to do my mocking. It’s a powerful and comprehensive library which can set expectations, stubs, and more. However, I am not married to it, and a new contender RR seems to promise even more concise syntax and just as much power. Obviously from my keystroke-pinching comparison of Factory Girl and Machinist, I am a big fan of cutting down on keystrokes to get things done, and RR seems to be just the ticket.

TATFT?

Do I TATFT? If this means writing unit tests, functional tests, integration tests, and view tests, then no.

I am a proponent of the Skinny Controller, Fat Model pattern. All complicated business logic goes into the model, where it belongs. What’s left in the controller action is ideally no more than five lines of code (down to one or two, if possible) which have the model do something and then render an appropriate response, whether that be a redirect, a template, or a JSON object. In this scenario, writing functional tests around simple message passing seems like a waste of time when the real guts are in the model.

For this reason, I favor very comprehensive unit testing of my models. If the models are solid, then it follows that the controller code which primarily triggers the model code will likely be just fine. If there’s anything in the controller complicated enough to warrant testing, then that might be a good sign that such logic is in the wrong place.

I have considered adding view tests to complement my unit tests, especially for pages involving complex JavaScript behavior, however I’ve yet to do much research in this area.

Your thoughts?

All of this said, I think that as a community we are all still trying to strike the right balance in our approaches to testing, and I think it is still a topic worth discussing, debating, and deconstructing further. If you take issue with anything I’ve said here, I wouldn’t be surprised, and I would welcome your retort in the comments. Furthermore, I’d love to hear about any other tools which I haven’t described here, as I would hate to be missing out on some sweet test-driven nirvana.