Ruby Test::Unit sucks! (and why I still use it)

I’ll be honest: I hated Test::Unit. After a year of immersing myself in Ruby, Test::Unit felt like a horrific Java throwback1. With the advent of RSpec, all of the cool kids were writing DSL-flavored tests.

I spent a year working with RSpec where I thought that I was happy. I loved the snazzy DSL. I could play at being one of the cool kids (um, hello, this is the twenty first century calling. You were 34 at the time so not much of a “kid” anymore. And cool? Heh.).

Somewhere along the way, I met RSpec Story Runner. It was my first exposure to the DDD-BDD (now TDD, I suppose) style Given-When-Then approach. It neatly handled the nesting that I experienced in Shoulda by making all of the aspects of the test linear while maintaining documentation on a per step basis. It wasn’t easy to set up but, hey, it was a work in progress.

But then I got a gig working with some folks who had hardly written tests much less TDD’d regularly.

So I brushed up on Test::Unit, learned myself up on shoulda, and then dove headfirst into writing Rails unit (really “model”) and integration (really “acceptance”) tests with Shoulda and good ol’ T::U.

And, after about a year, familiarity bred trust.

You see, when you get past the API, Test::Unit has a lot going for it:

  • It’s internally consistent. Expected values go on the left and, for comparative assertions, actual values the right.
  • It’s relatively small – at least when you compare it to RSpec (and, yes, even in Ruby 1.8.7). And it gets even smaller in Ruby 1.9.x with Minitest (and with some bitchin’ new assertions).
  • It’s ubiquitous. Using Ruby? Great. Then you already have Test::Unit. Look in ruby/lib/test.
  • It doesn’t do much (in Ruby 1.9.x at least). Test::Unit doesn’t try to solve every testing problem. It just provides enough to write and run tests2
  • It’s composable

Let’s talk about that last point.

When I start TDDing a new application, I very quickly begin to repeat myself in my tests. So, where possible, I DRY my tests such that they are readable yet still tractable (I add that caveat only because, in the past, I’ve “over-macro’d” on several occasions). This means that, very quickly, I’m extracting tests into methods. In essence, I wind up building my own DSL atop Test::Unit.

RSpec adherents will likely argue that, with RSpec, you can do the same thing. This is true. However, RSpec tends to obsess over the semantics of each and every assertion. Test::Unit is already just a little handicapped that way. That is:

i.should have(1).porkchop_sandwiches

… would read something more like:

assert_equal 1, i.porkchop_sandwiches.count

… which is less English-like (and bear with me if my RSpec isn’t spot on as its been a while since I’ve used RSpec in anger). If that assertion is living inside of a well-named method like i_have_1_porkchop_sandwich, do I really care that the one line doesn’t read like perfect English? It’s still very straight-forward code! It’s good enough for me!

Besides, if I am likely to resuse most of my tests anyway (and I do reuse a lot), I could just use assert, with a useful error message, for each and every one if I was feeling particularly lazy.

But, then, I found that Shoulda was missing a feature that I loved in RSpec: “spec –format nested”. So I wrote a klugey little Rake task to do something similar for Shoulda. But that wasn’t quite enough.

After another few months, I realized that Shoulda was just making acceptance testing too painful. Nesting contexts three or four levels deep made for some genuinely ugly tests. And code reuse in my tests? That wasn’t happening. Sure, Cucumber was already on the scene. But the call-by-regexp approach that it took and how it forced me to explicitly separate feature definition from step definition pissed me off.

So I wrote Coulda. The basic core of Coulda didn’t take too long at all to write.

Why?

Test::Unit doesn’t support mocking. If you want mocking, get another gem. It doesn’t support matchers. If you really want matchers instead of assertions, get another gem (but, really, why?). It’s my second-to-last point above: T::U just isn’t that complicated. This makes it inviting to extensions3.

Whereas looking in RSpec’s lib caused me to cringe back in the 1.x days (it’s still pretty complicated under the hood in 2.x too), Test::Unit (mostly, for 1.8.x) just makes sense.

Simple, specialized tools are often easier to work with. There is less of a learning curve because there is just less to learn. This means that you’ll become productive faster. Once you are, who cares if you’re using Test::Unit or RSpec?

So just use the simple one that’s already there. Or at least consider it.

Besides, being a cool kid is highly overrated.

UPDATED 12/20/11

Some folks here and several on Hacker News seem to be using vanilla Test::Unit. That’s ok, however, I find a lot more value in using it with nested contexts that use metaprogramming internally to create your test methods for you. To name a few:

Coulda is really for acceptance/integration testing but it that doesn’t prevent it from being used at lower levels. YMMV but, FWIW, I only use it for the former.

  1. As it turns out, Nathaniel Talbott told me a few years ago that Test::Unit was among the first code that he wrote in Ruby – after switching from Java. ;-)
  2. Well, ok, the 1.8.x version supports several I/O approaches which is the cause of a lot of the bloat in 1.8.x Test::Unit.
  3. With a few hiccups in 1.8.x such as how it runs using the on_exit callback

Posted by evan on Saturday, December 18, 2010

blog comments powered by Disqus