December 2010 Archives

Ruby Test::Unit sucks! (and why I still use it)

I’ll be honest: I hated Test::Unit. After a year of immersing myself in Ruby, Test::Unit felt like a horrific Java throwback1. With the advent of RSpec, all of the cool kids were writing DSL-flavored tests.

I spent a year working with RSpec where I thought that I was happy. I loved the snazzy DSL. I could play at being one of the cool kids (um, hello, this is the twenty first century calling. You were 34 at the time so not much of a “kid” anymore. And cool? Heh.).

Somewhere along the way, I met RSpec Story Runner. It was my first exposure to the DDD-BDD (now TDD, I suppose) style Given-When-Then approach. It neatly handled the nesting that I experienced in Shoulda by making all of the aspects of the test linear while maintaining documentation on a per step basis. It wasn’t easy to set up but, hey, it was a work in progress.

But then I got a gig working with some folks who had hardly written tests much less TDD’d regularly.

So I brushed up on Test::Unit, learned myself up on shoulda, and then dove headfirst into writing Rails unit (really “model”) and integration (really “acceptance”) tests with Shoulda and good ol’ T::U.

And, after about a year, familiarity bred trust.

You see, when you get past the API, Test::Unit has a lot going for it:

  • It’s internally consistent. Expected values go on the left and, for comparative assertions, actual values the right.
  • It’s relatively small – at least when you compare it to RSpec (and, yes, even in Ruby 1.8.7). And it gets even smaller in Ruby 1.9.x with Minitest (and with some bitchin’ new assertions).
  • It’s ubiquitous. Using Ruby? Great. Then you already have Test::Unit. Look in ruby/lib/test.
  • It doesn’t do much (in Ruby 1.9.x at least). Test::Unit doesn’t try to solve every testing problem. It just provides enough to write and run tests2
  • It’s composable

Let’s talk about that last point.

When I start TDDing a new application, I very quickly begin to repeat myself in my tests. So, where possible, I DRY my tests such that they are readable yet still tractable (I add that caveat only because, in the past, I’ve “over-macro’d” on several occasions). This means that, very quickly, I’m extracting tests into methods. In essence, I wind up building my own DSL atop Test::Unit.

RSpec adherents will likely argue that, with RSpec, you can do the same thing. This is true. However, RSpec tends to obsess over the semantics of each and every assertion. Test::Unit is already just a little handicapped that way. That is:

i.should have(1).porkchop_sandwiches

… would read something more like:

assert_equal 1, i.porkchop_sandwiches.count

… which is less English-like (and bear with me if my RSpec isn’t spot on as its been a while since I’ve used RSpec in anger). If that assertion is living inside of a well-named method like i_have_1_porkchop_sandwich, do I really care that the one line doesn’t read like perfect English? It’s still very straight-forward code! It’s good enough for me!

Besides, if I am likely to resuse most of my tests anyway (and I do reuse a lot), I could just use assert, with a useful error message, for each and every one if I was feeling particularly lazy.

But, then, I found that Shoulda was missing a feature that I loved in RSpec: “spec –format nested”. So I wrote a klugey little Rake task to do something similar for Shoulda. But that wasn’t quite enough.

After another few months, I realized that Shoulda was just making acceptance testing too painful. Nesting contexts three or four levels deep made for some genuinely ugly tests. And code reuse in my tests? That wasn’t happening. Sure, Cucumber was already on the scene. But the call-by-regexp approach that it took and how it forced me to explicitly separate feature definition from step definition pissed me off.

So I wrote Coulda. The basic core of Coulda didn’t take too long at all to write.


Test::Unit doesn’t support mocking. If you want mocking, get another gem. It doesn’t support matchers. If you really want matchers instead of assertions, get another gem (but, really, why?). It’s my second-to-last point above: T::U just isn’t that complicated. This makes it inviting to extensions3.

Whereas looking in RSpec’s lib caused me to cringe back in the 1.x days (it’s still pretty complicated under the hood in 2.x too), Test::Unit (mostly, for 1.8.x) just makes sense.

Simple, specialized tools are often easier to work with. There is less of a learning curve because there is just less to learn. This means that you’ll become productive faster. Once you are, who cares if you’re using Test::Unit or RSpec?

So just use the simple one that’s already there. Or at least consider it.

Besides, being a cool kid is highly overrated.

UPDATED 12/20/11

Some folks here and several on Hacker News seem to be using vanilla Test::Unit. That’s ok, however, I find a lot more value in using it with nested contexts that use metaprogramming internally to create your test methods for you. To name a few:

Coulda is really for acceptance/integration testing but it that doesn’t prevent it from being used at lower levels. YMMV but, FWIW, I only use it for the former.

  1. As it turns out, Nathaniel Talbott told me a few years ago that Test::Unit was among the first code that he wrote in Ruby – after switching from Java. ;-)
  2. Well, ok, the 1.8.x version supports several I/O approaches which is the cause of a lot of the bloat in 1.8.x Test::Unit.
  3. With a few hiccups in 1.8.x such as how it runs using the on_exit callback

Posted by evan on Dec 18, 2010

Why should I work for you again?

Literally, for months now, I have been discussing with other developers, “Why is our one sector booming when there is 10% unemployment?” Where is all of the money coming from? What makes us special?

Apparently, others are asking similar questions.

Today, Jon Steinberg posted to Hacker News that there is 10% unemployment yet every startup in NYC [is] struggling to hire.

One respondent posited that there may be too many startups. Another questioned whether the startups are limiting their search to people who can work co-lo’d. Yet another wondered if startups are being too picky. And still another complained of the lack of apprenticeship occurring in our industry.

Jon is certainly on to something. The past several years have witnessed downturns in graduates with computer science degrees. This probably correlates with the tech bubble that burst in ‘00[1]. No matter how you feel about the validity of a college education, this statistic is still telling.

While all of the aforementioned answers are likely partially responsible, the largest culprit is that the IT startup industry as a whole is not marketing itself well.

Two rounds of boom and bust in ten years? That doesn’t spell “career choice” for a lot of people.

Most people are risk averse. They want predictability and safety. They’re not entrepreneurs. But not every startup worker is an entrepreneur! The risks of working at a startup are largely (1) that your equity will be worthless and (2) the startup will fold losing causing you to once again ask, ”Who moved my cheese?

So what’s missing?

In a word: marketing.

To the layity, the typical “reward” for working in the startup industry are long hours for lower pay than you’d get in a more stable job (right… where are those again?) followed by the periodic need to search for a new job because, oh shit, my latest employer is laying people off again because they ran out of funding.

It’s worse than that.

There are job boards everywhere. But most job listings that I see on job boards (and receive via email) all read like form letters:

Hi, my name is Foo and I work for Bar, Inc.  
We're looking for developers who are skill in Froboz!

You should also be skilled in:
  * Baz
  * Blech
  * Bla
  * 42

We provide competitive pay and offer a, get this, 
whopping *2* whole weeks where you get to come up for air and not work.


This is supposed to appeal to the currently overemployed individual how exactly?

For crying out loud, people, be passionate! Tell candidates why what you’re doing is awesome. Make them want to come work for you.

If you’re not mentoring entry level people you’d better be!

If you can’t do that, well, you can always put another exciting job listing on 37Signals for $400/mo.

But Don’t panicTM! Computer Science enrollments are going up again[2] so, pretty soon, there will be parity or another bubble to ruin it all for us.

* = Yet Another Stupid Recruiter

[1] IT workers’ scarcity and skills

[2] Computer Science Major Counts Up Second Year in a Row

Posted by evan on Dec 12, 2010

Rule 0: Choose the right people

Patti Chan, from Intridea, released an interesting post today on managing remote teams. Having managed remote teams for a year now, I empathized with each and every point.

However, to that, I’ll add a Rule #0: choose the right people. While it may sound banal, I’m thoroughly convinced that only a subset of otherwise solid workers can adapt themselves to working remotely. It requires a certain attribute that, so far, I can only summarize as “entrepreneurship”.

That is, a good remote worker is someone who, under the right conditions, is likely capable of starting their own business and succeeding without you. They can communicate effectively, know when (and when not) to, can make and keep commitments, and they can do it without the short term accountability of pair-programming, other forms of micro-management (yes, Virginia, pair-programming is a social contract form of micro-management), and the implicit peer pressure of working co-lo (and don’t even pretend to tell me that there’s no implicit peer pressure when you see some stupid/crazy bastard arriving at 8am and working until 8pm).

Communication is truly key here. As Patti pointed out, presence in a chat room is important. However, it’s more important that individuals engage one another in that shared chat. It is too easy to lurk in a chat room.

To that extent, I’ve found that extroverts tend to be more successful remote workers. Extroverts are better at overcoming the additional friction in communicating with a remote team. Yes, typing an IM or into a chat room, double clicking a person’s name in Skype and hitting the “Call” button, or just picking up the phone is harder than turning to the guy next to you and asking, “Hey, Harvey, what do you think of this?”. In my experience, extroverts are more inclined to overcome that friction because the desire to share and to externalize is part of their nature.

If you found this useful or insightful, please do give me an upvote on Hacker News.

Posted by evan on Dec 10, 2010

Making Devise play nicely with Postgres (case sensitivity)

Devise, in case you haven’t used it, is a wonderful authentication solution for Rails. It’s essentially a Rails engine that handles authentication for you.

However, one speed bump that I encountered in paradise is that, when used with Postgres, the password reset and and login mechanisms can have issues. The problem is a simple one: Postgres is case-sensitive.

The solution is just a couple of minor tweaks. Add the below to your user (or whatever you call it) ActiveRecord model and all will be well.

Update: This appears to be fixed, at least in part or in total, in the current 1.2 release candidate.

Posted by evan on Dec 03, 2010

Stop writing code you don't need!

The situation

You’re writing unit tests for your new web app to let people create their own custom oatmeal-flavored razor blades when you have an epiphany. Ther razor blades are worthless without an accompanying yak scented razor!

So you start writing some unit tests for your innovative razor.

Then another idea drops on your brain like a Pan-Galactic Gargle Blaster. The razor should also be usable as a rake because, remember, Fall is here and there are leaves on the ground!

So then you start writing your tests like a good little TDD’er to add a rake to your yak-scented razor that uses oatmeal-flavored blades.

And then, ooh, what if we store the state of the face being shaved and yard being raked in a bag of holding. Everyone says that the bag of holding is web scale!


The problem

In a nutshell, bottom-up thinking leads to featureitis (yes, an inflamation of features). When you see your problem domain through the lens of unit tests, your inclined to consider features that you may not need.

The solution

You need to drive your development from the user’s level down through to the model level.

You need to change your perspective.

You need to do Test Driven Development (a.k.a Behaviour Driven Development) – but using acceptance/integration tests first and supporting them with unit tests as necessary. Start by writing an acceptance/integration test that describe a feature that your user needs. Then you make the test pass (hellooooooooo red-green-refactor). Rinse and repeat until you’ve written tests that describe each feature needed by your user(s).

Congratulations! Your app is (mostly*) complete!

More to the point: you probably wrote only the code that you need to satisfy the needs of your user(s). This will save you time and you, or your customers, money.

*: Complete is relative. You rarely know that your application handles every possible condition

Posted by evan on Dec 02, 2010