small medium large xlarge

Testing as Contract

A Modest Proposal

by Brian Tarbox

Generic image illustrating the article
  Are software tests too important to be left to just any programmer?  

One of the realities of development these days is that our software teams have heterogeneous skill levels. Most teams contain a mix of veterans and rookies. There can be large differences in how seriously people take continuing education, too. My team, for example, has a member who just announced that he never learned any of the Java 5 features and so couldn’t work on areas that used generics. Considering that we’re now migrating from Java 6 toward Groovy, this team member is pretty far behind the curve.

As a field we keep trying to find procedures, methods, or paradigms to deal with the fact that not all developers are equal. It goes against the grain to say that some developers are simply better than others, but the average developer is—well—average!

One of the approaches we’ve used to deal with differing skill levels is Test Driven Development. Most shops now at least pay lip service to the importance of unit testing—although from comparing notes at conferences, I’d say the number of shops that actually require that the tests be written before the “real” code is still pretty small.

I want to argue that TDD focuses on the wrong problem.

The developers who understand the importance of unit tests and who write comprehensive and subtle tests are exactly the group that need them the least. The “average” or inexperienced developer probably doesn’t believe in unit tests and will write bad or pointless tests if forced to do so. I have empirical evidence that they won’t write tests that aggressively try to break their classes. I’m embarrassed to admit that I’ve had to turn on CheckStyle’s “unit test must contain an assert” test so as to force some of my team to write unit tests that actually test something.

Programmers with Class

But differences in skill levels aren’t a new phenomenon. Back in the C/C++ days there was a clear separation of skill levels among developers: some people wrote classes, most people just used classes. Some people wrote libraries, but most people just used those libraries. It was understood that not everyone was capable of designing good classes. Class design was a higher level skill that was left to our top people.

Then along came Java. Since everything was a class, everyone was suddenly a class designer. Except that they weren’t. It was still the case that only a small subset of developers could design good classes, but we weren’t supposed to care anymore.

Today we try to manage this situation with TDD, but the differences in skill and commitment still gum up the works.

Testing as Contract

But consider this: instead of hoping that each developer will write their own (good) test cases and only then write their code, imagine that the best developers in a team write the interfaces and all of the unit tests that exercise the interfaces. We then let everyone else code until the tests pass. It’s very similar to standard TDD except that we have the most skilled people writing the unit tests. They are in fact writing the enforcement of the contract.

I am attracted to this approach because it gives me a way to protect my design. My group is working on so many projects that I tend to write the initial design and perhaps the initial implementation, and then hand it off to a remote team to finish. This approach has had very mixed results. After trying to write exhaustive Functional Specs, or racking up too many Frequent Flier miles, or doing too many late night/early morning video conference calls, the essence of the initial design sometimes gets lost.

The most successful approach for me has been to write an exhaustive set of unit tests that act as a guard against changes to the design: in effect, to the contract. One result is that I get a steady stream of complaints that my unit test suite makes it harder for people to change the code, and I’m glad of that. I want people to have to think before changing the design.

In the past we’ve chastised developers whose code coverage numbers were not high enough. I’ve come to think that was the wrong approach. Perhaps in a perfect world every developer would write their own tests, but in the real world it may make more sense for the senior developers to create the design, write the interfaces, and then write a good suite of unit tests for all of the code. After all, if we believe that the tests are so important, shouldn’t we be the ones to write them?

Brian Tarbox is a Principal Staff Engineer at Motorola where he designs server side solutions in the Video On Demand space. He writes a blog on the intersection of software design, cognition, and Tai Chi at