small medium large xlarge

Great Expectations

Better Testing Through DSLs with Ruby and Scala

by Paul Butcher

Generic image illustrating the article
  Paul explores how to improve your tests with custom expectations, and does so using parallel implementations in Ruby and Scala.  

Domain Specific Languages (DSLs) have become a hot topic in development recently. And deservedly so, because when done well they allow us to raise the level of abstraction at which we work, achieving more while simultaneously improving quality and saving effort.

DSLs can help with all sorts of problems, including testing. I recently came across a great example, where by investing the time to create custom expectations that encode knowledge about my problem domain, I could dramatically improve the expressiveness of my tests.

Thanks to Rails, DSLs are probably best associated with Ruby. But Ruby certainly isn’t the only language that can play this trick. Scala is a language that’s rapidly gaining in popularity, not least thanks to its DSL support.

As a PragPub reader, I’m guessing that you’re probably at least somewhat familiar with Ruby, but maybe not with Scala. So I thought that I’d challenge you by presenting the same solution implemented in both languages. They’re interesting both for their similarities and for their differences.

If you’re not interested in Scala, you can skip over the Scala sections without losing anything.


I’ve used Ruby 1.8.7 and Test::Unit for the Ruby examples. The general approach should certainly work for other Ruby test frameworks like RSpec, however.

The Scala examples use version 2.8 and the ScalaTest framework.

All the code for this article is available for download here.

The Problem

I recently found myself writing tests for a body of code that performs statistical analysis of large bodies of data. One of the key data structures was a weighted list, which is a list of items, together with their associated weight, ordered from highest weight to lowest.

You won’t be surprised to learn that many of our tests take the general form:

  1. Set up a particular test dataset

  2. Analyze that dataset

  3. Verify that the resulting weighted list matches our expectations

It is the last of these three steps that we’ll concentrate on.

A Naïve Solution

Here’s what a test might look like in Ruby:

 def test_naive
  items =
  assert_equal(3, items.length) // Line 1
  assert_equal("a", items[0].name) // Line 2
  assert_equal("b", items[1].name)
  assert_equal("c", items[2].name)
  weights = items.collect(&:weight) // Line 3
  assert_equal(weights, weights.sort.reverse, "unsorted weights")

The call to on Line 1 returns:

 [("a", 0.7), ("b", 0.2), ("c", 0.1)]

At Line 2 we verify that exactly 3 items are returned, and that they are “a,” “b,” and “c” in that order.

At Line 3 we verify that the items are returned in weight order by comparing a list of weights with a sorted version of itself (the call to reverse is necessary because we need to sort from highest to lowest).

Fine as far as it goes. But we’re going to have to do this kind of thing for pretty much every test that looks at a weighted list (and we have a lot of tests). Surely we can do better?

Naïvety in Scala

Before we look at what we can do to improve things, let’s take a look at what the same test might look like in Scala:

 def testNaive {
  val items =
  expect(3) { items.length } // Line 1
  expect("a") { items(0).name }
  expect("b") { items(1).name }
  expect("c") { items(2).name }
  val weights = // Line 2
  // Line 3:
  assert(weights.sortWith(_ > _) sameElements weights, "unsorted weights")

Pretty similar to the Ruby version.

The call to expect at Line 1 is basically just an assertion. In fact, we could have written:

 assert(3 == items.length)

but (like Test::Unit’s assert_equal) ScalaTest’s expect allows more helpful error messages to be displayed when an expectation fails.

The call to map at Line 2 is similar to Ruby’s collect—it passes each element of items to the supplied function and returns the resulting collection. But what does _.weight mean?

Well, we could have written this instead: => i.weight)

The argument to map is an anonymous function that, given an item, returns its weight. In many ways it’s very similar to a Ruby block, for instance:

 items.collect { |i| i.weight }

Scala allows us to simplify this, however. First, it allows us to avoid giving an explicit name to our anonymous function’s parameters by using _: => _.weight)

which we can then further shorten by removing the _ =>.

The same thing is going on with the argument to sortWith at Line 3. This is equivalent to:

 weights.sortWith((_, _) => _ > _)

which in turn is equivalent to:

 weights.sortWith((a, b) => a > b)

Something else a little odd is going on at Line 3. Scala’s sameElements method compares two collections and returns true if they contain the same elements. Scala allows us to use methods as though they were operators, however, so:

 x sameElements y

is equivalent to:


Getting Smarter

How cool would it be to encode all of our knowledge about what a weighted list should look like in a single assertion? Something that we can use like this:

 assert_items(["a", "b", "c"],

Well, it turns out that we can do exactly that:

 def assert_items(expected, actual)
  actual_names = actual.collect(&:name) // Line 1
  actual_weights = actual.collect(&:weight) // Line 2
  // Line 3:
  assert(expected == actual_names &&
  actual_weights == actual_weights.sort.reverse,
  "Expected [#{expected.join ', '}] but got [#{actual.join ', '}]")

At Line 1 and Line 2 we split our weighted list into a list of names and a list of weights. Then at Line 3 we verify two things—that the names match the expected list, and that the weights are ordered correctly.

We’ve not only gained brevity, we’ve also improved our reporting when a test fails. For example:

 assert_items(["a", "b", "c"], Items.abc_unsorted)

gives us:

 Expected [a, b, c] but got [(a, 0.2), (b, 0.7), (c, 0.1)].

Smarter Scala

Here’s what using our custom expectation looks like in Scala:

 expectItems("a", "b", "c") { }

and here’s how it’s implemented:

 def expectItems(expected: String*)(actual: Seq[Item]) { // Line 1
  val actualNames =
  val actualWeights =
  val sortedWeights = actualWeights.sortWith(_ > _)
  assert((expected sameElements actualNames) &&
  (actualWeights sameElements sortedWeights),
  "Expected "+ format(expected) +" but got "+ format(actual))

Most of this should be pretty obvious, given what you’ve already seen, but Line 1 introduces a few new things.

Firstly, we need to give types for our function parameters. Notice however that, unlike Java, the compiler can infer types in most cases. So, for example, it can work out that the type of actualNames is Seq[String] (a sequence of String) without us telling it explicitly.

Secondly, the type of the expected parameter is String*. This means that the function can be called with any number of string arguments—expected can then be used as though it were a Seq[String] containing all of the parameters.

Finally, our function takes two parameter lists. This is a trick commonly used in functional programming languages called currying. expectItems is actually a function that takes zero or more string arguments and returns another function which takes a sequence of Item objects as its argument.

Other than that, it follows pretty much the same structure as the Ruby equivalent.

But We Can Do Better

Our assert_items method is certainly an improvement over what we had before, but we can do better. For example, Items.abcdef returns:

 [("a", 0.6), ("b", 0.1), ("c", 0.1), ("d", 0.1), ("e", 0.06), ("f", 0.04)]

Testing it with:

 assert_items(["a", "b", "c", "d", "e", "f"], Items.abcdef)

would over-specify, because the “b,” “c,” and “d” items all have the same weight and could, therefore, be returned in any order.

We can address this by enhancing assert_items to allow it to be called like this:

 assert_items(["a", ["b", "c", "d"], "e", "f"], Items.abcdef)

Here’s the implementation:

 def assert_items(expected, actual)
  assert(matching_items(expected, actual),
  "Expected [#{format(expected)}] but got [#{actual.join ', '}]")

The heavy lifting has been moved to matching_items:

 def matching_items(expected, actual, last_weight = 1.0)
  if expected.empty? // Line 1
  e = expected.first
  case e // Line 2
  when Array:
  n = e.length
  a = actual.take(n)
  w = a.first.weight
  a.all? { |i| i.weight == w } &&
  w < last_weight &&
  e.sort == a.collect(&:name).sort &&
  matching_items(expected.drop(1), actual.drop(n), w)
  a = actual.first
  w = a.weight
  w < last_weight &&
  e == &&
  matching_items(expected.drop(1), actual.drop(1), w)

This is a recursive function that handles the first element of its expected argument and then calls itself to handle the remainder.

As with any recursive function we need to handle termination, which happens at Line 1, where we confirm that if we’ve run out of expected items, we also have no remaining actual items.

The heart of the function is the case statement at Line 2, which determines whether or not we’re dealing with an array.

If we are, we take the appropriate number of items from actual, verify that they all have the same weight, that that weight is less than the last we saw and that the names match what we’re expecting (sorting both so we can compare easily). We then recurse, passing the remainder again.

Otherwise, we take the first element of the actual array, verify that its weight is less than the last weight we saw and that its name equals the one we’re expecting. We then recurse, passing the remainder of expected and actual.

Doing Better in Scala

We could create a Scala implementation of expectItems that, like Ruby, takes a mixture of strings and arrays. But because Scala is statically typed, this isn’t the natural approach and not the one we’ll take.

Instead, we’re going to create a new class ItemSet that represents a set of equally weighted items:

 class ItemSet(names: Seq[String]) { // Line 1
  val sortedNames = names.sorted
  val size = names.size
  def matches(actual: Seq[Item], weight: Double) =
  (sortedNames sameElements &&
  actual.forall(_.weight == weight)

Scala’s constructors are specified rather differently from Ruby or Java—the arguments are given as parameters to the class (Line 1).

Our class has two fields, sortedNames and size and a method matches that returns true if its actual argument matches the items supplied when this object was created and they all have the supplied weight.

Here’s the implementation of expectItems:

 def expectItems(expected: ItemSet*)(actual: Seq[Item]) {
  assert(matches(expected, actual),
  "Expected "+ format(expected) +" but got "+ format(actual))

As in the Ruby implementation, the heavy lifting is performed by a recursive matches function:

 def matches(expected: Seq[ItemSet], actual: Seq[Item],
  lastWeight: Double = 1.0) : Boolean =
  if (expected.isEmpty)
  else {
  val (eh, et) = (expected.head, expected.tail) // Line 1
  val (ah, at) = actual.splitAt(eh.size) // Line 2
  val w = ah.head.weight
  w < lastWeight && eh.matches(ah, w) && matches(et, at, w) // Line 3

At Line 1, we split expected into its head element (eh) and tail (et) and at Line 2, we split actual into two lists, one with the same number of elements as eh, the other containing the remainder.

Finally at Line 3, we verify that the weight is less than the last we saw, that the expected items match the actual, and recurse.

We now have something that can be called like this:

 expectItems(new ItemSet(Seq("a")),
  new ItemSet(Seq("b", "c", "d")),
  new ItemSet(Seq("e")),
  new ItemSet(Seq("f")) {

which will work, but is rather more verbose than we would prefer.

Happily, we can do much better by defining a couple of helper functions:

 implicit def stringToItemSet(s: String) = new ItemSet(Seq(s))
 def ^(strings: String*) = new ItemSet(strings)

stringToItemSet converts a single String to an ItemSet. The trick is the implicit modifier, which means that the compiler will automatically call this function for us whenever we use a String where an ItemSet is required.

The ^ method takes a sequence of String parameters and converts them to an ItemSet.

So now we can call expectItems like this:

 expectItems("a", ^("b", "c", "d"), "e", "f") { Items.abcdef }

Much better!

Wrapping Up

Let’s review what we’ve achieved. We’ve created a custom expectation that allows us to write:

 assert_items(["a", ["b", "c", "d"], "e", "f"], Items.abcdef)

which asserts that:

  1. The returned sequence contains exactly 6 elements.

  2. The returned sequence is sorted in descending weight order.

  3. The first item is “a.”

  4. The next 3 items are “b,” “c,” and “d,” but these may be returned in any order.

  5. The weight associated with these three items is equal.

  6. The last two items are “e” and “f” in that order.

Not bad for a single line of code.

We’ve also seen that we can achieve the same effect in Scala, a statically typed and compiled language, just as easily as we can in Ruby, a dynamically typed and interpreted language.

Paul Butcher has worked in diverse fields at all levels of abstraction, from microcode on bit-slice processors to high-level declarative programming, and all points in between. Paul’s experience comes from working for startups, where he’s had the privilege of collaborating with several great teams on cutting-edge technology. He is the author of Debug It!: Find, Repair, and Prevent Bugs in Your Code..

Send the author your feedback or discuss the article in the magazine forum.