small medium large xlarge

Testing for Web Services

Keeping It Real in the Wilds of the Web

by Noel Rappin

Generic image illustrating the article
  Just because you are using an external web API for your site doesn’t mean that BDD principles need to go out the window.  

Chances are that the web site you’re developing will be dependent on an external web service for some of its functionality—since a third-party API is often part of the solution for unimportant features like, you know, getting paid. It can be challenging to write your integration with this external service using a clean Behavior-Driven Design (BDD) process. But by letting a few basic ideas guide the design of your code and your tests, you can craft your critical web service interaction with confidence.

Start With an Integration Test

For the sake of argument, we’re going to assume a situation where the third-party Internet service also has an existing Ruby gem that interacts with it. This is informed by my experiences using the Braintree gem for web payment, and the whois gem for, well, Whois data. We’ll use the Whois case for most of the code because it makes for a more straightforward example.

Our testing solution for this third-party tool is going to depend on a fair amount of mocking and stubbing. There’s no good way around using doubles of some kind to test an external tool. Having your actual tests be dependent on the existence of a remote web server is a recipe for slow and unreliable tests.

Because the mocking we’re going to do essentially isolates various parts of the system and assumes that the test of the system is working, it’s very important that there be an integration test that exercises all the pieces in concert. We’ll write the integration test, and where that fails, we’ll drop down to unit tests and write the code from there. In the whois example, the whois call is being made implicitly to match the request ID with the user’s provider, for the purpose of populating information about the user’s Internet service.

  Given I am a registered and confirmed user
 Scenario: Whois lookup for an existing provider
  When I make a request from an IP for a known provider
  Then my provider is the default choice on the comment form

The point of interaction with the third-party library is in the first step, When I make a request from an IP for a known provider. The results of that interaction show up in the second step. Note the explicit mention of a “known” provider. Among other things, this sets up the possibility of testing error conditions with a different step definition.

We’re not going to get the Cucumber step to pass without writing code but we can at least set up the step definition and get an actual failure that we can use to drive our RSpec tests and code. For the first time, we come up against the dilemma of testing a third-party library and having to avoid actually calling the library during testing.

The way out of the dilemma from a Cucumber test is to realize that while a Cucumber test is meant to treat your application as a black box, you don’t need to treat the entire Internet the same way. For the Whois example, I probably should have dug into the whois gem to determine how the gem makes the call to whois, and either mocked that actual call, or mocked the internals of the gem. Instead, I created a mock at the surface of the gem. (For the payment gateway or a service that uses HTTP, it’s easy to fake the expected HTTP call using Webmock or Fakeweb.)

 def mock_whois_client(expected_result)
  mock_answer = mock(:technical => (mock(:name => expected_result)))
  mock_client = mock
 When /^I make a request from an IP for a known provider$/ do
  @my_provider = "Xfinity"

This is sort of cheating, in the sense that I’m faking a call that I’m not confident in. As we’ll see in a bit, that will come back to bite me.

If you are doing the HTTP mocking, then for the initial test, I’d recommend you gather the sample data by using a web browser to make the actual call and save the text in a file in your app. At the end, we’ll talk about keeping this file up to date.

Move to Unit Tests

With the Cucumber test failing because there’s no code in place actually making the third-party call, it’s time to write that code. The most important thing you can do here is to wrap the external API in a module or class that fully belongs to you. This is true even if, as with Whois or Braintree, the third-party service already has a Ruby gem that wraps the actual Internet call.

The standard Object-Oriented rationale for doing something like this is that it allows you to have your own API in your own domain terms that are more meaningful and possibly simpler than the everything-to-everybody third-party tool. I’m not going to be contrarian about it. Even where you aren’t doing much beyond just calling the external API, you’ll have integration points with your own models, which are easiest to manage in a single place. In addition, having a wrapper makes it easy to consolidate common logic that you might need before or after your external call.

It’s very useful to have the rest of your program not care about the details of the actual library. This gives you a single, small, tightly controlled place where the interaction with the third party lives—which makes the actual interaction easier to test, and also makes the rest of your application less dependent on the specific third-party library you are using—which is great if you suddenly have to switch.

When testing, once you have the wrapper, other tests can mock or stub at the wrapper level, not at the third-party API level. This is helpful because usually your API will be easier to stub than the actual web service (which might involve, say, an annoying set-up call to something like Webmock).

All that said, you need to be 100% sure that the wrapper is golden, because none of your other tests are going to touch it.

The wrapper itself should have methods that are meaningful in your application domain, and should return real, usable data to the rest of your application. Meaning if the third party is giving data that needs to be parsed before use, that should be the wrapper’s responsibility. Ideally, the wrapper is returning model objects, but depending on the particulars of your system, returning a Hash might be better. As far as the rest of your application is concerned, this data is already suitable for being used by the actual business logic of your model.

A strict BDD approach, then, would have you testing this code in two stages. Stage one is in the logic code of the application, and establishes that the model calls our wrapper. In that test, we can mock, but at the wrapper level, not at the third-party level. In the Whois code, the logic that calls the whois wrapper is a class method of my ServiceProvider class, so that’s where my main logic tests go.

 describe ".find_or_create_from_whois_lookup" do
  let!(:provider) { Factory(:service_provider, :name => "Xfinity") }
  it "finds existing by whois lookup" do
  mock(:name_lookup => "Xfinity"))
  result = ServiceProvider.find_or_create_from_whois_lookup("") == provider == ["Xfinity"]
  it "creates non-existing by whois lookup" do
  mock(:name_lookup => "Verizon"))
  result = ServiceProvider.find_or_create_by_whois_lookup("") == "Verizon" =~ ["Xfinity", "Verizon"]
  it "does not create if nil" do
  mock(:name_lookup => nil))
  result = ServiceProvider.find_or_create_by_whois_lookup("")
  result.should be_nil == ["Xfinity"]

Testing the application model before you create the wrapper has two design advantages—it allows you to design the wrapper API based on actual use cases, and it ensures that you only add features to the wrapper that are actually driven by an application need. Building this kind of wrapper is definitely a case where it’s easy to get caught up in the wrapper ahead of any actual usage. I mean, so I hear.

The following code should pass those tests:

 def self.find_or_create_by_whois_lookup(ip)
  name =
  return nil if name.blank?
  provider = find_by_name(name)
  if provider.nil?
  provider = create(:name => name)

As for the wrapper itself, the most basic case is to have it be a class whose methods call methods of the original API. As a matter of personal preference, I don’t like for the wrapper to be a singleton, and I don’t like having all the wrapper methods be class methods. I’d rather actually create a new instance every time I make a call, as in In part this is because when the wrapper is a normal instance, it’s easier to add further transaction logic later on.

That’s the basic plan, but you can make the wrapper more complicated. The last time I worked with a payment gateway, our application had a lot of custom subscription logic that needed to be tracked on our side. We wound up implementing the basic wrapper as a class and the subscription logic in a Subscribable module that was mixed in to various entities that could own subscriptions, so that module was the only point of contact with the wrapper.

Writing a Wrapper

We’re building kind of a chain of test responsibility here. The model test assumes that the wrapper works, passing responsibility to the wrapper test, which assumes that the third-party API works as specified. Which prompts the question of how to test the wrapper to really nail down the wrapper functionality.

There are two basic solutions. One is the same as we used for the integration tests, namely stub the actual external requests that the wrapper will make. This approach is consistent with the general principle that you shouldn’t mock a library you don’t control. However, managing the Webmock calls for all the different kinds of API requests can be a pain. In general, you’ll need to make a larger number of API requests from the unit tests than from the cucumber tests.

If the third-party library exists as a gem, then another option is to break the general principle and mock the gem’s methods, like so:

 <code language="ruby">
  describe "WhoisWrapper" do
  it "#name_lookup" do
  mock_answer = mock(:technical => (mock(:name => expected_result)))
  mock_client = mock
  Whois::Client.should_receive(:new).and_return(mock_client)"").should == "Xfinity"
  it "#returns nil on timeout" do
  mock_client = mock
  Whois::Client.should_receive(:new).and_return(mock_client)"").should be_nil

The main downside to this plan is that if your test is mocking the wrong API method, you’ll never know. (To be fair, that’s a potential problem in the Webmock strategy as well, but actually exercising the API to generate the fake data helps find that kind of error.) I’d still lean toward this approach in cases where the wrapper is very simple, especially for the conversion of the output—simple JSON, for example. If the wrapper is doing more elaborate parsing on the output, I’d tend to want more realistic fake output and would be more likely to use the Webmock option.

Here’s the wrapper, at a first pass:

 class WhoisWrapper
  attr_accessor :whois
  def initialize
  @whois =
  def name_lookup(ip_addr)
  answer = whois.query(ip_addr)
  rescue Exception

A funny thing happened to me when I tested the Whois code in the browser. I was confident—the Braintree code had been much more complex and had gone pretty well when I went to manually test it. So I was surprised when the Whois code failed because the answer object above had no technical field. (I was basing the API on the online docs.)

Like I said, I took a shortcut in my integration tests, and now it was time to pay. I assumed the Whois output would be correctly parsed by the gem and, for the specific queries I was making, the gem didn’t parse the text. Luckily, my parsing needs were few, and because I had the wrapper already, I was able to get things going with only a few minor adjustments. I changed all my mocked Whois output to something like this:

 mock_answer = mock(
  :content => "Something\nOrgName: Xfinity\nSomething else")

That’s not perfect, it’s still a lightly processed version of the output, but it’s better, because it assumes less processing from the gem (the gem always puts raw content in the content attribute). Ideally, I’d go back and mock the whois gem’s actual external call—kind of complicated in this case because of the way the whois gem works. In any case, the wrapper method now looks more like this—the actual method is more complex to handle other cases, but this gives you the basic idea:

 def name_lookup(ip_addr)
  answer = whois.query(ip_addr)
  org_match = answer.content.match(/OrgName:\s*(.*?)$/m)
  if org_match then org_match[1] else nil end
 rescue Exception

Keeping it Fresh

At this point, we have, at least in theory, a well-tested wrapper that contacts the API, a well-tested model that interacts with the wrapper, and an integration test that merges the two. What’s missing? Protection against changes in the external service.

There are two related ideas that are handy for keeping the code in sync with the Internet service. You could do a smoke test, or you could do an active fixture refresh. Or both; they aren’t mutually exclusive.

A smoke test runs some or all of your cucumber tests that touch the third-party library, but without the Internet call being mocked. In other words, it’s kind of a super-integration test. You don’t want to run this all the time, of course, because it’s slow and unreliable. But if you run it daily, or every couple of days, you’ll have some warning if the web API changes or is otherwise flaky. (This is also a good thing to have your Continuous Integration tool do.)

For the logistics of the smoke test, mark any such test with a Cucumber tag—@smoke springs to mind. Then, head into your cucumber.yml file and add --tags ~@smoke to the std_opts line—you need to have a separate tags option so that the “don’t run smoke” tag is logically ANDed with the “don’t run wip” tag that should already be there.

That way, the regular cucumber task will skip smoke tests the same way it skips tests marked @wip. You’ll also want to add a rake task for cucumber:smoke.

 smoke: --tags @smoke features

Then the task can be added to the lib/tasks/cucumber.yml file, something like this:{:smoke => ’db:test:prepare’},
  ’Run smoke tests’) do |t|
  t.binary = vendored_cucumber_bin
  t.fork = true # You may get faster startup if you set this to false
  t.profile = ’smoke’

An active fixture refresh is simpler. All that entails is automating the manual process we used for getting sample data for our mocks. You can do this with a Rake task that goes to the shell and uses wget to access the remote page, or in the case of the Whois gem, just calls whois from the shell.

Again, this isn’t something you’d want to run all the time, but every few days or so will keep your confidence in the tests high.

And in the End

Just because you are using an external web API for your site doesn’t mean that BDD principles need to go out the window. Keep working in BDD and remember this:

  • Always start with an integration test.

  • All interactions with the API library should go through a wrapper that is part of your application. Tests should mock at the level of your wrapper.

  • Only use a mock for the actual third-party library if the only thing you are testing is your call to that library, and even then consider placing the mocks at the level below the third-party call.

  • Use a smoke test or an active fixture refresh to keep your tests in line with the API.

Armed with these tips, you can work with what might be the most important part of your application with confidence.

Noel Rappin is a Senior Consultant at Obtiva. A Rails developer for six years, Noel has spoken at RailsConf and Windy City Rails, and is the author of Rails Test Prescriptions. A blog relating to this book can be found at

Send the author your feedback or discuss the article in the magazine forum.