Joshua Wood Web Developer

Recording inbound Rack requests for test fixtures

One of my open source projects over at Honeybadger.io is a Ruby gem which receives webhooks from a variety of inbound email services (works with any Rack-based application) and converts them into a standard Ruby Mail::Message.

The main benefit of this is that one really doesn't need to know anything about the format of the webhooks they are consuming -- they just pass a Rack::Request to Incoming! and get back the well known Mail::Message. Incoming! also makes it a snap to switch email services.

Testing all of the various webhook payloads is not a snap, unfortunately; some post JSON, some post form parameters, and some post raw email messages as the request body. The other day I was trying to write some integration specs for the various strategies. I was looking for something which would basically record an inbound request and then allow me to replay it later. I found one option, but it wasn't quite what I was looking for -- what I wanted is basically VCR in reverse.

After trying a few failed approaches, it struck me that since a Rack::Request is the only dependency of Incoming!, that's really all I need to test it. And because a Rack::Request is basically just a Rack env Hash, then I can dump that Hash to a file and reload it as a fixture in my specs. To accomplish the first half of this (creating the fixtures), I built a little Rack application which I named FixtureRecorder:

# spec/recorder.rb
require 'rack'

FIXTURES_DIR = File.expand_path('../../spec/fixtures', __FILE__)

class FixtureRecorder
def initialize(app)
@app = app
end

def call(env)
env['fixture_file_path'] = file_path_from(env)
begin
@app.call(env)
ensure
File.open(env['fixture_file_path'], 'w') do |file|
file.write(dump_env(env))
end
end
end

def file_path_from(env)
file_path = env['PATH_INFO'].downcase.gsub('/', '_')[/[^_].+[^_]/]
f
ile_path = 'root' unless file_path =~ /\S/
F
ile.join(FIXTURES_DIR, [file_path, 'env'].join('.'))
end

def dump_env(env)
safe_env = env.dup
safe_env.merge!({ 'rack.input' => env['rack.input'].read })
safe_env = safe_env.select { |_,v| Marshal.dump(v) rescue false }
Marshal.dump(safe_env)
end
end

app = Rack::Builder.new do
use FixtureRecorder
run Proc.new { |env|
[200, {}, StringIO.new(env['fixture_file_path'])]
}
end

Rack::Handler::WEBrick.run app, Port: 4567

The approach is similar to requestb.in, except that it runs locally and records the Rack env directly to a fixture file. The fixtures are named using the requested path name so that I can point multiple webhooks to the same server:

A local request to http://localhost:4567/sendgrid results in the fixture spec/fixtures/sendgrid.env

In my specs, I have a little helper to load the file contents back into Ruby and instantiate the Rack::Request:

# spec/spec_helper.rb
require 'rspec'
require 'rack'

FIXTURE_DIR = File.expand_path('../../spec/fixtures', __FILE__)

RSpec.configure do |c|
# ...
module Helpers
def recorded_request(name)
fixture_file = File.join(FIXTURE_DIR, "#{name}.env")
env = Marshal.load(File.read(fixture_file))
env['rack.input'] = StringIO.new(env['rack.input'])
Rack::Request.new(env)
end
end

c.include Helpers
end

The rest is pretty simple. In order to record fixtures, I first start the Rack server:

ruby spec/recorder.rb
[...] INFO WEBrick 1.3.1
[...] INFO ruby 2.0.0 (2013-06-27) [x86_64-darwin12.4.0]
[...] INFO WEBrick::HTTPServer#start: pid=85483 port=4567

Next, I launch ngrok to expose the Rack server publicly:

ngrok                                              (Ctrl+C to quit)

Tunnel Status online
Version 1.6/1.5
Forwarding http://56e10a0f.ngrok.com -> 127.0.0.1:4567
Forwarding https://56e10a0f.ngrok.com -> 127.0.0.1:4567
Web Interface 127.0.0.1:4040
# Conn 0
Avg Conn Time 0.00ms

Lastly, I configure each webhook to post to http://56e10a0f.ngrok.com/strategy-name, and then send some emails. The requests are received, and the Rack env is dumped to spec/fixtures/strategy-name.env.

And that's it. Using the fixture in action looks like this:

# spec/integration_spec.rb
require 'spec_helper'

describe Incoming::Strategies::Mailgun do
let(:receiver) { test_receiver(:api_key => 'asdf') }

describe 'end-to-end' do
let(:request) { recorded_request('mailgun') }
before do
OpenSSL::HMAC.stub(:hexdigest).
and_return(request.params['signature'])
end

it 'converts the Rack::Request into a Mail::Message' do
expect(receiver.receive(request)).to be_a Mail::Message
end
end
end