Python Functional HTTP Testing With Gabbi

Continuing with my journey in contributing to Gnocchi, I have learned of an excellent tool which Gnocchi uses to test its API from a HTTP request approach. The tool is called Gabbi.

Coming from a Ruby, Rails, and Rspec background, I was very pleased on learning how Gabbi works and how to use it. Gabbi uses YAML to construct the tests that will hit the API endpoints, in a similar way to using Rspec's DSL when making request specs.

For my latest contribution to Gnocchi, I was working on an issue where newly created metrics were not being returned in the response from /v1/resource/generic/:id/metric. After applying the fixes and submitting the patch, it was also necessary to update some of the functional Gabbi tests that belong to this specific use case.

One of these original tests is shown below:

Testing JSON API Strong Parameters in Rails

Recently I was experiencing a problem with a Rails API where I would update a model with new attributes, and then proceeded to add the new attributes to the model's factory, and then add the corresponding model and request specs.

The problem was that I would forget to whitelist the new attributes in the controller. Then running the request specs would not detect the error since I only test against one updated attribute that was whitelisted long ago. This became really annoying since I had no way to know if the new attributes were being whitelisted or not, and my tests weren't saying anything about it. This problem becomes even more apparent when the list of whitelisted attributes starts becoming very long.

In an API following the JSON API specification, here is an example of how the strong parameters method would look like:

def thing_params
  params.require(:data).require(:attributes).permit(:color, :size, :age, :name,
      :owner, :origin, :location, :purpose, :price, :alive)
end

As you can see, the list of attributes is getting pretty long. And we do not want to explicitly keep adding more attributes to the test.

Testing Strong Parameters in Controller Specs

After the release of Rails 5 I started ditching controller specs in favor of request specs. I like this approach and I think it's great, but I think it would be too much unecessary work to test that all the necessary attributes are whitelisted, in the request spec. Therefore I proceeded to create very simple controller specs that would test that all attributes of importance are whitelisted in the controller.

Understanding Gnocchi Measures

Measures are simple objects that represent the timeseries data. They simply contain a timestamp and a value, and they belong to metrics. You could say that a measure has many metrics.

Adding Measures to a Metric

Using the REST API, we can easily add measures to a metric:

POST /v1/metric/511b61a1-8d67-42d5-8add-66d4209a0469/measures HTTP/1.1
Content-Type: application/json
Content-Length: 198

Understanding Gnocchi Metrics

Metrics are one of the main object types in Gnocchi. They are identified by a UUID and they can also be attached to a resource by using a resource name. Metrics store measures, and the way they do this is defined by archive policies. These are concepts that I will cover in future articles.

Basically, a metric designates any thing that can be measured: the CPU usage of a server, the temperature of a room or the number of bytes sent by a network interface.

In the Gnocchi architecture, the storage back-end is responsible for storing measures of created metrics. It receives timestamps and values, and pre-computes aggregations according to the defined archive policies.

Testing a Rails API With RSpec

As I continue to improve me API testing skills in Rails, I have come to point where I really feel comfortable with all the tools I need to correctly add useful tests to my API. This post explains how I usually test my Rails APIs using RSpec.

Setting Up the Necessary Tools

In addition to RSpec, there are a few tools that make my testing experience much easier. These are:

• Factory Girl
• Faker
• Shoulda Matchers
• Database Cleaner

After Adding these gems to the Gemfile. I proceed to create a /support directory under the /spec directory. This directory will contain the additional configuration of the above tools, as well as some helper definitions that our specs will use.

/spec/support/shoulda.rb:

Shoulda::Matchers.configure do |config|
  config.integrate do |with|
    # Choose a test framework:
    with.test_framework :rspec

    # Or, choose the following (which implies all of the above):
    with.library :rails
  end
end