API Design in Rails

We’re starting on a new version of the internal PrettyQuick API that powers our native clients and AJAX. The original version was based largely on what our consumer iOS app needed, but going forward we want to build a more general API to better support multiple clients for multiple constituencies.

Luckly, Heroku has thoughtfully prepared a list of API best practice recommendations. I’ve logged my thoughts and usage notes below as I work through each of the items on their list. While some of their recommendations felt like overkill for an API of our size, I was happy to find many of their ideas scale nicely for smaller APIs.

Some Stuff Basically Free!

We got “always return minified json” and “always use SSL” for free. Rails takes care of minification, and we already use config.force_ssl in our production environment.

We already have token authentication, so it’s relatively easy to implement the Authorization header:

class Api::V1::ApplicationController < ActionController::Base
  def authenticate!
    @user = User.find_by_authentication_token(request.authorization)
    unless @user
	    render json: {errors: {id: "A1", message: "Unauthorized"}}, status: 401
	    return
	end
  end
end

Rails also gives us “accept JSON in post bodies” for free! We’re going one step further and specifying that we ONLY accept JSON:

class Api::V1::ApplicationController < ActionController::Base
  before_filter :ensure_json
  def ensure_json
    return if request.content_type == "application/json"
    render json: { errors: {
    	id: "CT1", 
        message: "The only acceptable content-type is application/json"
      }}, status: 406
  end
end

A Bunch Of Low-Hanging Fruit

Versioning In Accept Header

Not only does Heroku recommend this, but Github also uses extensive header-based versioning and media types. I’m definitly willing to trust them on this point - but how to do it myself?

As always, Railscasts has my back! I copy-and-pasted from Episode 350 to set up header-based versioning. The header looks like Content-Type: application/vnd.prettyquick.v1. The rails code is beautifully minimal:

# config/routes.rb
scope module: :api do
  scope module: :v1, constraints: ApiConstraints.new(version: 1) do
    resources :products, format: :json
  end

  # ...all our old routes...
end

# lib/api_constraints.rb
class ApiConstraints
  def initialize(options)
    @version = options[:version]
  end
  def matches?(req)
    req.headers['Accept'].include?("application/vnd.prettyquick.v#{@version}")
  end
end

This lets us separate versions into folders:

app/
  controllers/
    api/
      v1/
        products_controller.rb   # defines Api::V1::ProductsController
  views/
  	api/
  	  v1/
  	    products/
  	      index.rabl

Response Format

The JSON API spec recommends that every response contain exactly one of the two keys “data” and “errors”. We’re combining that with Heroku’s recommendation to use structured errors. Our error responses look like this:

HTTP 406
{
  errors: {
    id: "A1",
    message: "Unauthorized"
  }
}

Heroku’s own error IDs have a simple prefix and numeric code. I don’t think the error ID matters too much as long as it stays constant and documented.

Nest Foreign Key Relations

We’re using Rabl for our API responses. Heroku argues that foreign keys should be nested ({city: {id: 12}} vs {city_id: 12}), so when you want to include more information about the related object your changes are purely additive. Rabl makes this relatively easy:

object @user
attributes :id, :email
node(:city) {|u| {id: u.city_id}}

I’ve used this node approach instead of a more Rabl-esque phrasing:

child(:city) do
  attributes :id
end

because the child approach hits the database to get the full city object, but I already have the city_id on the user. If/when I need more attributes from the city I’ll switch to the child approach.

Standardize All Timestamps In UTC and ISO8601

Almost free! I added a filter to my application controller to force UTC throughout my API:

class Api::V1::ApplicationController < ActionController::Base
  around_filter :use_utc
  def use_utc
    Time.use_zone("UTC") { yield }
  end
end

Time zones are always messy and complicated, but I cribbed this directly from the rails docs so I’m confident it’s a stable solution.

All this, and more!

Many of the rest of the Heroku recommendations are more of a style guide to follow, than things to implement once and get out of the way. We’ll be keeping these in mind:

• shallow nesting
• downcase paths and attributes
• use appropriate HTTP status codes
• use plurals for resource names

Things to Revisit

That’s a lot of low-hanging fruit! I think I’m in a good spot. However, there are a few more things Heroku recommends that I want to get to soon. These are things that will be important as we start to use this API more heavily.

Split Long Responses Into Ranges

AKA “paging”. Returning every single record from a 100k-object result set is going to take too long. Rather, Heroku suggests returning smaller result sets with an easy way to move from one page to the next. Personally I’ve mostly used paged result sets in the Stripe API, so I’ll look to them for inspiration alongside the Heroku suggestions when we prioritize this project.

ETags

ETags are used to give caching hints to everyone in between the client and the server. They’re included in the HTTP header, and I think are based on a hash of the response body. That’s all I know offhand! I used them a while ago so I know rails has relatively easy support. I definitely want a fast API so I’ll be refreshing my knowledge soon.

Human-Readable Docs

Nothing can put you down a rabbit-hole like tracing through code to figure out how something works. It’s certainly an important skill, but provides so many opportunities to get distracted. Having used some APIs with really nice documentation, I appreciate how much they speed and ease my development process. I suspect we’ll have to make some trade-offs on this front since we’re building internal APIs and not that many people will be using them. Heroku recommends some tools for auto-generating the docs from the code, hopefully we can find a good balance.

More Security

This one might get some extra priority ASAP, I’m checking with experts to see if my auth-token security setup is sufficient. The next step would be adding some verification to the full request, eg providing nonces for the client to create a one-time hash of the request, then checksumming on the server to verify no one has messed with it on the way up.

Long Term But Very Cool Stuff

Rate Limits/Show Rate Limit Status

Nothing sucks like accidentally DDoS-ing yourself. Even though we think we understand the traffic patterns for these APIs, we’re securing them properly, and don’t expect to be the target of hackers - even an accidental inefficiency on a client could send a huge spike of traffic we weren’t anticipating. Rate limiting is important for any API, not just APIs where you don’t personally know all the developers.

This is a problem we’re pushing off for now so we can start building more quickly, but as we include these APIs in our native mobile clients and as our install base grows, rate limiting will become more important. A lot of smart people have solved this problem already so I’m hoping to find an easy, production-tested solution recommended by one of the big players.

UUIDs

Heroku recommends using object GUIDs that are unique across your entire system. We’re currently using traditional rails IDs: auto-incrementing integers unique per database table. The best argument I’ve heard for GUIDs is that it allows you to offload ID generation from your database instead of relying on a single auto-incrementer. That also makes it much easier to spliit your data into multiple databases.

We aren’t yet at the scale where this is required, but it seems relatively easy to implement so I’d like to get to it sooner rather than later.

Full JSON API Spec

The JSON API spec has a ton of great ideas in it, and they’ve solved a lot of problems. But, it feels like overkill for us right now. We’ll definitely keep it in mind and include as much as feels useful, but our v1 API won’t include a full implementation.

Machine-Readable Schema

Again, feels like overkill. For now we’re just trying to get a better, more consistent API out the door. We’re happy to manually code against it, we don’t need any automated discovery or anything. Long-term I think a machine-readable schema could help with validations and tests though.

Problem Solved! But So Many More Questions…

Working through this Heroku guide has made it really easy to get up and running with API best practices. I was able to take what I need and leave the rest, I didn’t feel like I had to fully buy in to “the Heroku way” to make use of their ideas.

However, these suggestions are mostly about the transfer apparatus and the technical underpinnings of the API. I still have a lot of design questions:

• how to structure client/server separation, ie what work happens on the client and what happens through API actions?
• how much data to include in any given request? a user client and an admin client have very different views
• how to structure permissioning? users and admins have very different sets of abilities
• how does development flow through staging/production environments?
• should we be running the API in a separate heroku app?

More on that later, as I figure it out! For now, let me know if you have any other recommendations for API best practices.