We’ve talked before about how to build a JSON API with Rails 5. We also discussed using Rails 5 in — API mode, serializing our JSON responses, caching, and also rate limiting/throttling.
These are all important topics, but even more important is producing a clear, Standards-compliant API. We’re going to look at how to build an API that conforms to the JSON: API spec.
The JSON: API spec isn’t just limited to how The server should format JSON responses. It also speaks to how The client should format requests, how to handle sorting, pagination, errors, and how new resources should be created. It even speaks to which media types and HTTP codes should be used.
We’re going to be implementing many of these things in our Rails app, Lastly, we’ll also look at how we might test them. We’ll take a quick look at one possible authentication strategy and some tools we can use to produce beautiful documentation for our API.
json:api Is Big
The json:api spec is pretty big! It covers many of the possible features you may want to implement in a JSON API. May is the key word… You don’t have to implement all of them for your API to be JSON: API compliant. You don’t have to have to implement sorting, but if You do, the JSON: API specs will tell You how it must be done.
Media types
One of the first things you’ll notice with JSON: API is that it doesn’t actually use the application/ JSON media type. The reason for this is that the group behind json:api actually went through the proper process and channels to register their own media type, which is application/vnd.api+json. Part of the contract of the server is that it will respond with the Content-Type header set properly to this media type.
The easiest way I was able to find to get this working without having to set it manually in every action is to create an initializer to set them. I called it register_json_mime_types.rb, and it contains:
api_mime_types = %W(
application/vnd.api+json
text/x-json
application/json
)
Mime::Type.register 'application/vnd.api+json', :json, api_mime_typesCopy the codeNow whenever a request is made it will automatically have Content-Type:application/vnd.api+json; charset=utf-8 set in the response headers.
Creating a Resource
One of the questions that came up from my last article was how we should go about creating a resource. What format should the data come in? What URL should the data be sent to? How should the server respond on success? On errors? Thankfully all of those answers are available, and my goal is to show how to create a resource while following the json:api spec.
For these examples, we’ll be working with two models:
Their fields don’t matter too much, and they have the relationship of a RentalUnit belonging to a User and a User having many RentalUnits.
Routing
Routing is probably one of the easiest parts. The specification follows RESTful routing pretty much identically to what Rails gives you by default when you define the route with resources :rental_units.
The specific route we are interested in for creating a resource is POST /rental_units HTTP/1.1.
Format of data posted
If you’ve done very much Rails development before, you’re probably used to data being posted to the server in a fairly simple format, which may look somewhat like this:
{
rental_unit: {
price_cents: 100000,
rooms: 2,
bathrooms: 1
}
}Copy the codeIn the guide We see that it must be formatted like this:
data: {
type: 'rental_units',
attributes: {
price_cents: 100000,
rooms: 2,
bathrooms: 1
}
}Copy the codeThis matches how json:api is formatted on response, so it is at least familiar/consistent!
To ensure that our API responds well, let’s write a test. The test will post The data To The correct URL, and will then verify that the server responded with a 201 HTTP status code (which means resource created). After that, We’ll look for a Location header, which tells us where we can find this new resource that was created.
Lastly, we’ll look to verify that it responded in the correct json:api response format using a custom matcher which I’ll include below.
require 'rails_helper'
RSpec.describe "Rental Units", :type => :request do
let(:user) { create(:user) }
describe "POST create" do
it "creates a rental unit" do
post "/rental_units", {
params: {
data: {
type: 'rental_units',
attributes: {
price_cents: 100000,
rooms: 2,
bathrooms: 1
}
}
},
headers: { 'X-Api-Key' => user.api_key }
}
expect(response.status).to eq(201)
expect(response.headers['Location']).to match(/\/rental_units\/\d$/)
expect(response.body).to be_jsonapi_response_for('rental_units')
end
end
endCopy the codeHere is the custom rspec matcher I used. It does a simple check to make sure that the response conforms to the json:api spec when responding with a resource.
RSpec::Matchers.define :be_jsonapi_response_for do |model|
match do |actual|
parsed_actual = JSON.parse(actual)
parsed_actual.dig('data', 'type') == model &&
parsed_actual.dig('data', 'attributes').is_a?(Hash) &&
parsed_actual.dig('data', 'relationships').is_a?(Hash)
end
endCopy the codeSo if this is what our test looks like, what might the controller look like? It ends up looking fairly similar to how it might have before. The only difference here is that I have to dig a little deeper to get to the attributes coming in through the params object.
class RentalUnitsController < ApplicationController
def create
attributes = rental_unit_attributes.merge({user_id: auth_user.id})
@rental_unit = RentalUnit.new(attributes)
if @rental_unit.save
render json: @rental_unit, status: :created, location: @rental_unit
else
respond_with_errors(@rental_unit)
end
end
private
def rental_unit_params
params.require(:data).permit(:type, {
attributes: [:address, :rooms, :bathrooms, :price_cents]
})
end
def rental_unit_attributes
rental_unit_params[:attributes] || {}
end
endCopy the codeWhen Errors Occur
You might have noticed above that I have a special method called respond_with_errors for when the @rental_unit object is Unable to save. But before we get to that, let’s take a look at how JSON: API expects us to format the errors:
{
errors: [
status: 422,
source: {pointer: "/data/attributes/rooms"},
detail: "Must be present."
]
}Copy the codeTo help with this, I’ve created a small method that lives in the ApplicationController to help respond with errors:
def respond_with_errors(object)
render json: {errors: ErrorSerializer.serialize(object)}, status: :unprocessable_entity
endCopy the codeThe ErrorSerializer class it is referring to is just a simple module which helps format the errors in the correct way.
module ErrorSerializer
def self.serialize(object)
object.errors.messages.map do |field, errors|
errors.map do |error_message|
{
status: 422,
source: {pointer: "/data/attributes/#{field}"},
detail: error_message
}
end
end.flatten
end
endCopy the codeI’ve written a small test to make sure that the API responds well. It again uses a custom Rspec matcher which I’ll include below. What I am looking for here is that it responds with the 422 HTTP code (unprocessable entity) and that it contains an error for a specific field.
it "responds with errors" do
post "/rental_units", {
params: {
data: {
type: 'rental_units',
attributes: {}
}
},
headers: { 'X-Api-Key' => user.api_key }
}
expect(response.status).to eq(422)
expect(response.body).to have_jsonapi_errors_for('/data/attributes/rooms')
endCopy the codeHere is the custom matcher which looks to see if there is an error for a specific field, in this case the rooms field:
RSpec::Matchers.define :have_jsonapi_errors_for do |pointer| match do |actual| parsed_actual = JSON.parse(actual) errors = parsed_actual['errors'] return false if errors.empty? errors.any? do |error| error.dig('source', 'pointer') == pointer end end endCopy the codeSorting Results and Pagination
The next thing we are going to look at is how to sort results within our API. I’ve grouped this together with pagination because the way I coded it they go hand in hand within the same class.
You may be thinking at this point why I mentioned class. Doesn’t this normally happen within the index action of the RentalUnitsController? Normally yes, but while doing this, I found that it was a bit more code than I was comfortable with to leave it all inside the controller. It’s also a good example of how you might extract some complicated logic out of the controller into its own class or module.
The action itself is dead simple:
def index
rental_units_index = RentalUnitsIndex.new(self)
render json: rental_units_index.rental_units, links: rental_units_index.links
endCopy the codeThe RentalUnitsIndex class has one job, To handle the queries and data necessary to respond to the GET /rental_units HTTP/1.1 Request. It receives self (the controller) so that it can access things such as the params object as well as the URL helpers.
class RentalUnitsIndex DEFAULT_SORTING = {created_at: :desc} SORTABLE_FIELDS = [:rooms, :price_cents, :created_at] PER_PAGE = 10 delegate :params, to: :controller delegate :rental_units_url, to: :controller attr_reader :controller def initialize(controller) @controller = controller end def rental_units @rental_units ||= RentalUnit.includes(:user). order(sort_params). paginate(page: current_page, per_page: PER_PAGE) end def links { self: rental_units_url(rebuild_params), first: rental_units_url(rebuild_params.merge(first_page)), prev: rental_units_url(rebuild_params.merge(prev_page)), next: rental_units_url(rebuild_params.merge(next_page)), last: rental_units_url(rebuild_params.merge(last_page)) } end private def current_page (params.to_unsafe_h.dig('page', 'number') || 1).to_i end def first_page {page: {number: 1}} end def next_page {page: {number: [total_pages, current_page + 1].min}} end def prev_page {page: {number: [1, current_page - 1].max}} end def last_page {page: {number: total_pages}} end def total_pages @total_pages ||= rental_units.total_pages end def sort_params SortParams.sorted_fields(params[:sort], SORTABLE_FIELDS, DEFAULT_SORTING) end def rebuild_params @rebuild_params ||= begin rejected = ['action', 'controller'] params.to_unsafe_h.reject { |key, value| rejected.include? (key.to_s) } end end endCopy the codeThe nice thing about the way this is written is that it would be quite easy to test what might end up being complicated logic to perform sorting, pagination, and maybe at some point in the future, filtering of the rental units.
If this were an API I was developing for real, I would most likely extract a lot of these generic methods into a parent class.
Sorting with json:api
Sorting in json:api is done through a single query param called sort which comes through the URL. It might look like this ? sort=-rooms,price_cents, which would sort descending by the rooms field, and then ascending by the price_cents field.
This functionality is handled by the sort_params method, which farms out the work to a module called SortParams. This module has the job of taking a string such as -rooms,price_cents and converting it into the usual Hash that the order method wants to receive. From -rooms,price_cents to {rooms: :desc, price_cents: :asc}.
module SortParams def self.sorted_fields(sort, allowed, default) allowed = allowed.map(&:to_s) fields = sort.to_s.split(',') ordered_fields = convert_to_ordered_hash(fields) filtered_fields = ordered_fields.select { |key, value| allowed.include? (key) } filtered_fields.present? ? filtered_fields : default end def self.convert_to_ordered_hash(fields) fields.each_with_object({}) do |field, hash| if field.start_with? ('-') field = field[1..-1] hash[field] = :desc else hash[field] = :asc end end end endCopy the codeWe’ve been trying to test functionality as we build it, so here are the tests to ensure that it is sorting correctly when the sort param is included.
We’ve created some rental units and are requesting that they return in descending order based on the amount of rooms they have.
describe "GET index" do it "returns sorted results" do create(:rental_unit, rooms: 4) create(:rental_unit, rooms: 5) create(:rental_unit, rooms: 3) get "/rental_units", { params: { sort: '-rooms' }, headers: { 'X-Api-Key' => user.api_key } } expect(response.status).to eq(200) parsed_body = JSON.parse(response.body) rental_unit_ids = parsed_body['data'].map{ |unit| unit['attributes']['rooms'].to_i } expect(rental_unit_ids).to Eq ([5, 3]) end to endCopy the codePagination with json:api
With pagination, most of the details are left up to the programmer to decide. That is because pagination could be done quite differently from app to app, either page by page or by where your cursor is on the screen (say in an infinite scroll view). What is dictated by the spec is that you should pass this information through query params in the page key.
It may end up looking like this: ? page[number]=1. You are also required to include a links key in the response which provides links to the current page (self), the first, previous, next, and last links. Most of these details are handled by the RentalUnitsIndex class in combination with the will_paginate gem.
These are included in the rendered response from within the controller which calls out to our helper class:
render json: rental_units_index.rental_units, links: rental_units_index.linksCopy the codeHere we will test to ensure that it is returning paginated results along with the links correctly.
it "returns paginated results" do
(RentalUnitsIndex::PER_PAGE + 1).times { create(:rental_unit) }
get "/rental_units", {
params: { sort: '-rooms', page: {number: 2} },
headers: { 'X-Api-Key' => user.api_key }
}
expect(response.status).to eq(200)
parsed_body = JSON.parse(response.body)
expect(parsed_body['data'].size).to eq(1)
expect(URI.unescape(parsed_body['links']['first'])).to eq('http://www.example.com/rental_units?page[number]=1&sort=-rooms')
expect(URI.unescape(parsed_body['links']['prev'])).to eq('http://www.example.com/rental_units?page[number]=1&sort=-rooms')
expect(URI.unescape(parsed_body['links']['next'])).to eq('http://www.example.com/rental_units?page[number]=2&sort=-rooms')
expect(URI.unescape(parsed_body['links']['last'])).to eq('http://www.example.com/rental_units?page[number]=2&sort=-rooms')
endCopy the codeOne interesting but slightly unrelated thing I discovered while working on this article was that params has changed in Rails 5. It is now a different object instead of the old HashWithIndifferentAccess object. Thanks to Eileen Uchitelle for a great article pointing out what has changed.
Authentication
The type of authentication you need depends on the type of application you are building. If it is a public API, you might not need it at all, but more and more I have seen a simple API key given out, allowing the server to track your usage and giving them more control over its access.
Alternatively, it may be an API which powers a single page application or a mobile app. Here you will still go with a token solution, perhaps like the one that devise_token_auth gives you, but it is a little bit more complicated to implement because there is more security involved (the token may change on every single request).
The main thing to take away is that there are no cookies involved… there is no magical state that the browser handles for us. In fact, Because we’ve built this app using the — API version of Rails 5, the cookies functionality doesn’t even exist.
We’re going to go with the first solution I mentioned, a simple API key that is attached to the user’s account. When they request a page, they’ll include that in the request headers under the X-Api-Key value.
Here is what our User model looks like. It will assign each new user a key (which could later be regenerated and/or modified if we need to shut this account out because of abuse).
class User < ActiveRecord::Base
has_many :rental_units
before_create :set_initial_api_key
private
def set_initial_api_key
self.api_key ||= generate_api_key
end
def generate_api_key
SecureRandom.uuid
end
endCopy the codeInside of the ApplicationController, we’ll include a few small methods which will help us validate that the token is in fact correct, who it belongs to, and to respond with a 401 unauthorized response if it is invalid.
class ApplicationController < ActionController::API
before_action :authenticate
private
def authenticate
authenticate_api_key || render_unauthorized
end
def authenticate_api_key
api_key = request.headers['X-Api-Key']
@auth_user = User.find_by(api_key: api_key)
end
def auth_user
@auth_user
end
def render_unauthorized
render json: 'Bad credentials', status: 401
end
endCopy the codeHere we have a test that ensures it is working correctly:
it "responds with unauthorized" do
post "/rental_units", {
params: {},
headers: { 'X-Api-Key' => '12345' }
}
expect(response.status).to eq(401)
endCopy the codeAs far As I can tell, JSON: API doesn’t speak much or at all about authentication, and it is left up to the implementor to decide what works best for their specific API and its use case.
Documenting Your API
Part of what makes APIs great — such as the ones from Stripe, Twilio, Or GitHub — is how good their documentation is. I won’t go into too much detail here other than to recommend two of the The best approaches I ‘ve seen.
The first approach, a gem called apipie-rails, is done by adding extra details to your tests, which automatically generates documentation. This has the benefit of being easy to keep in sync with your code as it is right there alongside it, but I find that it can tend to clutter up the tests a little bit.
The second approach is one called slate which allows you to create some really beautiful API documentation, much like the ones I listed above. This is done in markdown, and it ends up generating static HTML/CSS files that you can host.
Conclusion
In this article, we expanded upon a previous article about creating JSON APIs within Rails. We tackled some common problems that occur when creating APIs, such as wanting pagination, sorting, errors, and authentication. We looked at how we can develop these features in keeping with the json:api spec.
By doing so, we avoid needing to have conversations about how to implement each feature, and we can instead focus on the specifics of our application.