When it comes to the API, the changes are not welcome. For software developers, they have long been accustomed to rapid and frequent functional iterations; API developers, on the other hand, even if only one user calls the API, then it is very difficult to change the API, because it affects everything. Many of us are familiar with the evolution of the Unix operating system. In 1994, the UNIX-Haters manual was released, and it contained a lot of mail about the software — everything from overly cryptic command names optimized for Teletype machines, to irreversible file deletion, to the option-laden, unintuitive program itself. More than 20 years later, and even among many modern derivative systems, most of these quips still apply. This is because Unix has become so widely used that changing its behavior can have a huge impact. But in any case, the contracts it makes with its customers define how Unix interfaces behave.
Similarly, an API represents a communication contract that cannot be modified without concerted effort and a lot of work. Because so many businesses use Stripe as their infrastructure, we’ve been thinking about these contracts since Stripe was founded. So far, we have maintained compatibility with every version of every API since the company was founded in 2011. In this article, we’ll share how we manage API versions at Stripe.
Some fixed expectations are built into the process of writing a code integration API. If an endpoint returns a Boolean field called Verified to indicate the status of a bank account, the user might write the following code:
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__if bank_account[:verified]
...
else
...
end
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__Copy the codeIf we later replace the Bank account’s Boolean verified field with a status field containing the value of Verified (as we did in 2014), the above code would be broken because it relies on a field that at this time no longer exists. This type of change is not backward compatible and should be avoided. The old fields should always be retained, and the types and names should remain the same. However, not all changes are backward incompatible; For example, it is safe to add a new API endpoint, or to add a new field to an existing API endpoint that has not been used before.
On a collaborative basis, we might be able to let our users know about the changes we’re making and let them update their integration code, but even if we could, it wouldn’t be in a very friendly way. Like a grid connection or water supply, the API should be kept running as uninterrupted as possible after the connection is made.
Stripe’s mission is to provide economic infrastructure for the Internet. Just as a power company should not change the voltage every two years, we believe we should give our customers confidence that the Web API we provide will be as stable as possible.
API version control scheme
A common approach to Web API evolution is the use of version control. Users specify the version when they request it, and the API provider can modify the next version as needed while remaining compatible with the current version. When a new version is released, users can upgrade at their convenience.
This is often seen as a major versioning scheme, passing names like v1, v2, and v3 as URL prefixes (e.g. /v1/widgets) or through HTTP headers (e.g. Accept). This is an effective approach, but the main drawback is that the changes from version to version are so large, and the impact on users, that they are almost as painful as reintegration. This approach also has no obvious advantage, since users who are unwilling or unable to upgrade are stuck with the older version. At this point, the provider must make the difficult choice of decommissioning the API version, abandoning those users, or maintaining the old version endlessly at considerable cost. While having vendors maintain older versions may at first appear to be good for users, they also indirectly pay the price of slower updates because engineering time is spent maintaining older code rather than developing new features.
In Stripe, version control is implemented by scrolling versions, with versions named using API release dates (such as 2017-05-24). Although backward incompatible, each version contains a small number of changes, making incremental upgrades relatively easy so that integration can keep pace with version updates.
The first time a user makes an API request, their account is automatically pinned to the latest available version, and every SUBSEQUENT API call they make is implicitly assigned to that version. This approach ensures that users do not suddenly receive disruptive changes and makes the initial integration less painful by reducing the necessary configuration. Users can manually set the stripe-version header, or update the pinched Version of their account from the Stripe control board, overwriting the Version of the call for any single request.
As you may have noticed, the Stripe API also uses prefix paths to define major versions (such as /v1/charges). While we do use this approach at some point, the current approach will not change for some time. As mentioned above, major release changes often make upgrades painful, and it’s hard to imagine an API redesign being significant enough to affect users to this extent. Our current approach has enabled us to complete nearly 100 backward incompatible upgrades over the past six years.
Underlying version control
Versioning is always about balancing the cost of improving the development experience with the cost of maintaining older versions. We tried to achieve the former while minimizing the latter, and implemented a version control system to help us achieve this goal. Let’s take a quick look at how it works. Every possible response from the Stripe API is written into a class, which we call an API resource. API resources use DSLS to define available fields:
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__class ChargeAPIResource
required :id, String
required :amount, Integer
end
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__Copy the codeThe API resources are recorded, and the structure they describe is what we want the current version of the API to return. When we need to make a backward incompatible change, we encapsulate it in a version change module that defines the change-related annotations, a transformation, and a set of API resource types that need to be modified if conditions are met:
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__class CollapseEventRequest < AbstractVersionChange description \ "Event objects (and webhooks) will now render " \ "`request` subobject that contains a request ID " \ "and idempotency key instead of just a string " \ "request ID." response EventAPIResource do change :request, type_old: String, type_new: Hash run do |data| data.merge(:request => data[:request][:id]) end end end __Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__Copy the codeAssign a corresponding API version to a version change in the main list:
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__class VersionChanges
VERSIONS = {
'2017-05-25' => [
Change::AccountTypes,
Change::CollapseEventRequest,
Change::EventAccountToUserID
],
'2017-04-06' => [Change::LegacyTransfers],
'2017-02-14' => [
Change::AutoexpandChargeDispute,
Change::AutoexpandChargeRule
],
'2017-01-27' => [Change::SourcedTransfersOnBts],
...
}
end
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__Copy the codeVersion changes are recorded so they can be expected to be automatically applied backwards in order from the current API version. But each version change assumes that “even though new changes may follow, they should receive the same data as when the API was originally written.”
When generating a response, the API first formats the data by describing the current version of the API resource, and then determines the version of the target API based on one of three things:
- If provided, according to the stripe-version header;
- If the request is sent in the name of the user, based on the version of the OAuth authorized application;
- This is set when a user first sends a request to Stripe, depending on which version the user pinches.
We will then go back in time and request every version change module found in this process until the target version is found:
The request is processed by the version change module before the response is returned
The version change module removes older versions from the core code path. Developers can mostly ignore them when building new products.
Changes with side effects
Most backward incompatible API changes modify the response, but this is not always the case. Sometimes, you may need to make complex changes that go beyond the module in which it is defined. We add has_side_effects annotations to these modules, which define transformations as empty operations:
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__class LegacyTransfers < AbstractVersionChange
description "..."
has_side_effects
end
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__Copy the codeThey need to be checked elsewhere in the code to see if they are still valid:
__Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__VersionChanges.active? (LegacyTransfers) __Mon Aug 28 2017 17:33:53 GMT+0800 (CST)____Mon Aug 28 2017 17:33:53 GMT+0800 (CST)__Copy the codeThis weak encapsulation makes changes with side effects harder to maintain, so we try to avoid them.
Declarative change
One of the benefits of the self-contained version change module is that it can define comments that describe the fields and resources they affect. We can use this annotation again to quickly provide more useful information to the user. For example, our API change log is procedurally generated and receives updates as soon as a new version of the service is deployed.
We also tailored the API reference documentation to specific users. It knows who is logged in and annotates the field based on the API version of the account. Here, we warn developers that the API they are using has backward-incompatible changes since the nailed version. The request field of the Event, which was previously a string, is now a child object that also contains idempotent keys (created in the above version change) :
Our documentation detects the user’s API version and issues a warning
Minimize change
Providing extensive backward compatibility is not free; Each new release means more code to understand and maintain. We do our best to make the code we write clean, but a project full of version changes that can’t be wrapped clearly and require sufficient time and extensive review can slow down the project, reduce readability, and make the API more vulnerable. We used some metrics to try to avoid incurring this kind of expensive technical debt.
Even though we had a version control system available, we avoided it as much as possible and tried to get the API right in the initial design. Output changes are collected through a lightweight API review process, written to a simple support document and submitted to the mailing list. This allows each proposed change to be seen by more people in the company, allowing us to spot errors and inconsistencies before they are released.
We are always careful about discontinuation and use. It is important to maintain compatibility, but even so, we will eventually want to start decommissioning older API versions. Helping users migrate to the new version of the API allows them to take advantage of the new features and simplifies the basis on which we build them.
Principle of change
The combination of rolling releases and an internal framework to support this mechanism allowed us to attract a large number of users, and we made a number of API changes while minimizing the impact on existing integrations. This approach relies on some of the rules we’ve developed over the years. The API updates we think are important are:
- Lightweight. Keep the cost of upgrades as low as possible (both for users and for ourselves).
- First class. Make version control a first-class concept of the API, so you can use it to keep documents and tools accurate and up to date, and automatically generate change logs.
- “Fixed-cost”. Minimize maintenance costs by sealing older versions into version change modules. In other words, the less old behavior you need to consider when writing new code, the better.
We’re excited about the debate around REST, GraphQL, gRPC, and the evolution of these technologies, and more broadly, what the future of Web apis is going to look like, and we expect to continue supporting version control solutions for a long time to come.
APIs as Infrastructure: Future -proofing Stripe with versioning
Thanks to Yuta Tian guang for correcting this article.
To contribute or translate InfoQ Chinese, please email [email protected]. You are also welcome to follow us on Sina Weibo (@InfoQ, @Ding Xiaoyun) and wechat (wechat id: InfoQChina).
