Original address: 12factor.net/

Introduction to the

Today, software is often delivered as a service, called web applications, or software as a service (SaaS). The 12-Factor provides a methodology for building SaaS applications that:

  • Use a standardized process to automate configuration so that new developers can join the project with minimal learning costs.
  • Keep the boundaries as clear as possible between operating systems and provide maximum portability across systems.
  • Suitable for deployment on modern cloud computing platforms to save resources in server and system administration.
  • Minimize differences between development and production environments and implement agile development using continuous delivery.
  • Extensions can be implemented without significant changes to the tools, architecture, and development process.

This theory applies to applications developed in any language and backend services (databases, message queues, caches, and so on).

background

Contributors to this article have been involved in the development and deployment of hundreds of applications, and have indirectly witnessed the development, operation, and scaling of hundreds of thousands of applications through the Heroku platform.

This article combines almost all of our experience and wisdom on SaaS applications and is an ideal practice standard for developing such applications, with a special focus on how applications can keep growing healthy, how developers can collaborate effectively on code, and how to avoid software contamination.

Our goal was to share and deepen our understanding of some of the systemic problems we have identified in modern software development. We provide a shared vocabulary for discussing these problems, and use related terms to provide a broad set of solutions to these problems. The format of this article was inspired by Martin Fowler’s book Patterns of Enterprise Application Architecture, Refactoring.

Who should the readers be?

Developer of any SaaS application. Operation and maintenance engineers who deploy and manage such applications.

I. Base code

One base code (Codebase), multiple deployments (Deploy)

12-factor applications are usually managed using version control systems such as Git, Mercurial, and Subversion. A database that keeps track of all revisions of code is called a code repository (code repo).

In a centralized version control system like SVN, the benchmark code is the code base in the control system; In a distributed version control system like Git, the base code is the most upstream code base.

There is always a one-to-one correspondence between the benchmark code and the application:

  • Once you have multiple benchmarks, it’s not an application, it’s a distributed system. Each component in a distributed system is an application, and each application can be developed separately using 12-factor.
  • It is against the 12-factor principle that multiple applications share a benchmark code. The solution is to split the shared code into separate class libraries and then load them using a dependency management strategy.

Although there is only one copy of benchmark code per application, multiple deployments can exist simultaneously. Each deployment is equivalent to running an instance of an application. There is usually a production environment and one or more pre-release environments. In addition, each developer runs an application instance in his local environment, which is equivalent to a deployment.

The benchmark code is the same for all deployments, but a different version of it can be used for each deployment. For example, the developer may have some commits that have not been synchronized to the pre-release environment; The pre-release environment also has some commits that are not synchronized to production. But they all share a benchmark code, and we assume they’re just different deployments of the same application.

II. Rely on

Explicitly declare dependencies (Dependency)

Most programming languages provide a packaging system that provides packaging services for individual libraries, like Perl’s CPAN or Ruby’s Rubygems. Libraries installed by packaging systems can be system-level (called “site packages”) or used only by an application, deployed in the appropriate directory (called “vendoring” or “bunding”).

Applications under the 12-factor rule do not implicitly rely on system-level libraries. It must declare all dependencies exactly through a dependency list. In addition, dependency isolation tools are used during runtime to ensure that the program does not invoke dependencies that exist in the system but are not declared in the manifest. This approach applies equally to both production and development environments.

For example, Ruby’s Bundler uses Gemfile as a list of dependency declarations and Bundle Exec for dependency isolation. Two separate tools are available in Python – Pip for dependency declarations and Virtualenv for dependency isolation. Even C has similar tools, with Autoconf for dependency declarations and static link libraries for dependency isolation. Regardless of the tool, dependency declarations and dependency isolation must be used together or the 12-factor specification cannot be met.

One of the advantages of explicitly declaring dependencies is that it simplifies the environment configuration process for new developers. A new developer can check out the application’s benchmark code, install the programming language and its dependency management tools, install all the dependencies with a single build command, and get to work. For example, Ruby/Bundler uses Bundle install, while Clojure/Leiningen uses Lein Deps.

12-Factor apps also don’t implicitly rely on system tools such as ImageMagick or curl. Even though these tools exist on almost all systems, there is no guarantee that all future systems will support or be compatible with applications. If the application must use some system tools, those tools should be included in the application.

III. The configuration

Store the configuration in the environment

Typically, the configuration of an application will vary widely between deployments (pre-release, production, development, and so on). These include:

  • Database, Memcached, and other back-end service configurations
  • Certificates for third-party services, such as Amazon S3, Twitter, etc
  • Configuration specific to each deployment, such as domain names

Some applications use constants in their code to hold configuration, which is a far cry from the strict separation of code and configuration required by 12-Factor. Configuration files vary widely from deployment to deployment, but the code is identical.

A simple way to determine if an application is correctly excluding configuration from its code is to see if the application’s benchmark code is immediately open source without fear of exposing any sensitive information.

It should be noted that the definition of “configuration” does not include the internal configuration of the application, such as Rails config/routes.rb, or the dependency injection relationships between code modules when using Spring. This type of configuration does not vary from deployment to deployment, so it should be written into the code.

Another solution is to use configuration files without incorporating them into the version control system, as Rails does with config/database.yml. This is a long way from using constants in your code, but there are still drawbacks: you can always accidentally check configuration files into the code base; Configuration files may be scattered in different directories and in different formats, making it impractical to find a single place to manage all configurations. To make matters worse, these formats are often language – or frame-specific.

12-factor recommends storing the application configuration in environment variables (env Vars, env). Environment variables can be easily changed between deployments without touching a line of code; Unlike configuration files, the chance of accidentally checking them into the code base is minimal; In contrast to some traditional mechanisms for solving configuration problems, such as Java’s property configuration files, environment variables are language – and system-independent.

Another aspect of configuration management is grouping. Sometimes applications group configurations according to specific deployments (or “environments”), such as the Development, Test, and Production environments in Rails. This approach does not scale easily: more deployments mean more new environments, such as staging or QA. As the project progresses, developers may also add their own environments, such as Joes-staging, which can lead to a proliferation of configuration combinations and add a lot of uncertainty to the management deployment.

In 12-factor applications, the granularity of environment variables should be small and relatively independent. They will never be combined into a so-called “environment”, but exist independently in each deployment. This configuration management approach smoothes the transition as applications scale and require more types of deployments.

IV. Back-end services

Backing services as an additional resource

Back-end services are the various services that are invoked over the network to run the program, such as databases (MySQL, CouchDB), message/queue systems (RabbitMQ, Beanstalkd), SMTP mail delivery services (Postfix), and Memcached systems.

Back-end services, like databases, are typically managed together by the system administrators who deploy the applications. In addition to local services, applications may use services published and managed by third parties. Examples include SMTP (such as Postmark benchmark), data collection services (such as New Relic or Loggly), data storage services (such as Amazon S3), And services accessed using apis (such as Twitter, Google Maps, last.fm).

The 12-Factor app does not discriminate between local or third-party services. Both are additional resources to the application, retrieving data through a URL or other service location/service certificate stored in the configuration. Any deployment of the 12-Factor application should be able to replace the local MySQL database with a third-party service (such as Amazon RDS) without any code changes. Similarly, local SMTP services should also be interchangeable with third-party SMTP services (for example, Postmark). In the preceding two examples, you only need to change the resource address in the configuration.

Each different back-end service is a resource. For example, if a MySQL database is one resource, two MySQL databases (used for data partitioning) are treated as two different resources. 12-Factor applications treat these databases as additional resources that remain loosely coupled to their attached deployments.

Deployment can load or unload resources on demand. For example, if the applied database service fails due to hardware problems, an administrator can restore a database from the most recent backup, uninstall the current database, and then load a new database — all without changing the code.

V. Build, publish, run

Strictly separate build and run

The transformation of benchmark code into a deployment (non-development environment) requires three phases:

  • The construction phaseThe process of converting a code repository into an executable package. The code for the specified version is used when building, retrieved, and packageddependency, compiled into binaries and resource files.
  • Release phaseThe results of the build are combined with what is required for the current deploymentconfigurationAnd can be immediately put into use in an operational environment.
  • Operation phase(or “run time”) is a pointer to the execution environment that starts a series of applications for a selected releaseprocess.

12-Factor apps strictly distinguish between build, publish, and run. For example, it is not advisable to modify running code directly because these changes are difficult to synchronize back to the build step.

Deployment tools often provide release management tools, most notably the ability to fall back to older releases. Capistrano, for example, stores all releases in a subdirectory called Releases, and maps the current online version to the corresponding directory. The tool’s rollback command makes it easy to rollback versions.

Each release must correspond to a unique release ID, such as a release time stamp (2011-04-06-20:32:17), or a growing number (v100). A release is like a ledger that can only be appended. Once released, it cannot be modified, and any changes should result in a new release.

Before new code can be deployed, developers need to trigger build operations. However, the run phase does not need to be triggered manually, but can occur automatically. For example, the server restarts, or the process manager restarts a crashed process. Therefore, the runtime should be as few modules as possible, so that if a system failure occurs in the middle of the night and the developer is stretched, it won’t cause much of a problem. The construction phase can be relatively complex because error messages are immediately exposed to developers and can be handled properly.

Process of VI.

Run the application as one or more stateless processes

In a runtime environment, applications typically run as one or more processes.

In the simplest scenario, the code is a stand-alone script, the environment is the developer’s own laptop, and the process is run from a command line (for example, python my_script.py). At the other extreme, complex applications may use many process types, i.e., zero or more process instances.

12-factor Application processes must be stateless and not shared. Any data that needs to be persisted is stored in a back-end service, such as a database.

An area of memory or disk space can be used as a buffer for a process doing some kind of transactional operation, such as downloading a large file, operating on it, and writing the results to a database. The 12-factor application does not have to worry about whether the cached content can be reserved for future requests. This is because the application starts multiple types of processes and future requests will most likely be serviced by other processes. Even in the case of a single process, previously saved data (in memory or on the file system) can be lost due to restarts (such as code deployment, configuration changes, or the runtime scheduling of the process to another physical area for execution).

The source file packing tool (Jammit, Django-compressor) uses the file system to cache the compiled source file. 12-factor applications are more likely to do this during the build step — like Rails resource pipelines — than during the run phase.

Some Internet systems rely on “sticky sessions,” which cache data from a user’s session into the memory of a process and route subsequent requests from the same user to the same process. Sticky sessions are something the 12-Factor is strongly against. The data in the Session should be stored in a cache with an expiration date such as Memcached or Redis.

VII. Port binding

Services are provided through Port binding

Internet applications sometimes run in a server container. PHP, for example, often runs as a module of Apache HTTPD, just as Java runs on Tomcat.

A 12-Factor application can create a web-oriented service by completely self-loading and not relying on any web server. Internet applications provide services through port binding and listen for requests sent to that port.

In the local environment, the developer accesses the service through an address like http://localhost:5000/. In an online environment, requests are uniformly sent to a public domain and then routed to a network process bound to a port.

The usual implementation approach is to load the network server library into the application through dependency declarations. Examples include Tornado for Python, Thin for Ruby, Java, and other JVM-based languages like Jetty. It is entirely up to the client, or rather the application code, to initiate the request. Ports bound to the runtime environment can handle these requests.

HTTP is not the only service that can be provided by port binding. Virtually all server software can wait for requests through process-bound ports. For example, ejabberd using XMPP, and Redis using the Redis protocol.

It should also be noted that the port binding approach also means that one application can become a back-end service for another application, with the caller storing the corresponding URL provided by the server as a resource in the configuration for future invocation.

VIII. The concurrent

Extend through the process model

Any computer program, once started, generates one or more processes. Internet applications run in a variety of processes. For example, PHP processes exist as child processes of Apache and are started on request. Java processes take the opposite approach. The JVM provides a super process that stores a large amount of system resources (CPU and memory) at the start of the program and manages internal concurrency through multiple threads. In both cases, the process is the smallest unit a developer can operate on.

** In 12-Factor applications, processes are first-class citizens. ** 12-factor uses processes that are largely borrowed from the Unix daemon model. Developers can use this model to design application architectures that allocate different work to different process types. For example, HTTP requests can be handed over to web processes, while resident background work is handed over to worker processes.

This does not include individual more specialized processes, such as processing concurrent internal operations through virtual machine threads, or using asynchronous/event-triggering models such as EventMachine, Twisted, and Node.js. But scaling on a single virtual machine has a bottleneck (vertical scaling), so applications must be able to work across multiple physical machines.

The above process model will come in handy when the system needs to be extended. The share-free, horizontally partitioned nature of processes in 12-factor applications means that adding concurrency is easy and safe. The types of these processes and the number of processes in each type are called process composition.

The 12-factor application processes do not require daemons or write PID files. Instead, use the operating system’s process manager (such as Systemd, a distributed process management cloud platform, or a Foreman like tool) to manage output streams, respond to crashed processes, and handle user-triggered requests to restart and shut down super processes.

IX. Easy to deal with

Fast start and graceful end maximizes robustness

Processes in 12-factor apps are disposable, meaning they can be started or stopped instantly. This facilitates fast, resilient scaling applications, rapid deployment of changing code or configuration, and robust deployment of applications.

Processes should aim for minimum startup time. Ideally, the time between a process typing a command and actually starting up and waiting for a request should be very short. Less startup time provides a more agile distribution and extension process, in addition to increased robustness, because the process manager can easily move the process to a new physical machine under licensing conditions.

A process terminates gracefully once it receives a SIGTERM. In the case of a network process, graceful termination means stopping the port on which the listening service is performed, that is, rejecting all new requests, continuing with the currently received requests, and then exiting. The implicit requirement of this type of process is that HTTP requests are mostly short (no more than a few seconds), whereas in long-running polling clients should try to reconnect as soon as they lose a connection.

For worker processes, graceful termination means sending the current task back to the queue. For example, the RabbitMQ, the worker can send a signal [NACK] (http://www.rabbitmq.com/amqp-0-9-1-quickref.html#basic.nack). In Beanstalkd, task termination and return to queue will be automatically triggered when the worker is disconnected. A system with a locking mechanism such as Delayed Job requires that system resources be released. The implicit requirement of this type of process is that tasks should be repeatable, mainly by wrapping results into transactions or idempotent repeated operations.

Processes should also be robust in the face of sudden death, such as underlying hardware failure. These are rarer than graceful terminations, but they can happen. One recommended approach is to use a robust back-end queue, such as Beanstalkd, which automatically falls back to the task when the client disconnects or times out. In any case, 12-Factor apps should be designed to handle unexpected, inelegant endings. Crash-only Design turns this concept into a logical theory.

X. The development environment is equivalent to the online environment

Keep development, pre-release, and online environments the same as much as possible

Historically, there have been many differences between the development environment (that is, local deployment by developers) and the online environment (real deployment accessed by external users). These differences are reflected in the following three aspects:

  • Time difference: Developers are writing code that may take days, weeks, or even months to come online.
  • People differences: Developers write code, operations deploys code.
  • Tool differences: Developers may use Nginx, SQLite, OR OS X, while online environments use Apache, MySQL, or Linux.

For 12-factor apps to continue to deploy, they must bridge the gap between local and online. Go back to the three differences described above:

  • Narrow the time gap: Developers can deploy code in hours, even minutes.
  • Close the human divide: Developers should not only write code, but also be intimately involved in the deployment process and how the code behaves online.
  • Narrow tool differences: Try to ensure consistency between the development environment and the online environment.

Turn the above summary into a table as follows:

The traditional application 12 – Factor application
Each deployment Interval A few weeks A few hours
Developers vs. operations Different people The same person
Development environment vs. online environment different As far as possible close to the

Back-end services, such as databases, queuing systems, and caches, are an important part of keeping development equivalent to online. Many languages provide libraries that simplify the acquisition of back-end services, such as adapters for different types of services. The table below provides some examples.

type language The class library The adapter
The database Ruby/Rails ActiveRecord MySQL, PostgreSQL, SQLite
The queue Python/Django Celery RabbitMQ, Beanstalkd, Redis
The cache Ruby/Rails ActiveSupport::Cache Memory, filesystem, Memcached

Developers sometimes find it attractive to use lightweight back-end services in a local environment, while heavier, robust back-end services should be used in a production environment. For example, use SQLite locally and PostgreSQL online; Or local caching in process memory and Memcached online.

Developers of 12-Factor applications should be opposed to using different back-end services across different environments, even though adapters can almost eliminate the usage differences. This is because different back-end services can mean sudden incompatibilities that can cause problems online with code that has been tested and pre-released normally. These errors can cause resistance to continued deployment. Eliminating this resistance is costly in terms of the application’s life cycle.

At the same time, lightweight local service isn’t as compelling as it once was. With Homebrew, apt-get and other modern packaging systems, back-end services such as Memcached, PostgreSQL, RabbitMQ, etc. are not complicated to install and run. In addition, declarative configuration tools like Chef and Puppet, combined with lightweight virtual environments like Vagrant, can make a developer’s local environment as close to the online environment as possible. These systems are clearly worth it when compared to the benefits of a synchronous environment and continuous deployment.

Adapters for different back-end services are still useful because they make it easy to migrate back-end services. But all deployments of the application, including development, pre-release, and online environments, should use the same version of the same back-end service.

XI. The log

Think of logs as a stream of events

Logging makes the actions of an application run transparent. In a server-based environment, logs are usually written to a file on the hard disk, but this is just an output format.

A log should be a summary of the flow of events, gathering the output streams of all running processes and back-end services in chronological order. The original format of a log is really an event line, although you may need to look at many lines when looking back at a problem. The log does not have a definite start and end, but continues to increase as the application runs.

The 12-factor application itself never considers storing its own output stream. You should not attempt to write or manage log files. Instead, every running process will directly stream the standard output (STDOUT) events. In the development environment, developers can see the application activity in the terminal in real time through these data streams.

In pre-release or online deployment, the output streams of each process are captured by the runtime environment, collated together with other output streams, and then sent together to one or more final handlers for viewing or long-term archiving. These archive paths are not visible or configurable to the application and are entirely managed by the application’s runtime environment. Open source tools like Logplex and Fluentd can do this.

These streams of events can be output to a file or observed in real time at a terminal. Most importantly, the output stream can be sent to a log indexing and analysis system like Splunk, or a general-purpose data storage system like Hadoop/Hive. These systems provide powerful and flexible capabilities for viewing your application’s history of activity, including:

  • Look for specific events in the past.
  • Graph a large-scale trend, such as requests per minute.
  • Trigger alerts in real time based on user-defined conditions, such as exceeding a certain threshold of errors per minute.

XII. Manage processes

Background administrative tasks run as one-off processes

Process formation refers to a group of processes used to handle the normal business of an application, such as web requests. In contrast, developers often want to perform one-off tasks that manage or maintain applications, such as:

  • Run data migration (in Djangomanage.py migrateIn the Railsrake db:migrate).
  • Run a console (also known asREPLShell) to execute some code or do some checks against an online database. Most languages provide a REPL facility through the interpreter (python 或 perl), or other commands (used by Rubyirb, used Railsrails console).
  • Run some one-time scripts that are submitted to the code repository.

One-off managed processes should use the same environment as normal resident processes. These administrative processes use the same code and configuration as any other process and run on a release. The admin code should be distributed with other application code to avoid synchronization problems.

All process types should use the same dependency isolation technique. For example, if Ruby’s Web process uses the command bundle exec thin start, the database migration should use bundle exec rake db: Migrate. Similarly, if a Python application uses Virtualenv, you will need to introduce bin/ Python when running Tornado Web server and any manage.py management processes.

12-Factor particularly favors languages that provide the REPL shell because it makes it easy to run one-off scripts. In a local deployment, the developer invokes the one-time management process directly from the command line with shell commands. In online deployment, developers can still run such processes using SSH or other mechanisms provided by the runtime environment.