In the course of project iterations, it is inevitable to “go live”. Going online corresponds to deployment, or redeployment; Deployment corresponds to modification; Modification means risk.
There are many deployment technologies, some simple, some complex; Some require downtime, while others can be deployed without downtime. The purpose of this paper is to make a summary of the current commonly used deployment scheme.
First, blue and green layout
Blue/Green Deployment
1, define,
A blue-green deployment is to run the old version, deploy the new version, test it, verify that it’s OK, cut traffic to the new version, and upgrade the old version to the new version at the same time.
1, the characteristics of
Blue-green deployments require no downtime and are less risky.
2. Deployment process
Step 1, Deploy the VERSION 1 application (initial state)
All traffic from external requests is directed to this version.
Step 2: Deploy the VERSION 2 application
Version 2 code is different from version 1 (new features, Bug fixes, etc.).
Step 3. Switch traffic from version 1 to version 2.
Step 4. If the VERSION 2 test works, delete the resources (such as instances) that are being used in version 1 and use version 2 officially.
3, summary
As you can see from the process, our application is always online during the deployment process. In addition, during the launch of the new version, no content of the old version is changed, and the status of the old version is not affected during the deployment. There is little risk and, in theory, we can roll back to the old version at any time, as long as the old version of the resource is not deleted.
4, blue and green issued matters needing attention
When you switch to blue, you need to properly handle unfinished business and new business. If your database backend can’t handle it, it can be a problem;
There may be cases where both “microservice architecture applications” and “legacy architecture applications” need to be handled, and if the two are not properly coordinated in a blue-green deployment, there is still a chance that the service will stop. Synchronous migration/rollback of database and application deployment needs to be considered in advance. Blue-green deployments require infrastructure support. With blue-green deployments on non-isolated infrastructures (VM, Docker, etc.), the blue and green environments risk being destroyed.
2. Rolling Update
1. Rolling publish definitions
Rolling publishing: Typically taking one or more servers out of service, performing updates, and putting them back into use. This cycle continues until all instances in the cluster are updated to the new version.
2, the characteristics of
This deployment is more resource-efficient than a blue-green deployment — it does not require running two clusters with twice the number of instances. We can deploy in parts, such as taking out 20% of the cluster at a time to upgrade.
This approach also has many disadvantages, such as:
(1) There is no sure OK environment. With a blue-green deployment, we can clearly tell that the old version is OK, while with a rolling release, we can’t be sure.
(2) Modify the existing environment.
(3) If rollback is required, it is difficult. For example, in one release, we needed to update 100 instances, 10 at a time, and each deployment took 5 minutes. When you scroll to the 80th instance, a problem is found and you need to roll back, which is a painful and lengthy process.
(4) Sometimes we can scale the system dynamically. If the system automatically expands/shrinks during deployment, we need to determine which node is using which code. Despite some automated operations tools, it’s still scary.
(5) Since it is gradually updated, there will be a temporary inconsistency between the old version and the new version when we put the code online. If there is a high requirement for putting the code online, we need to consider how to do a good job of compatibility.
Gray publishing/Canary deployment
1, define,
Grayscale publishing is a smooth transition between black and white. AB test is A grayscale release method, in which some users continue to use A and some users start to use B. If users have no objection to B, the scope will be gradually expanded and all users will be migrated to B. Grayscale release can ensure the stability of the overall system. Problems can be found and adjusted at the initial grayscale to ensure its impact. The canary deployment we commonly call is also a way of grayscale release.
In the 17th century, English miners discovered that canaries were very sensitive to gas. Canaries will stop singing at the slightest trace of gas in the air; And when the gas levels rise above a certain limit, the canaries die of the poison, though dull humans do not notice. With relatively rudimentary mining equipment, workers took a canary with them every time they went down to the mine as a “gas detection indicator” to evacuate in case of danger.
Grayscale release structure diagram is as follows:
2, gray release/canary release consists of the following steps:
Prepare artifacts for each phase of deployment, including build artifacts, test scripts, configuration files, and deployment manifest files. Remove the Canary server from the load balancing list. Upgrade the Canary application (drain the traffic and deploy it). Automate test your application. Add the Canary server back to the load balancing list (connectivity and health check). If canary is successfully tested online, upgrade the remaining servers. In addition, grayscale publishing can also set the route weight, and dynamically adjust different weights to verify the new and old versions.