Grayscale publishing is not a very new concept. If a product needs rapid iterative development and launch, it needs to ensure quality and ensure that the newly launched system can quickly control the impact surface once problems occur. Therefore, it is necessary to design a grayscale release system.

The function of grayscale publishing system is that it can guide the user’s traffic to the new online system according to its own configuration, to quickly verify the new function modification, and once there is A problem, it can also be immediately restored. In short, it is A set of A/BTest system.

It should be something like this:



It is divided into several parts:

  1. The access layer, which accesses client requests and forwards qualified requests to the old and new systems based on the delivered configuration.

  2. Configure the management background, which can configure different forwarding policies for the access layer.

  3. New and old business servers that handle client requests.

As for the design of access policies, the protocol layer needs to design the parameters for forwarding from the very beginning, and these parameters should be separated from the specific protocol body to reduce the parsing of protocols at the access layer. For example, if the request from the client is HTTP, these parameters can be placed in the HEADER section, and the access layer can obtain the parameters required by the forwarding policy without having to parse the body data. Of course, the data placed in the HEADER, since it is not encrypted, is another concern.

Of course, the simplest forwarding policy can be based on the CLIENT IP address, which is a rough division policy.

Similarly, the old and new servers should be compatible with the new and old client protocols, which is also the basis for gray scale publishing. How to design a good application protocol with good scalability is not considered here.

Then, if the management background sends a new forwarding policy, the access layer can immediately sense and switch to the new policy. There are several different ways of doing it. If the access layer is a server such as Nginx, users only write their own Nginx module on it to implement policy forwarding, then it may need to deploy an Agent service on each access server, mainly used for:

  1. Receive policies from the admin background, update the Nginx configuration, and gracefully restart the Nginx service.

  2. Periodically check the Nginx service status of this machine and report.

If the access layer is not a service like Nginx, then you can also do a pub-sub model of the subscriber, ZK or Redis can be used, subscription management background services can be processed.