Generally speaking, the enterprise r&d process is generally like this: first develop and test features in the test environment, then gray, and finally release to production environment. In addition, for the stability of the production environment, the test environment needs to be isolated from the production environment; One problem is inevitable: the multi-environment problem, that is, how can data from multiple environments (such as test and production environments) be isolated? How to gracefully isolate (no user changes required).

Practical experience in Nacos environmental isolation

What is the environment

When it comes to environmental isolation, the first step is to figure out what environment. At present, there is no unified definition of the word environment. Some companies call it environment, such as Region in Aliyun and Namespace in Kubernetes architecture. In this paper, an environment is considered to be a logically or physically independent set of systems that contain all the components (gateways, service frameworks, microservice registeres, configuration centers, messaging systems, caches, databases, etc.) that handle a given class of requests. For example, many sites have the concept of user ids, which can be divided by user ID. Requests ending in an even number of user ids are processed by one system, and requests ending in an odd number are processed by another system. As shown in the figure below. By environmental isolation we mean physical isolation, that is, different environments refer to different clusters of machines.


What is the use of environmental isolation

Environment concept: a system that contains all the components necessary to process user requests for a specified category of requests. What are the benefits of environmental isolation? From the definition of the concept, environmental isolation has at least three benefits:

  • Fault isolation
  • Fault recovery
  • Gray level test

Fault isolation

First, because the environment is a single component unit capable of processing user requests, that is, the processing link of user requests does not break out of the specified cluster of machines. Even if this part of the machine fails, only some users will be affected, thus isolating the failure within the specified scope. If we divide all machines into ten environments by user ID, the impact of one environment failure on users will be reduced to one tenth, greatly improving system availability.

Fault recovery

Another important advantage of environmental isolation is the ability to quickly recover from failures. When a service in an environment is faulty, you can quickly change the routing direction of user requests by delivering the configuration and route the requests to another environment to achieve second-level fault recovery. Of course, this requires a powerful distributed system support, especially a powerful configuration center (such as Nacos), which needs to quickly push routing rule configuration data to the application process of the whole network.

Gray level test

Gray scale testing is an indispensable part of the R&D process. In the traditional RESEARCH and development process, testing and grayscale links require testing students to do various configurations, such as binding host, configuring JVM parameters, environment variables, etc., which are quite troublesome. After years of practice, Alibaba’s internal test and gray scale are very friendly to development and testing. Through the environment isolation function, requests are processed in the designated machine cluster. Development and testing do not need to do any configuration, which greatly improves the efficiency of research and development.

How does Nacos do environment isolation

Nacos is isolated into multiple physical cluster environments, and the Nacos client does not need to make any code changes to implement automatic routing of the environment

The principle of

Before we begin, let’s make a few constraints:

Applications deployed on one machine are all in one environment; By default, only one environment’s Nacos is connected to an application process. By some means you can get the IP of the machine where the client is; The user has planned the network segment of the machine; Here is a brief introduction to the basic principles:

We know that 32-bit ipv4 in the network can be divided into many network segments, such as 192.168.1.0/24, and generally larger companies have network segment planning, according to a certain use of the network segment. We can use this principle to perform environment isolation. That is, IP addresses in different network segments belong to different environments. For example, 192.168.1.0/24 belongs to environment A and 192.168.2.0/24 belongs to environment B. Those of you who have used Nacos know that there are two ways to initialize a Nacos client instance. One is to tell the client the IP address of the Nacos server directly. The other is to tell the client an endpoint, to which the client queries the LIST of NACOS server IP addresses through HTTP requests. We use Nacos for the second initialization method. Enhance the endpoint function. The mapping between the network segment and the environment is configured on the endpoint. After receiving the request from the client, the endpoint calculates the environment to which the client belongs based on the network segment to which the source IP address of the client belongs, and then finds the IP address list of the corresponding environment and returns it to the client. The following figure


An example of an environment isolated server

The above mentioned constraints and basic principles of IP segment isolation, so how to implement an address server. The simplest method is based on nginx implementation, using Nginx geo module, do IP side and environment mapping, and then use Nginx to return static file content.

  • Install nginx
  • Configure geo mapping in nginx-proxy.conf
geo $env { default ""; 192.168.1.0/24 - env - a; 192.168.2.0/24 - env - b; }Copy the code
  • Configure the nginx root path and forwarding rules, here only need to simply return the content of the static file;
# configure root/TMP /htdocs Rewrite ^(.*)$/$1$env break; }Copy the code
  • Configure the configuration file of the Nacos server IP list. In the/TMP /hotdocs/ Nacos directory, configure the file ending with the environment name. The content of the file is IP, one in a row
$ll /tmp/hotdocs/nacos/
total 0
-rw-r--r-- 1 user1 users 0 Mar  5 08:53 serverlist
-rw-r--r-- 1 user1 users 0 Mar  5 08:53 serverlist-env-a
-rw-r--r-- 1 user1 users 0 Mar  5 08:53 serverlist-env-b
 $cat /tmp/hotdocs/nacos/serverlist 192.168.1.2 instead192.168.1.3 Copy the code
  • validation
curl 'localhost:8080/nacos/serverlist'
192.168.1.2 instead192.168.1.3
Copy the code

At this point, a simple example of environment isolation by IP network segment works, with different NACOS clients on different network segments automatically fetching different NACOS server IP lists for environment isolation. In this way, users do not need to configure any parameters. The code and configuration of each environment are the same. However, students who provide underlying services need to plan and configure the network.