background

Suppose we have a set of back-end services that provide some functionality to the user, the (simplified) invocation relationship between the services might be shown below

Feature_x, Feature_y, feature_z, feature_z

These feature code changes may involve several same or different back-end services, for example:

1) Feature_x modifies the < User Account Display > interface of app_srv0 (assuming the name is user_account_show), The user_account_show interface depends on the < Account balance interface >user_account_balance of APP_SRv1

2) Feature_y has modified the user_Account_balance interface. This change promises that the outgoing output of the interface will not change, but only changes the user account balance calculation formula.

The promise of “user_Account_balance interface will not be changed” until the Feature_y test is passed cannot be guaranteed. If we have only one test environment, multiple features can only be tested in this test environment. So the tests for Feature_x and feature_y are only serial. If tested in parallel, the service deployment would look like this (light green for Feature_x, purple for Feature_Y) :

During the test, if feature_y has a bug that causes user_Account_balance to return incorrect data, the user_Account_show interface will return incorrect information. But the actual feature_x changes.

Common solutions

1) Serial test, disadvantages: constant time cycle, slow on-line of project feature

2) Parallel deployment of multiple sets of test environments, disadvantages: high resource cost and maintenance cost, if there are many services or multiple versions of parallel development, it is basically not executable

3) The use of a set of environments to force parallel testing. Disadvantages: The interaction between features may lead to rework of tests of multiple features, which may prolong the time of each feature test

Solution idea

Let’s cut the crap. Last picture

From the above analysis, we can know that what should be considered is “how to achieve test environment feature isolation at a low cost”. “Low cost” means to deploy as few services as possible, only the modified services can be deployed. Feature isolation means that requests from different feature branches are routed to the service version that supports the corresponding feature. Refer to the picture above:

Feature_x has modified app_SRv1, app_SRV4, and app_srv6 to get new versions of the services app_SRv1_x, APP_SRv4_x, and app_SRv6_x, respectively

Feature_y has modified app_SRv4 and app_srv7 to get new versions of the services app_SRv4_y and app_SRv7_y, respectively

Assuming that feature_X request is Rx, Feature_Y request is Ry, and the baseline request is R0, as long as Rx and Ry request go through different call chains, the tests of Feature_x and Feature_Y will not affect each other, that is:

Call chain of Rx: APP_SRv0 – > APP_SRv1_x – > APP_SRv2 – > APP_SRv4_x – > APP_SRv6_x – > APP_SRv7

Call chain of Ry: APP_SRv0 – > APP_SRv1 – > APP_SRv2 – > APP_SRv4_Y – > APP_SRv6 – > APP_SRv7_Y

How do you do this

Let the request carry routing information (the following format is in non-standard JSON format, but similar formats are easy to display)

The common header of the request carries a version-bound route information, such as feature_X_20180101, feature_Y_20180101

//feature_x
//comm_head
{
  "route_id":"feature_x_20180101"
}

//feature_y 
//comm_head
{
  "route_id:feature_y_20180101"
}
Copy the code

Store a routing table

//route_table
{
  "feature_x_20180101": [
    {
      "appid": "app_srv1",
      "srv_version": "app_srv1_xxx",
      "addr": "ip:port"
    },
    {
      "appid": "app_srv4",
      "srv_version": "app_srv4_xxx",
      "addr": "ip:port"
    },
    {
      "appid": "app_srv1",
      "srv_version": "app_srv6_xxx",
      "addr": "ip:port"
    }
  ],
  "feature_y_20180101": [
    {
      "appid": "app_srv4",
      "srv_version": "app_srv4_yyy",
      "addr": "ip:port"
    },
    {
      "appid": "app_srv7",
      "srv_version": "app_srv7_yyy",
      "addr": "ip:port"
    }
  ]
}
Copy the code

Each service maintains its own APPID. The rPC-initiating service obtains routing table configuration in some way, searches for mapped AppID from the obtained routing table, compares it with the target AppID to be called (the caller must know the appID of the dropped party), and finds the address and port of the service to be called, for example:

After feature_x’s request Rx reaches APP_srv0, app_srv0 calls app_srv1, and APP_SRv0 resolves the route_id of the request header (comm_head) : Feature_x_20180101: Use route_id to get all the configurations under feature_X_20180101 in the routing table. Look for the items with appID =app_srv1 in the configuration (the number of services is limited here, so don’t worry about traversal performance). Get the corresponding IP and port to initiate the call, so that the Rx request triggers the first RPC call to feature_x version of app_SRv1_x instead of the baseline version of app_SRv1 or Feature_Y version of app_SRv1_y. If there is no corresponding call, the default route is used. That is, calling the baseline version; The route_ID in the request header is passed through when the call is made, so that each service in the subsequent call chain can use the above procedure to find the version branch that the request is to call. What needs to be added here is the maintenance of the routing table and the writing of the request header when the client initiates the request. This method can also be used to perform AB tests after they are published online. If it is not necessary, the configuration can be modified or the front end does not carry routing information when sending requests.

Manage and maintain routing tables

1) Hand it to Mesh

2) When the service is published, the routing information of its own version is registered through the registry and delivered to the machine where the service is located through the registry. The service framework reads the configuration locally through the Agent

Own a little thinking, welcome more communication!!