Since the GA release in late October last year, Sentinel recently released another milestone version, V1.4 (the latest version is V1.4.1), adding cluster flow control functionality that developers have been focusing on.

Overview of Cluster flow control

Why cluster flow control? Suppose we want to limit the total QPS for calling an API to a user to 50, but the number of machines may be large (say, 100). At this point, it is natural to think of a server to count the total number of calls, and other instances to communicate with this server to determine whether they can be called. This is the most basic cluster flow control approach.

So how is the server deployed? The most intuitive way is to start as a separate Token Server process, deployed independently:

The other is Embedded mode, that is, as a built-in token server and services are started in the same process without separate deployment:

In addition, cluster flow control can also solve the problem of poor overall flow limiting effect caused by uneven flow. Suppose there are 10 machines in the cluster, and we set the single-machine traffic limiting threshold to 10 QPS for each machine. Ideally, the traffic limiting threshold for the whole cluster is 100 QPS. However, in practice, the flow to each machine may be uneven, leading to some machines starting to limit the flow when the total amount is not reached:

Therefore, it is impossible to limit the total flow accurately only by the single machine dimension. Cluster flow control can accurately control the total number of calls in the whole cluster, combined with the single-node flow limiting pocket, can better play the effect of flow control.

Sentinel 1.4.0 introduced the cluster flow control module, which is mainly divided into two parts: Token Client and Token Server:

  • A Token Client is a cluster flow control Client that communicates with the owning Token Server to request a Token. The cluster traffic limiting server returns the result to the client to determine whether to limit traffic. The Sentinel cluster flow control communication layer is implemented by Netty.
  • The Token Server is the flow control Server of the cluster. It processes the requests from the Token Client and determines whether to issue the Token according to the configured cluster rules.

Sentinel cluster flow control supports traffic limiting rules and hotspot rules. Cluster flow control supports two types of threshold calculation:

  • Cluster overall mode: The total QPS of a resource in the cluster cannot exceed this threshold.
  • Single machine sharing mode: In single-node equalization mode, the configured threshold is equal to the threshold that can be supported by a single node. The Token Server calculates the total threshold based on the number of connections (for example, in independent mode, three clients are connected to the Token Server and the configured single-node equalization threshold is 10). Then the total number of clusters calculated is 30), which is restricted according to the total threshold calculated. This approach computes the total threshold in real time based on the current number of connections and is ideal for environments where machines are constantly changing.

Deployment way

The Sentinel cluster flow control server supports both standalone mode and Embedded mode. The advantages and disadvantages of the two:

  • The independent mode is started as an independent token Server process. It is independent and isolated, but requires additional deployment operations. Standalone mode is suitable for providing flow control services for the entire cluster as Global Rate Limiter.
  • The embedded pattern is embedded in the application process as the built-in Token Server. In the embedded mode, all instances in the cluster are peer, and the token server and client can be changed at any time without independent deployment. However, the disadvantage is poor isolation, and the total QPS of the Token Server needs to be limited to prevent the application itself from being affected. The embedded mode is suitable for flow control within an application cluster.

Sentinel provides apis to configure client/server and specify modes, but it is not easy to manage when there are many machines. Generally, we need to use the cluster flow control management function of Sentinel console to centrally manage all token servers and token clients in an application cluster and flexibly allocate them.

configuration

Configuration is an important part of cluster flow control. The flow control configuration of Sentinel cluster mainly includes the following parts:

Cluster Rule Configuration

Dynamic rule sources are required for cluster rule configuration. Take the cluster flow control rule as an example. For the client, we can register the dynamic rule source with the FlowRuleManager in the previous way. For Token Server, we need to register the rule source with the ClusterFlowRuleManager. The recommended method is to register the dynamic rule source on the application side and push the rule directly to the configuration center in Sentinel console, i.e. push mode:

Token Server/Client allocation

Taking the embedded mode as an example, a good practice is to select several idle machines as Token servers and other machines as Token clients in the service cluster based on traffic distribution and real-time load, and divide them into several groups, which are managed by their own Token servers. Finally, form a mapping table similar to:

// ip: token server IP, port: token server port, clientSet: A collection of managed Token clients [{" clientSet ": [" 112.12.88.66 @ 8729", "112.12.88.67 @ 8727"], "IP" : "112.12.88.68", "machineId" : "112.12.88.68 @ 8728", "port" : 11111 }]Copy the code

Then configuration sources such as Token Client/Token Server communication configuration and cluster flow control mode can listen to the data source corresponding to the allocation mapping table to resolve their own identity and related communication configuration. When the mapping table is changed, the identity and configuration of each machine are changed in real time. Sentinel 1.4.1 improves the management page of Sentinel console cluster flow control to allocate Token Servers directly by application dimension. Refer to the instructions later in this article to use them.

Other configuration

Other configurations such as the namespace set of the Token Server (which specifies which applications/groups the Token Server can serve), the maximum allowed total QPS, etc. The configuration can be changed either through the HTTP API reserved for Sentinel or by registering dynamic configuration sources.

Use cluster flow control quickly

Let’s take a look at how to use cluster flow control quickly. To connect a flow control module to a cluster, perform the following steps:

(1) Introduce cluster flow control dependency

Here we run the Token Server in embedded mode, that is, one machine in the application cluster is designated as the Token server and the other machines are designated as token clients.

First, we introduce the dependency of cluster flow control:

<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-cluster-client-default</artifactId>
    <version>1.4.1</version>
</dependency>
<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-cluster-server-default</artifactId>
    <version>1.4.1</version>
</dependency>
Copy the code

(2) Configure the dynamic rule source

To use cluster flow control, we need to configure dynamic rule sources on the application side and push them in real time through the Sentinel console. The process is as follows:

For example, if we use ZooKeeper as the configuration center, we can register the ZooKeeper dynamic rule source with the FlowRuleManager client:

ReadableDataSource<String, List<FlowRule>> flowRuleDataSource = new ZookeeperDataSource<>(remoteAddress, path, source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {}));
FlowRuleManager.register2Property(flowRuleDataSource.getProperty());
Copy the code

We also need to register the cluster rules data source for the Token Server. Since token Server and client can change at any time in embedded mode, we only need to register the dynamic rule source with the ClusterFlowRuleManager for each instance. Token Server abstracts the concept of namespace and supports multiple applications/services, so we need to register a generator that automatically creates dynamic rule sources based on namespace:

// Supplier makes different rule sources for different namespaces based on the dynamic rule sources generated by the namespace, of type SentinelProperty
      
       >.
      
// The default namespace is the application name (project.name).
/ / ClusterFlowRuleManager current-limiting rules for cluster, ClusterParamFlowRuleManager hotspot for cluster rules, a similar configuration mode
ClusterFlowRuleManager.setPropertySupplier(namespace -> {
    return new SomeDataSource(address, dataIdPrefix + namespace).getProperty();
});
Copy the code

(3) The console is modified to adapt to the dynamic rule source

We simply retrofitted the Sentinel console to push the flow control rules directly to the configuration center. Since Sentinel 1.4.0, The Sentinel console provides DynamicRulePublisher and DynamicRuleProvider interfaces for implementing rule push and pull of application dimensions. An example of Nacos push (in the test directory) is provided. We just need to implement our DynamicRulePublisher and DynamicRuleProvider interfaces and specify the corresponding bean name in the FlowControllerV2 class using the @Qualifier annotation. Similar to:

@Autowired
@Qualifier("flowRuleNacosProvider")
private DynamicRuleProvider<List<FlowRuleEntity>> ruleProvider;
@Autowired
@Qualifier("flowRuleNacosPublisher")
private DynamicRulePublisher<List<FlowRuleEntity>> rulePublisher;
Copy the code

The Sentinel console provides pages for application dimension push (/v2/flow). After the above configuration is complete, we can push rules to the configuration center on this page:

(4) The console allocates the Token Server

When all the above steps are complete, we can manage the allocation of Token Servers on the cluster Flow Control Token Server list page in the Sentinel Console. Suppose we start three application instances and select one instance as token Server and the other two as Token clients:

The display mode of the machine on the page is ip@commandPort, where commandPort is the port that the application side exposes to the Sentinel console. After selecting the token server, click the save button to refresh the page and you can see that the token Server has been allocated successfully:

And we can check the connection status of token Server on the page:

(5) Configure rules and observe the effect

Next we configure a cluster traffic limiting rule, Limit com. Alibaba. CSP. Sentinel. Demo. Cluster. App. Service. DemoService: sayHello (Java. Lang. String) resources of cluster total QPS for 10, whether “cluster” option is selected, Threshold Mode Select the overall threshold:

The simulated traffic requests all three machines at the same time, and the effect is observed after a period of time. On the monitoring page, you can see that the total QPS of the cluster dimension of the corresponding resource is stable at 10:

conclusion

Cluster flow control can accurately control the QPS of the whole cluster. Combined with the single-machine flow limiting pocket, the flow control effect can be better played. There are more scenes to discover, such as:

  • The total number of visits to an API is counted at the API Gateway and the total QPS for an API or service is limited
  • The Service Mesh provides global flow control for calls between services
  • The total access frequency of hotspot products in the cluster is limited

Although cluster flow control is useful, it is not a panacea and is recommended only when it is really necessary.

In addition, if cluster traffic limiting is used in the production environment, the management and control end must pay attention to the following issues:

  • Automatic Token Server management (assigning/electing Token Servers)
  • The Token Server is highly available. When a Server becomes unavailable, it automatically failover to another Server

In the future, we also plan to implement a multi-language version of the cluster flow control client and connect to The Service Mesh, so that Sentinel cluster flow control can be used in more scenarios.