Several dimensions.

time

1S how many times. 1 s.

The number of

1S how many times. Times, counter.

Time dimension

Time window algorithm.

Dimension of Token set

Controls the rate at which tokens are written to and from collections.

interface

A single interface, or an entire service.

Standalone or distributed

Because this is going to affect the algorithm.

Standalone, that is, standalone traffic limiting algorithm.

Distributed traffic limiting: 1. If traffic limiting is implemented for a single service, use the single-node traffic limiting algorithm. At this point, the total number should be the number of single machines * the number of machines. 2. For clusters, use the distributed traffic limiting algorithm. In this case, the total number of degrees is the number of degrees of the central counter.

reference

6, how to choose single-machine or distributed traffic limiting first need to explain: the single-machine traffic limiting and distributed traffic limiting and previously mentioned single-machine traffic limiting algorithm and distributed traffic limiting algorithm is not a concept! To improve service performance and availability, microservices are deployed in multi-instance clusters. Single-node traffic limiting means that traffic limiting on interfaces of each instance in a cluster is implemented independently. For example, the interface access frequency of each instance is limited to 1000 times/second. Distributed traffic limiting means that service-level traffic limiting is provided to limit the access frequency of microservice clusters. For example, the caller of A is limited to request user services for A maximum of 10,000 times per minute. Distributed traffic limiting can be implemented using either the stand-alone traffic limiting algorithm or the distributed traffic limiting algorithm.

The original purpose of single-node traffic limiting is to prevent sudden traffic from overwhelming the server, so it is suitable for limiting concurrent traffic. Distributed traffic limiting applies to fine-grained traffic limiting or access quota. Different callers implement different traffic limiting rules for different interfaces. Therefore, hits per second traffic limiting is more suitable. From the point of view of ensuring system availability, single-node traffic limiting is more advantageous, while distributed traffic limiting is more suitable for preventing a certain caller from over-competing for service resources.

The common deployment architectures between distributed traffic limiting and microservices are as follows:

  1. Access layer (API-gateway) integration of traffic limiting function is the most reasonable architecture mode under the premise of API-Gateway in microservice architecture. If apI-Gateway is deployed in single-instance mode, use the single-instance traffic limiting algorithm. If apI-Gateway is deployed with multiple instances, a distributed traffic limiting algorithm must be used in order to achieve service-level traffic limiting.

  2. Traffic limiting encapsulated as RPC Service After receiving an interface request, the micro service checks whether the interface request exceeds the traffic limiting threshold through the RPC interface exposed by the traffic limiting service. This architecture requires the deployment of a traffic limiting service, which increases o&M costs. In this deployment architecture, the performance bottleneck will appear in the RPC communication between microservices and traffic limiting services. Even though the single-node traffic limiting algorithm can achieve 2 million TPS, the request traffic limiting of 100,000 TPS is not bad after the RPC framework.

  3. Integration of traffic limiting function in microservice system This architecture mode does not need to deploy services independently, which reduces operation and maintenance costs. However, traffic limiting code will have some coupling with business code. However, traffic limiting function can be integrated at the section layer and decoued from business code as far as possible. If service-level distributed traffic limiting is implemented, the distributed traffic limiting algorithm must be used. If single-node traffic limiting is implemented for each micro-service instance, the single-node traffic limiting algorithm can be used.

7. Use different traffic limiting fuse policies for different services. The fuse policy in this section refers to how to process interface requests when the interface reaches the traffic limiting upper limit. Some traffic limiting fuses have been mentioned before. The so-called denial traffic limiting means that the request is rejected after the maximum allowed access frequency is exceeded, such as returning HTTP status code 429, etc. The so-called blocking traffic limiting means that the request is queued after the maximum allowed access frequency is exceeded. In addition, there are other traffic limiting fuses, such as logging, sending alarms, and service degradation.

The same system may have different fusing policies for different callers. For example, for callers sensitive to response time, we may adopt the fusing policy of direct rejection, and for callers insensitive to response time, such as background jobs, we may adopt the fusing policy of blocking queuing processing.

We’ll look at some other fusing strategy application scenarios, such as current limiting function has just launched, in order to verify the validity of the current limit algorithm and the rationality of the current limiting rules, to ensure that no shot request, can first use logging + the alarm current limiting fuse strategy, through the analysis of the log to determine current limiting function normal work, further upgraded to other current-limiting fuse strategy.

Different fusing strategies also have influence on the selection of traffic limiting algorithms. For example, token bucket and leak-bucket algorithms are more suitable for blocking fusing scenarios, and time window based fusing algorithms are more suitable for denying fusing scenarios.

Mp.weixin.qq.com/s/k9tm-4lBw…