1. Why do we limit traffic?

Traffic limiting includes crash prevention and service traffic limiting.

Current limiting mode:

  • Single current limiting
  • Cluster current-limiting
  • The gateway current-limiting
  • Automatic fuse downgrade

2. Traffic limiting algorithm

Sliding window counter

The traffic at the sending end is subject to the size of the traffic window. Excess traffic cannot be processed. After processing a request, the window slides one bit;

For example, if our service can handle 60 requests per minute, we can divide a minute into 60 Windows and move one window every second. Service is blocked when the number of requests exceeds the maximum number that a window can handle.

Bucket algorithm

Compare the requested flow to filling a bucket with water at an arbitrary rate and out at a fixed rate;

Because the volume of the bucket is unchanged, the finishing rate is guaranteed;

Implementation: the whole queue, add tasks full after the denial of service, at a constant speed from the queue to the task;

Token bucket algorithm

It’s still fetching tasks at a certain rate, but the new way to add tasks is to go to the token bucket and then add buckets;

We can limit traffic by producing tokens and buckets at a constant rate;

3. Current limiting framework

alibab-sentinel

Can realize single machine, distributed current limiting; For Dubbo, the service provider can limit traffic only for specific applications.

There is a problem

1) Traffic limiting and rules are stored in the memory of the application node. After the application is released and restarted, they will become invalid and cannot be accepted in the production environment.

2) The default rule delivery is based on the machine node dimension rather than the application dimension; A normal company is deployed in a cluster, which cannot support traffic limiting in a cluster.

3) Data records are stored in memory. Dashboard data is only kept for 5 minutes. If missed, it may be impossible to restore the “crime scene” and see the traffic trend;

4) If there are 500 applications involved in traffic limiting and each application has 5 nodes, Dashboard will definitely become a bottleneck and the thread pool on a single machine cannot handle it;

Redis current-limiting

Redis implements distributed traffic limiting and generally adopts the token bucket approach. Use lua script to solve the concurrent problem of INCR() resetting the number of tokens in the token bucket;

Lua script content flowchart

This section describes redis commands

incr

expire

Springboot use adaptation

Which items need to be adapted to springboot access traffic limiting? Here are a few of them;

feature

Implementation scheme

  • ✅ is used based on annotations

Spring aop implementation

  • ✅ User-defined traffic limiting key (for example, user-based parameter transfer)

The Spring EL expression takes a parameter name object

  • ✅ Downgrading after traffic limiting

1) Throw an exception. 2) Handle the exception according to user-defined methods

  • ✅ Dynamic adjustment of traffic limiting times is supported

The dynamic parameter is introduced in the annotation, and the set flow limit value is obtained from Redis

The above functions have been implemented: source address

see

API call times limit implementation