Why is it necessary to limit traffic

Flow limiting has become a standard technical solution in dealing with high concurrent pressure scenarios such as seckilling and panic buying, which plays a key role in ensuring the smooth operation of the system. Regardless of the application scenario, traffic limiting is to selectively fuse certain requests based on preset traffic limiting rules to limit traffic that exceeds expectations. Through current limiting, we can control the QPS of the system well, so as to achieve the purpose of protecting the system

  1. System resources are limited, bearing capacity is limited, instantaneous flow is too high, the system is easy to slow down, system downtime, avalanche, etc
  2. Malicious users frequently patronize an interface module, causing the server to break down
  3. Messages are consumed too quickly, causing the database to become overloaded, degrade performance, or even crash

Common traffic limiting algorithms

Counter fixed window algorithm

Fixed Window is also known as Fixed Window (also known as counter algorithm, Fixed Window) traffic limiting algorithm, is the simplest traffic limiting algorithm, through the maintenance of the counter in a unit of time to control the maximum access within the unit of time

The biggest advantage of fixed Windows is that they are easy to implement; And the memory footprint is small, we only need to store the count in the time window; It ensures that more recent requests are processed and that new requests do not starve as old ones pile up. Of course, there is also a critical problem, when the junction of two Windows, the instantaneous flow may be 2N.

bad case

If a malicious user sends 100 requests at 0:59 and 100 requests at 1:00, the user sends 200 requests in 1 second. We just specified a maximum of 100 requests per minute, which is 1.7 requests per second, and the user can exceed our rate limit instantaneously by popping requests at the reset node in the time window. Users could overwhelm our app with this flaw in the algorithm

Single current limiting

Assuming the limit of requests per second is no more than 30, set a counter that will reject requests when they arrive if the counter reaches the threshold, otherwise increment the counter by 1; Reset the counter to 0 per minute. The code implementation is as follows:

package com.lm.java.share.thread2.limit; import java.util.concurrent.atomic.AtomicInteger; /** * @author lm * API can only be accessed N times in a minute * @created 2021/7/7 PM 3:12 **/ public class CounterLimiter {private int windowSize; // Window size in milliseconds private int limit; Private AtomicInteger count; Public CounterLimiter(int windowSize, int limit) {this.limit = limit; this.windowSize = windowSize; count = new AtomicInteger(0); // Start a thread, @override public void run() {while (true) {try {thread.sleep (windowSize); } catch (InterruptedException e) { e.printStackTrace(); } count.set(0); } } }).start(); Public synchronized Boolean tryAcquire() {int newCount = count.addandGet (1);} public synchronized Boolean tryAcquire() {newCount = count.addandget (1); if (newCount > limit) { return false; } else { return true; }} // Test public static void main(String[] args) throws InterruptedException {CounterLimiter CounterLimiter = new CounterLimiter(1000, 30); int count = 0; For (int I = 0; i < 50; i++) { if (counterLimiter.tryAcquire()) { count++; }} system.out. println(" + count + ", "+ (50-count)); Thread.sleep(2000); // Simulate 50 requests and see how many pass count = 0; for (int i = 0; i < 50; i++) { if (counterLimiter.tryAcquire()) { count++; }} system.out. println(" + count + ", "+ (50-count)); }}Copy the code

Results:

30 is passed in the first 50 requests, and traffic limit is 20. 30 is passed in the second 50 requests, and traffic limit is 20Copy the code

Distributed current limiting

General distributed we are with the help of Redis + Lua to achieve, put two Lua script reference

  • One-second limiting (how many requests are limited per second)
  • A custom parameter flow limit (custom how much time limit how many requests)

Second limiting (how many requests are limited per second)

Each request will put the current time, accurate to the second, into Redis as the key. The timeout period is set to 2s, and Redis increments the value of the key. Redis is used to write to Lua scripts. The single thread mechanism of Redis can ensure the atomicity of each Redis request Local currentLimit = tonumber(redis. Call ('get', ARGV[1])) Key) or "0") if currentLimit > 1 then Redis. call("EXPIRE", key, 2) return currentLimit + 1 endCopy the code
Public Long limit(String maxRequest) {public Long limit(String maxRequest) { String key = LIMIT + String.valueof (System.currentTimemillis () / 1000); List<String> args = new ArrayList<>(); args.add(maxRequest); return eval(getScript("redis/limit-seckill.lua"), Collections.singletonList(key), args); }Copy the code
/** * private Long eval(String script, List<String> keys, List<String> args) {// Execute the script Object result = jedisutil. eval(script, keys, args); Return (Long) result; return (Long) result; }Copy the code
Public static Object eval(String script, List<String> keys, List<String> args) {Object result = null; try (Jedis jedis = jedisPool.getResource()) { result = jedis.eval(script, keys, args); return result; } catch (Exception e) {throw new CustomException(" Script =" + script + "keys=" + keys.toString() +") args=" + args.toString() + " cause=" + e.getMessage()); }}Copy the code

Custom parameter flow limiting (customize how much time limits how many requests)

- Determines whether the maximum number of traffic limiting requests in the current time window exceeds the threshold. - Returns an error when the threshold is reached. Indicates that the request is restricted. If not, write to Redis using Lua scripts. If not, write to Redis using Lua scripts. If not, write to Redis using Lua scripts. Key name local requestKey = KEYS[2] Local maxRequest = tonumber(ARGV[1]) -- local nowTime = tonumber(ARGV[2]) -- timeout, Local timeRequest = tonumber(ARGV[3]) Local currentTime = tonumber(redis. Call ('get', timeKey) or "0") Local currentRequest = tonumber(redis. Call ('get', If currentTime + timeRequest > nowTime is greater than currentTime + timeRequest > nowTime then If currentRequest + 1 > maxRequest then if currentRequest + 1 > maxRequest then return 0. Redis. call("INCRBY", requestKey, 1) return currentRequest + 1; redis.call("INCRBY", requestKey, 1) return currentRequest + 1; End else -- reset after timeout, Call ('set', timeKey, nowTime) redis. Call ('set', requestKey, '0') -- set expiration time redis. Call ("EXPIRE", TimeKey, timeRequest / 1000) redis. Call ("EXPIRE", requestKey, timeRequest / 1000) -- add one redis. Call ("INCRBY", requestKey, 1) return 1; endCopy the code

Counter sliding window algorithm

In order to prevent instantaneous flow, a fixed Window can be divided into multiple grids one step at a time and moved backward one bit at a time instead of a fixed Window size. This is called a Sliding Window.

For example, each minute can be divided into six 10-second cells, with a counter maintained in each cell and the window sliding forward one cell at a time. Each time a request arrives, it is allowed as long as the total count of all cells in the window does not exceed the threshold. In TCP protocol, the transmission of data packets is also controlled by sliding window.

The sliding window solves the instantaneous flow peak problem in the counter. In fact, the counter algorithm is also a kind of sliding window, but the window is not divided into more fine-grained units. Compared with the counter, it can be seen that when the granularity of window division is finer, the flow control is more accurate and strict.

However, when the traffic in the window reaches the threshold, the traffic will be cut off instantly. In practical applications, the effect of traffic limiting is not to cut off the traffic all at once, but to let the traffic enter the system smoothly.

Single implementation

package com.lm.java.share.thread2.limit; import com.sankuai.meituan.banma.biz.common.util.JsonUtils; import java.time.LocalDateTime; import java.time.ZoneOffset; import java.util.Iterator; import java.util.Map; import java.util.Random; import java.util.TreeMap; / * * * * @ version 1.0 * @ @ author lm desc CounterSildeWindowLimiter * @ created 2021/7/7 afternoon she went * * / public class CounterSildeWindowLimiter {/ * * * * / private limited number of requests per minute long permitsPerMinute; */ private final TreeMap<Long, Integer> counters; */ private final TreeMap<Long, Integer> counters; public CounterSildeWindowLimiter(long permitsPerMinute) { this.permitsPerMinute = permitsPerMinute; this.counters = new TreeMap<>(); } public synchronized Boolean tryAcquire() {public synchronized Boolean tryAcquire() { 10s Long currentWindowTime = localDatetime.now ().toepochSecond (zoneoffset.utc) / 10 * 10; Int currentWindowCount = currentWindowCount (currentWindowTime); int currentWindowCount = currentWindowCount (currentWindowTime); if (currentWindowCount >= permitsPerMinute) { return false; } // merge(currentWindowTime, 1, Integer::sum); return true; } /** * gets all requests in the current window (and removes all invalid child window counters) ** @param currentWindowTime The current child window time * @return The count in the current window */ private int Currentwindowcount (long currentWindowTime) {currentWindowTime = currentwinDowTime-60; int result = 0; // Iterate over the currently stored counters, delete invalid child window counters, Iterator< map.entry <Long, Integer>> Iterator = counters.entryset ().iterator(); Iterator< map.entry <Long, Integer>> Iterator = counters.entryset (). while (iterator.hasNext()) { Map.Entry<Long, Integer> entry = iterator.next(); if (entry.getKey() < startTime) { iterator.remove(); } else { result += entry.getValue(); } } System.out.println(JsonUtils.encode(counters)); return result; } public static void main(String[] args) throws InterruptedException { CounterSildeWindowLimiter counterSildeWindowLimiter = new CounterSildeWindowLimiter(10); for (int i = 0; i < Integer.MAX_VALUE; I++) {if (counterSildeWindowLimiter tryAcquire ()) {System. Out. Println (I + ", "success"); Thread.sleep(new Random().nextInt(10) * 1000); } else {system.out.println (I + "-- "failed); Thread.sleep(new Random().nextInt(10) * 1000); }}}}Copy the code

The results of

0-- "success {"1627416820":1} 1--" success {"1627416820":1,"1627416830":1} 2-- "success {"1627416820":1,"1627416830":2} 3--" success {"1627416820":1,"1627416830":3} 4-- "succeeded {"1627416820":1,"1627416830":3,"1627416840":1} 5--" succeeded {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":1} 6-- "succeeded {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":2} 7-- "succeeded {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3} 8-- "succeeded {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":1} 9-- "succeeded {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 10-- "failed {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 11-- "failed {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 12-- "failed {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 13-- "failed {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 14-- "failed {"1627416820":1,"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 15-- "failed {"1627416830":3,"1627416840":1,"1627416850":3,"1627416860":2} 16-- "succeeded {" 1627416830 ": 3," 1627416840 ": 1," 1627416850 ": 3," 1627416860 ": 2," 1627416890 ": 1}Copy the code

Bucket algorithm

How to smooth the flow limiting? Consider the Leaky Bucket algorithm, where requests are pumped like water at any rate and the Bucket leaks water at a fixed rate. When the injection rate continues to be greater than the drain rate, the drain bucket becomes full and new incoming requests are discarded. Flow limiting and shaping are two core capabilities of the leaky bucket algorithm.

Single implementation

package com.lm.java.share.thread2.limit; import java.util.LinkedList; public class LeakyBucketLimiter { private int capaticy; // private int rate; Private LinkedList<Integer> requestList; public LeakyBucketLimiter(int capaticy, int rate) { this.capaticy = capaticy; this.rate = rate; requestList = new LinkedList<>(); New Thread(new Runnable() {@override public void run() {while(true){if(! Requestlist.isempty ()){system.out.println (" outflow: "+requestList.removeFirst()); } try {// Thread.sleep(1000 / rate); } catch (InterruptedException e) { } } }}).start(); } public synchronized boolean tryAcquire(Integer i){ if(capaticy - requestList.size() <= 0){ return false; }else{ requestList.addLast(i); return true; } } public static void main(String[] args) throws InterruptedException { LeakyBucketLimiter leakyBucketLimiter = new LeakyBucketLimiter (5, 2); for(int i = 1; i <= 10; I + +) {if (leakyBucketLimiter tryAcquire (I)) {System. Out. Println (I + "the request is accepted"); Thread.sleep(100); }else{system.out.println (I + "number request rejected "); Thread.sleep(100); }}}}Copy the code

advantages

  • The leakage rate is fixed, which can play the role of rectification. After the funnel algorithm, it becomes a stable flow with a fixed rate, thus protecting the downstream system

disadvantages

  • The problem of sudden traffic cannot be solved. When there are a large number of sudden requests in a short time, each request has to wait in the queue for a period of time before being responded to or directly discarded, even if the server has no load at this time
  • System resources cannot be fully utilized because the leakage rate of the leaky bucket is fixed, and the leaky bucket does not allow burst traffic to pass through, even though more traffic can be handled downstream at a certain time

Token bucket algorithm

The principle of the token bucket algorithm is that the system will put tokens into the bucket at a constant rate, and if the request needs to be processed, it needs to get a token from the bucket first. When no token is available in the bucket, the request will be blocked or wait. The simple process is as follows

  1. All requests require an available token before they can be processed
  2. Set the rate at which tokens are added to the bucket based on the flow limiting size
  3. The bucket sets a maximum placement token limit, and when the bucket is full, newly added tokens are discarded or rejected
  4. After the request is received, the token in the token bucket should be obtained first, and then other business logic can be carried out after holding the token. After processing the business logic, the token can be deleted directly
  5. The token bucket has a minimum limit, and when the number of tokens in the bucket reaches the minimum limit, the token will not be deleted after the request is processed to ensure adequate flow limiting

The token bucket algorithm supports consumption-before-payment, for example, a request can obtain multiple or all tokens, but later requests need to pay. This means that subsequent requests cannot be fetched until the bucket is filled with tokens

Single implementation

  • You can use RateLimiter in the Guava package directly. Here is your own code mock implementation
package com.lm.java.share.thread2.limit; /** * lm * Token bucket algorithm is an improvement on the leaky bucket algorithm, in addition to limiting the average rate of calls while allowing a certain degree of traffic burst. **/ public class TokenBucketLimiter { private int capaticy; Private int rate; // Token generation rate private int tokenAmount; Public TokenBucketLimiter(int capaticy, int rate) {this.capaticy = capaticy; this.rate = rate; tokenAmount = capaticy; New Thread(new Runnable() {@override public void run() {while (true){synchronized (this){tokenAmount ++; if(tokenAmount > capaticy){ tokenAmount = capaticy; } } try { Thread.sleep(1000 / rate); } catch (InterruptedException e) { } } }}).start(); } public synchronized boolean tryAcquire(int i){ if(tokenAmount > 0){ tokenAmount --; System.out.println(I +" get token "); return true; }else{ return false; } } public static void main(String[] args) throws InterruptedException { TokenBucketLimiter tokenBucketLimiter = new TokenBucketLimiter (5, 2); for(int i = 1; i <= 10; I + +) {if (tokenBucketLimiter tryAcquire (I)) {System. Out. Println (I + ", the request is processing "); Thread.sleep(100); }else{system.out.println (I + "number request rejected "); Thread.sleep(100); }}}}Copy the code

The results of

One gets token number one and gets processed 2 gets token number two and gets processed 3 gets token number three and gets processed 5 gets token number five gets processed 7 gets token number seven gets processed 8 gets denied 9 gets denied Request number 10 was deniedCopy the code

Distributed implementation

-- Token bucket flow limit -- unique token identifier Local bucketKey = KEYS[1] -- time of last request local last_mill_request_key = KEYS[2] -- Token bucket capacity local limit = Local permits = tonumber(ARGV[1]) -- number of requested tokens Local Permitting = tonumber(ARGV[2]) -- token influx rate Local rate = Tonumber (ARGV[3]) -- current time local Curr_mill_time = tonumber(ARGV[4]) -- Add tokens -- get the number of current tokens local current_limit = tonumber(redis. Call ('get', Local last_mill_request_time = tonumber(redis. Call ('get', bucketKey) or "0") -- last_mill_request_time = tonumber(redis. Last_mill_request_key) or "0") -- select last_mill_request_time (last_mill_request_time == 0 then -- select last_mill_request_time (last_mill_request_key) from last_mill_request_key (last_mill_request_key) or "0") -- select last_mill_request_time (last_mill_request_time) from last_mill_request_key (last_mill_request_key) redis.call("HSET", last_mill_request_key, curr_mill_time) return 0 else local add_token_num = math.floor((curr_mill_time - last_mill_request_time) * rate) end -- If current_LIMIT + add_token_num > LIMIT then current_LIMIT = LIMIT else current_limit = current_limit + Add_token_num end redis. Pcall ("HSET",bucketKey, current_limit) -- Set expiration time redis. Call ("EXPIRE", bucketKey, current_limit) Current_limit-permits < 1 THEN return 0 else current_limit-permits = current_limit-permits - Permitting redis.pcall("HSET", bucketKey, current_limit) -- redis.call("EXPIRE", bucketKey, Redis. Call ("HSET", last_mill_request_key, curr_mill_time) endCopy the code

A leaky bucket is compared to a token bucket

The leaky bucket algorithm and the token bucket algorithm look similar on the surface, and it is easy to confuse the two. But in fact, the two have very different characteristics and are used for different purposes. The difference between the leaky bucket algorithm and the token bucket algorithm is:

  • Leaky bucket algorithm can restrict data transfer rate forcibly
  • The token bucket algorithm can limit the average transmission rate of data while allowing some degree of burst transmission

In some cases, the leak-bucket algorithm cannot use network resources efficiently. Because the leaky bucket leakage rate is fixed, the leaky bucket algorithm cannot make a single data flow reach the port rate even if there is no congestion in the network. Therefore, leaky bucket algorithm is inefficient for traffic with burst characteristics. The token bucket algorithm can satisfy the traffic with burst characteristics. In general, the leaky bucket algorithm is combined with the token bucket algorithm to provide more efficient control of network traffic

Guava current-limiting ratelimiter

Guava’s RateLimiter provides an implementation of the token bucket algorithm. RateLimiter has two flow limiting modes: SmoothBursty (constant token generation speed), One is SmoothWarmingUp: the token generation rate increases slowly until it remains at a steady value.

The inheritance structure is as follows The core idea: After the current request is received, the system dynamically calculates the next available service time. If the next request is received before this time, the system needs to wait. The nextFreeTicketMicros attribute in the SmoothRateLimiter class represents the time when the next response is available. For example, if we set QPS to 1, after this request is processed, the earliest time to be able to respond to the next request is one second later

Commonly used method

Stable mode (SmoothBursty: token generation speed is constant)

The test code

Public static void main(String[] args) {// Create a RateLimiter, RateLimiter RateLimiter = RateLimiter. Create (0.5); Int [] a = {1,6,2}; for(int i = 0; i < a.length; ++ I) {// Acquire (x); Println (System.currentTimemillis () + "acq" + a[I] + ": system.out.println (system.currentTimemillis () + "acq" + a[I] +" wait " + rateLimiter.acquire(a[i]) + "s"); }}Copy the code

The results of

1627393585611 acq 1: wait 0.0s
1627393585613 acq 6: wait 1.99704s
1627393587613 acq 2: wait 11.997597s
Copy the code

Analysis of the

1. Create (double permitsPerSecond) RateLimiter RateLimiter RateLimiter = RateLimiter. Create (0.5);

2, Acquire (int permits) obtain X tokens from RateLimiter, the method will be blocked until the request is obtained; Three main things have been done

public double acquire(int permits) { //1. Permitting = reserve(permits); permitting = reserve(permits); / / 2. Sleep microsToWait stopwatch time window. The sleepMicrosUninterruptibly (microsToWait); Return 1.0 * microsToWait/seconds.tomicros (1L); } final long reserve(int permits) {final long reserve(int permits) {final long reserve(int permits) { Synchronized (mutex ()) {/ / computing time need to wait for the return reserveAndGetWaitLength (permits, a stopwatch. ReadMicros ()); }}Copy the code

The reserveAndGetWaitLength method also updates the token count and calculates the wait time by calling the reserveestAvailable method. This method returns the time required to wait and is the core interface of RateLimiter

@Override final long reserveEarliestAvailable(int requiredPermits, long nowMicros) { resync(nowMicros); //1. Determine whether a new token is generated based on the current time and the expected time of the next second. StoredPermits and next request time nextFreeTicketMicros Long returnValue = nextFreeTicketMicros; //2. According to the number of required tokens requiredPermits and storedPermits The number of tokens currently held storedPermits the number of available tokens held storedPermitsToSpend and freshPermits the number of tokens to be prepaid are calculated respectively double storedPermitsToSpend = min(requiredPermits, this.storedPermits); double freshPermits = requiredPermits - storedPermitsToSpend; long waitMicros = storedPermitsToWaitTime(this.storedPermits, storedPermitsToSpend)//3. Calculate wait time of storedPermitsToSpend and freshPermits + (Long) (freshPermits * stableIntervalMicros); try { this.nextFreeTicketMicros = LongMath.checkedAdd(nextFreeTicketMicros, waitMicros); / / 4. Update nextFreeTicketMicros} the catch (ArithmeticException e) {this. NextFreeTicketMicros = Long. MAX_VALUE; } this.storedPermits -= storedPermitsToSpend; //4. Update StoreDpermitting return returnValue; }Copy the code

The essence of RateLimiter’s support for burst traffic is to split requiredpermitStoSpend (hold the number of available tokens) and FreshperpermitstoSpend (hold the number of tokens that need to be advanced) into storedPermitsToSpend (hold the number of available tokens) and Freshpermitpermits (hold the number of tokens that need to be advanced). Calculate the time required to wait separately, and then update nextFreeTicketMicros for the next token acquisition

For example: The current RateLimiter holds 4 tokens and the current request requires 6 tokens; Four out of the six tokens can be obtained directly from the tokens held, while the other two tokens requiring advance payment need to be calculated separately;

GetReqWaitTime (6) = getWaitTime(4) + getFreshWait(6-4)Copy the code

In SmoothBursty mode, getWaitTime(4) is available directly, i.e. time=0; GetFreshWait (6-4) = freshPermits * stableIntervalMicros

SmoothWarmingUp: the token generation rate increases slowly until it reaches a steady value

SmoothWarmingUp is another instance of RateLimiter. Unlike SmoothBursty, there is a “warm-up” concept. That is, if the current system is in a “cooling off period” (that is, the number of tokens currently held is greater than a certain threshold), the wait time for the next token acquisition is longer than the linear time for SmoothBursty mode and gradually decreases to a stable value

Public static void main(String[] args) throws InterruptedException { RateLimiter RateLimiter = RateLimiter. Create (5, 4000, timeunit.milliseconds); for(int i = 1; i < 50; i++) { System.out.println(System.currentTimeMillis() + " acq " + i + ": wait " + rateLimiter.acquire() + "s"); if(i == 15) { Thread.sleep(2000); System.out.println(System.currentTimeMillis() + " acq " + 15 + ": wait " + rateLimiter.acquire() + "s"); }}}Copy the code

The results of

1627394552090 ACq 1: Wait 0.0s 1627394552093 ACq 2: wait 0.577101s 1627394552679 ACq 3: Wait 0.530931s 1627394553215 ACq 4: wait 0.494789s 1627394553715 ACq 5: wait 0.454858s 1627394554174 ACq 6: Wait 0.415823s 1627394554594 ACq 7: wait 0.376547s 1627394554975 ACq 8: Wait 0.335186s 1627394555312 ACq 9: Wait 0.298016s 1627394555611 ACq 10: Wait 0.259247s 1627394555871 ACq 11: Wait 0.219174s 1627394556095 ACq 12: Wait 0.195104s 1627394556294 ACq 13: Wait 0.196015s 1627394556495 ACq 14: Wait 0.194949s 1627394556695 ACq 15: Wait 0.194917s 1627394558897 ACq 15: Wait 0.194917s 1627394558897 ACq 16: Wait 0.341318s 1627394559239 ACq 17: Wait 0.257622s 1627394559544 ACq 18: Wait 0.257622s 1627394559544 ACq 19: Wait 0.215828s 1627394560028 ACq 20: Wait 0.195105s 1627394560227 ACq 21: Wait 0.19514s 1627394560427 ACq 22: Wait 0.19554s 1627394560627 ACq 23: Wait 0.19557s 1627394560828 ACq 24: Wait 0.195113s 1627394561028 ACq 25: Wait 0.195055s 1627394561226 ACq 26: Wait 0.196228s 1627394561428 ACq 27: Wait 0.195024s 1627394561628 ACq 28: Wait 0.194914s 1627394561828 ACq 29: Wait 0.194966s 1627394562027 ACq 30: Wait 0.19527s 1627394562227 ACq 31: Wait 0.196018s 1627394562427 ACq 32: Wait 0.195358s 1627394562627 ACq 33: Wait 0.195354s 1627394562827 ACq 34: wait 0.19538s 1627394563026 ACq 35: Wait 0.196817s 1627394563226 ACQ 36: Wait 0.19613s 1627394563426 ACQ 37: Wait 0.19613s 1627394563628 ACQ 38: Wait 0.194922s 1627394563827 ACq 39: Wait 0.195142s 1627394564027 ACq 40: Wait 0.195918s 1627394564226 ACq 41: Wait 0.196608s 1627394564428 ACq 42: Wait 0.195045s 162739456426 ACq 43: Wait 0.196887s 1627394564827 ACq 44: Wait 0.195699s 1627394565028 ACq 45: Wait 0.194653s 1627394565227 ACq 46: Wait 0.19539s 1627394565427 ACq 47: Wait 0.195834s 1627394565627 ACQ 48: wait 0.195929s 1627394565827 ACQ 49: wait 0.195981sCopy the code

As can be seen from the output results, RateLimiter has the ability of pre-consumption:

At ACQ 1, there is no wait and a token is directly pre-consumed at ACQ 2 ~ 11. Since the current system is in the cooling period, the waiting time is relatively long and gradually decreases to a stable value acQ 12 ~ 15, and the waiting time tends to be 0.2 seconds, that is, 1/QPS ACQ 15 at the same time, Sleep2 seconds, that is, add 5*2 tokens on the current basis; The system transitioned to the cooling period ACQ 15 ~ the end, and the process of ACQ 2 ~ 15 was repeated