Open API Gateway Practice iii – Limiting traffic

How to design and implement a lightweight open API gateway for limiting traffic

The article addresses: blog.piaoruiqing.com/2019/08/26/…

preface

The development of high concurrency system has multiple system protection means, such as cache, limiting, degrade and so on. At the gateway layer, traffic limiting is widely used. In many cases, it can be considered that traffic limiting on the gateway is not strongly associated with services (related to the carrying capacity of the system), and all subsystems have limited traffic requirements. Therefore, it is appropriate to put some traffic limiting functions on the gateway.

What is current limiting

As we all know, the processing capacity of the server and website application has an upper limit. No matter how high the configuration is, there will always be a limit. If the limit is exceeded, uncontrollable consequences may occur if the request continues to be received.

Online ticket for chestnut 🌰, holiday, often will encounter in the queue, system is busy, please try again later, such as tip, this is the service side has carried on the limits to the amount of unit time to handle the request, beyond the limit will line up, down or even denial of service, or if the system broke down, everyone can’t get tickets ╮ (￣ del ￣) ╭.

First, we give the definition of current limiting: Current limiting is one of the protection methods of high concurrency system, which is widely used in the gateway layer. Its purpose is to speed up the number of concurrent requests or limit the number of requests in a time window, and queue up, degrade or even deny service once the threshold is reached.

The ultimate goal is to damage the service instead of not serving it if the concurrency is too high.

Commonly used to limit the flow of play

The token bucket

The token bucket algorithm is a bucket that holds a fixed number of tokens and adds tokens at a fixed rate. As shown in figure:

• Add tokens to the bucket at a fixed rate.
• Refuse to add new token when bucket is full.
• Each request consumes one token (or the corresponding token number can be consumed depending on the packet size).
• Reject the request (or wait) when the token is insufficient.
• Features: Can deal with a certain degree of emergency.

Take a more common example in real life to understand, cinema ticket, each movie is sold a certain number of tickets, if you come late (later request) no tickets, either wait for the next one (waiting for a new token issue) or do not watch (rejected).

bucket

Leaky bucket is a bucket with a hole at the bottom, and water can flow out at a uniform speed (at this time, pressure is not considered, no bar (￣ ￣)). Therefore, unlike token bucket, leaky bucket algorithm is uniform consumption, which can be used for flow shaping and flow control. As shown in figure:

• Fixed capacity leakage tank, according to the fixed rate of water (do not bar water depth and pressure problems).
• The inflow rate is fixed, and the overflow is discarded.
• Features: Smooth processing speed.

Apply stage limiting

A single application has its limits, and in high concurrency it is necessary to protect against overload to prevent the system from crashing with too many requests. The simplest and most crude way to do this is to use a counter for control, +1 on request and -1 on completion, but we can also use the aforementioned token buckets and leaky buckets for more sophisticated limiting. If the gateway is a single application, we can limit the flow directly at the application level without using other media.

counter

This is the simplest and most crude way to implement it,

try {
    if (counter.incrementAndGet() > limit) {
        throw new SomeException();
    }
    // do something
} finally {
    counter.decrementAndGet();
}
Copy the code

The token bucket

Guava provides an implementation of the token bucket algorithm.

@Test
public void testGuavaRateLimiter(a) throws InterruptedException {
    RateLimiter limiter = RateLimiter.create(5);
    TimeUnit.SECONDS.sleep(1);	// Wait a second to issue several tokens
    for (int index = 0; index < 10; index++) {
        System.out.println(limiter.acquire()); // Prints the wait time}}Copy the code

The output is:

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.196108 0.194372 0.19631 0.198373Copy the code

After the tokens are exhausted, subsequent requests wait for a new token before they can proceed.

The application of level-limiting is simple to implement, but its limitation is that it can not carry out full limiting, and it is helpless for the cluster.

Distributed current limiting

The key to fully limiting streams in a cluster is to record limiting information in a shared medium, such as Redis or memcached. In order for limiting to be accurate, writes must be atomic operations.

Redis+Lua is a good choice. Here is a sample Lua script:

local key = KEYS[1] -- KEY of traffic limiting
local limit = tonumber(ARGV[1])	-- Current limiting size
local current = tonumber(redis.call('get', key) or '0')
if current + 1 > limit then
    return 0
else
    redis.call('INCRBY', key,'1')
    redis.call('expire', key,ARGV[2])	-- Expiration time
    return current + 1
end
Copy the code

• Distributed traffic limiting will distribute the license to the shared media.
• The token acquisition (consumption) operation must be atomic.
• Shared media should be highly available (Redis cluster)

conclusion

As a layer of barrier outside the internal system, the gateway plays a certain protective role inside, and current limiting is one of them. Traffic limiting at the gateway layer can simply limit traffic on interfaces of different services. It can also be considered as a function module of the gateway (such as configuration of traffic limiting rules, statistics, statistics based on user dimensions, and traffic limiting).

If this article is helpful to you, please give a thumbs up (~ ▽ ~)”

Series of articles:

• Open API Gateway Practice # 1 — Design an API gateway
• Open API Gateway Practice ii – Replay attack and Defense
• Open API Gateway Practice iii – Limiting traffic

Welcome to our official account: