Double salary – distributed limiting

What is current limiting

There are three tools you can use to protect a high-concurrency system: caching, degradation, and limiting traffic. The purpose of cache is to improve the system access speed and increase the capacity of system processing, which can be described as a silver bullet against high concurrent traffic. Degradation means that when the service has a problem or affects the performance of the core process, it needs to be temporarily shielded and then opened after the peak or the problem is solved. There are scenarios where caching and downgrading are not possible, such as scarce resources (seconds, snap), write services (comments, orders), frequent complex queries (the last few pages of comments), so there needs to be a way to limit the amount of concurrency/requests in these scenarios, i.e., flow limiting.

Why limit the flow

When Java programmers interview for a job at a big company or want to earn a decent salary, distributed programming is a sure bet. There are all kinds of problems we need to solve in distributed programs. Today we will talk about distributed limiting.

First of all, let’s talk about why we need to limit the flow. Let’s take a look at the following two pictures

This is the wonder of the National Day golden Week scenic spot, the scenic spot if there are no restrictions on tourists will appear in the picture. Because no limit will exceed the service limit of the scenic spot, once the number of requests exceeds its processing limit, it will collapse. In order to avoid the worst crash, we can only delay the time to enter the scenic area.

Due to the huge traffic volume of Internet companies, the system will conduct an evaluation of the traffic peak when it goes online. In particular, in order to ensure that the system will not be overwhelmed by the huge traffic volume, it will reject part of the traffic when the system traffic reaches a certain threshold.

Traffic limiting causes the system to become unavailable for a short period of time (milliseconds). Generally, the processing capacity of the system is measured by QPS or TPS per second. Assume that the traffic threshold of the system per second is 1000.

Current limiting is the purpose of through to the concurrent access/request for speed or a request within the time window for the speed limit to protect the system, once reached limit rate could be denial of service (directed to the error page or inform resources without), line, or wait for (such as seconds kill, reviews, order), the drop (return out data or the default data, If the product details page inventory is available by default).

Algorithms for limiting traffic

There are three main types of traffic limiting algorithms: counter traffic limiting, leaky bucket and token bucket. The counter can be divided into fixed time window current limiting and sliding time window current limiting.

Counter can also perform rough current limiting algorithm – Counter (fixed window)

A certain period of time is regarded as a window, in which there is a counter to record the number of times the window receives requests. Each time a request is received, the value of the counter will be increased by one. If the value of the counter is greater than the request threshold, the current limiting will start. When this period ends, the window’s counter data is initialized, which is equivalent to re-opening a window to re-monitor the number of requests. The client requests 100 times in 59 seconds of the first minute, and another 100 times in the first second of the second minute. Within two seconds, the back end receives 200 requests, resulting in a traffic spike

Current Limiting Algorithm – Counter (Sliding window)

The sliding window is actually a counter after subdivision. It subdivides each time window into several time segments. After each time segment, the whole time window will move one space to the right

The client will be denied access. The finer the time window is divided, the smoother the rolling of the sliding window will be, and the more accurate the effect of flow limiting will be

bucket

Bucket algorithm is similar to a speed limit water bucket, through a fixed size + FIFO queue time take the way of queue elements, after the request into the queue will be out of uniform processing (at the bottom of the barrel opening water at a constant speed), when the queue filled after subsequent requests will be refused to directly (water quickly spilled from the pail to)

The advantage is that the peak load can be cut and valley filled. No matter how many requests are made, they will only be sent to the back-end at a uniform speed without spikes, ensuring the normal operation of downstream services. The disadvantage is that requests in bucket queues will queue up and the response time will be long

The token bucket

The token bucket algorithm places tokens in the bucket at a constant rate (if the bucket is full, it is discarded), and every time a request comes in, it looks for tokens in the bucket. If there are tokens, it takes them away and continues processing. If there are no tokens, it rejects the request

The advantage of the token bucket is that it can deal with sudden traffic, and the request can respond quickly when there are tokens in the bucket, and there is no waiting time in the leaky bucket queue. The disadvantage is that compared with leaky bucket, the protection of downstream services is reduced to a certain extent

Current limiting solutions are mainly divided into single-machine current limiting and distributed current limiting. These two current limiting solutions have many specific implementations as follows:

Single current limiting

Java stand-alone stream limiting can be implemented using AtomicInteger, RateLimiter, or Semaphore

Distributed current limiting

Common solutions include Nginx limiting and Spring Cloud Gateway Sentinel

AtomicInteger private int maxCount = 10; private long interval = 60; Private AtomicInteger AtomicInteger = new AtomicInteger(0); private long startTime = System.currentTimeMillis(); public boolean limit(int maxCount, int interval) { atomicInteger.addAndGet(1); if (atomicInteger.get() == 1) { startTime = System.currentTimeMillis(); atomicInteger.addAndGet(1); return true; } // The interval is exceeded, If (system.currentTimemillis () -startTime > interval * 1000) {startTime = System.currentTimemillis (); atomicInteger.set(1); return true; If (atomicInteger.get() > maxCount) {return false; if (atomicInteger.get() > maxCount) {return false; } return true; }

Distributed traffic limiting Gateway Distributed traffic limiting is essentially a cluster concurrency problem. Redis single-process single-thread feature can naturally solve the distributed cluster concurrency problem. So a lot of distributed traffic limiting is based on Redis, such as Gateway, the Gateway component of Spring Cloud. The underlying implementation of Gateway is Redis + Lua.

Redis executes Lua scripts atomically, single-threaded, without executing other scripts or commands. In addition, once Redis starts executing the Lua script, it will continue to execute the script before performing other operations. Therefore, time-consuming operations cannot be performed in Lua scripts. Using Lua scripts also reduces interaction with Redis and the number of network requests.

The specific configuration of the Gateway in SpringCloud is as follows

Spring: redis: host: 127.0.0.1 port: 6379 Application: name: lagou-cloud-gateway Cloud: gateway: routes: -id: service-resume-router # we can define the route ID, keep the unique URI: http://127.0.0.1:8081 # predicates: Predicate: Route conditions, Predicate takes an input parameter and returns a Boolean result. The interface contains multiple tacit methods for grouping Predicate into other complex logic (for example, and, or, or not). – Path=/resume/** filters: – name: RequestRateLimiter args: redis-rate-limiter.replenishRate: 1 redis-rate-limiter.burstCapacity: 1 key-resolver: “#{@ipKeyResolver}”

If you are interested in the case of stream limiting, you can add wechat babadeerya520 to get relevant information.

Distributed Current-limiting Sentinel

Sentinel is a high-availability traffic protection component for distributed service architecture. It mainly takes traffic as the entry point and helps developers guarantee the stability of microservices from multiple dimensions such as traffic limiting, traffic shaping, fuse downgrading, system load protection and hotspot protection. Sentinal has a lot of powerful features, and these features will be covered in the course of This course. Of course, the course of this course is more than limited flow. You can consult the course syllabus of This course on the website of The education of Java. If you are interested in the above courses, you can add wechat babadeerya520 to get relevant materials.

Long press scan code 👆, lock 0 yuan quota for free access to the current limiting course materials! First 50 people only, first come, first served!Copy the code