Pay more attention to the taste of little sister public account.

Traffic limiting can be considered a degradation and is usually a threshold that is estimated in advance based on the background load (it can also be adjusted dynamically). Above this value, some bypassing is required. Depending on the service type, the response modes include direct rejection, delayed processing, hold wait, partial penetration, and default return.

Semaphores in the Concurrent package are widely used because they are simple to use and easy to understand. However, if you use the simple code shared online without serious testing, there is a movie you can watch: When trouble Hits.

Take a look at the simple code below. Acquire and Release are a pair of lovebirds with the same fate. We put Release in the Finally block carefully, and everything seems very harmonious. 2) Acquire parameter 5 means that 5 threads are allowed to process at the same time. 3) After each execution, output the specific execution time, plus the waiting time

SemaphoreLimiterBadChecker limiter = new SemaphoreLimiterBadChecker();
ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < 1000; i++) {
    executor.submit(() -> {
        while (true) { System.out.println(limiter.req()); }}); }Copy the code

The following is the result.

As you can see, although our interface takes only 100ms, the actual execution time is much longer, and there are no failures. Run a little longer, and you’ll find a large number of threads starving to death. Changing to fair locks does not improve the situation.

That’s the breakdown.

Here’s why. Resources on the Web side (such as Tomcat) are also limited. When our current limiter kicks in and actual concurrent requests exceed capacity, this thread blocking is passed down. The server may respond in the following processes: 1) Normal pressure, normal service, and normal time consumption. 2) The pressure rises, the service starts to have a large amount of timeouts, and occasionally there are normal time consuming demands due to unfair lock competition. 3) As the pressure continues to build, the server goes into suspended animation, barely able to accept new requests.

On the client side, there is no hint that the service can’t handle it, and there is no way to interrupt the request, and all the requests are going in circles. Increasing the number of connections and threads for Tomcat will not help much.

Change acquire to tryAcquire? Still doesn’t solve the problem. TryAcquire returns a bool and can proceed on failure, including the finally block. What’s the use?

if(! tryAcquire()){return TOO_MANY_REQUESTS;
}
Copy the code

One more judgment. That’s the way to go. TryAcquire can also add a timeout parameter so that it does not return an immediate failure or make the caller wait indefinitely, but rather restricts successful requests to a reasonable response time.

Response time = Timeout time + service processing timeCopy the code

To do this, in the case of Spring, you can obtain this license in preHandle and release it in postHandle; A timer can also be used to reproduce the semaphore at a certain frequency.

Of course you have to treat it differently. 1. Like the Web service mentioned above, you can simply refuse the service. Quick response is important, as some 2 seconds kill, orders, etc., can be solved through the queue or wait for (some) 3, like news consumption, etc., if there is no order requirements, I think, the infinite waiting also can be a good way to 4, for the majority of dispensable business results, use some default values returned directly, the effect will be better. Although it is limited current, but dry is the work of fuse

Users must be careful to distinguish.

End

Oddly enough, Java abstracts cyclicbarriers that are not very high (relatively) in usage scenarios, but there is no universal method for limiting traffic. Semaphores can simulate this process, but they are less friendly and error-prone. It is better to use Guava’s components for control (non-distributed), which we will discuss in a later article.

More:

No warm-up, no call high concurrency, call concurrency high

Such high availability, I do not want!

Balance, dangerous operation, save some luck for 996

JAVA multithreaded usage scenarios and precautions brief version