Writing in the front
In Internet applications, high concurrency systems will face a major challenge, that is, a large number of streams and high concurrency access, such as Tmall’s Double Eleven, Jingdong 618, Miaoxai, shopping promotion, etc., which are typical scenarios of high traffic and high concurrency. On seckill, you can refer to my other article “[High Concurrency] High concurrency seckill System architecture decryption, not all seckill is seckill!”
About the “Glacier Technology” wechat official account, unlock more “high concurrency” feature articles.
Note: Due to the length of the article, it is divided into three parts: theory, algorithm, practice (HTTP interface practice + distributed traffic limiting practice).
Theory: “[High Concurrency] how to achieve distributed traffic limiting under 100 million levels? These theories you must master!!”
Algorithm chapter: “[high concurrency] how to achieve a hundred million level of distributed traffic limiting? These algorithms you must master!!”
The project source has been submitted to github: github.com/sunshinelyz…
How to limit the HTTP interface traffic in the scenario of [High Concurrency] 100 million traffic? On the basis of this article, the construction of the project can be seen in the “[High Concurrency] Billion level traffic scenario how to limit HTTP interface traffic?! The content of the article. You can follow the official wechat account of Glacier Technology to read the above article.
The traffic limiting scheme introduced in the previous section has one defect: it is not global or distributed, so it cannot cope with the impact of large traffic in distributed scenarios. So, next, we will introduce how to realize distributed traffic limiting under 100 million traffic.
The key of distributed traffic limiting is to make the traffic limiting service global and unified. It can be implemented using Redis+Lua technology, through which high concurrency and high performance traffic limiting can be achieved.
Lua is a lightweight scripting language. It is an open source script written in the standard C language. It is designed to be embedded into applications and provide flexible extension and customization functions for applications.
Redis+Lua script to achieve distributed limiting ideas
We can use the way of Redia+Lua script to our distributed system for a unified full limited stream, Redis+Lua implementation of Lua script:
local key = KEYS[1] -- Limiting KEY(one per second)
local limit = tonumber(ARGV[1]) -- Current limiting size
local current = tonumber(redis.call('get', key) or "0")
if current + 1 > limit then -- If the traffic limit is exceeded
return 0
else -- Number of requests +1 and set expiration to 2 seconds
redis.call("INCRBY", key, "1")
redis.call("expire", key "2")
return 1
end
Copy the codeWe can understand the above Lua script code as follows.
(1) In Lua script, there are two global variables used to receive the KEYS and other parameters passed by the Redis application, respectively: KEYS, ARGV;
(2) It is an array list when KEYS are passed in the application side, and the values in the array are obtained by index subscript in Lua script.
ARGV can be one or more independent parameters. However, ARGV is received in Lua script and obtained by array subscript.
(4) The above operation is in a Lua script, and because I am currently using Redis version 5.0 (Redis 6.0 supports multi-threading), the execution of the request is single thread, therefore, Redis+Lua processing is thread-safe, and atomic.
If an operation is indivisible and multithreaded safe, it is called an atomic operation.
Next, we can use the following Java code to determine if we need to limit the flow.
//List KEYS for Lua [1]
String key = "ip:" + System.currentTimeMillis() / 1000;
List<String> keyList = Lists.newArrayList(key);
//List sets Lua ARGV[1]
List<String> argvList = Lists.newArrayList(String.valueOf(value));
// Call the Lua script and execute it
List result = stringRedisTemplate.execute(redisScript, keyList, argvList)
Copy the codeAt this point, we briefly introduced the overall idea of using Redis+Lua script to achieve distributed flow limiting, and gave the core code of Lua script and the core code of Java program to call Lua script. Next, we will start to write a distributed traffic limiting case using Redis+Lua script.
Redis+Lua script to implement distributed traffic limiting cases
How to limit the HTTP interface traffic in the scenario of [High Concurrency] 100 million traffic? The implementation method in this paper is similar, which is also to achieve traffic limiting in distributed and heavy traffic scenarios through the form of custom annotations, but here we use Redis+Lua script to achieve the global unified traffic limiting mode. Let’s implement this case manually together.
Create annotation
First, we define an annotation in the project named MyRedisLimiter, as shown below.
package io.mykit.limiter.annotation;
import org.springframework.core.annotation.AliasFor;
import java.lang.annotation.*;
/ * * *@author binghe
* @version 1.0.0
* @descriptionCustom annotations implement distributed limiting */
@Target(value = ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface MyRedisLimiter {
@AliasFor("limit")
double value(a) default Double.MAX_VALUE;
double limit(a) default Double.MAX_VALUE;
}
Copy the codeInside the MyRedisLimiter annotation, we have alias limit for the value attribute, so when we actually use the @MyRedislimiter annotation, we can use @MyRedislimiter (10), You can also use @myredislimiter (value=10) or @myRedislimiter (limit=10).
Create the section class
Once the annotation is created, we will create a section class MyRedisLimiterAspect. MyRedisLimiterAspect’s main purpose is to parse the @MyReDislimiter annotation and enforce the rules for limiting the flow. Instead of performing specific limiting logic for each method that needs limiting, we simply add the @MyRedislimiter annotation to the method that needs limiting, as shown below.
package io.mykit.limiter.aspect;
import com.google.common.collect.Lists;
import io.mykit.limiter.annotation.MyRedisLimiter;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.scripting.support.ResourceScriptSource;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import javax.servlet.http.HttpServletResponse;
import java.io.PrintWriter;
import java.util.List;
/ * * *@author binghe
* @version 1.0.0
* @descriptionMyRedisLimiter annotation section class */
@Aspect
@Component
public class MyRedisLimiterAspect {
private final Logger logger = LoggerFactory.getLogger(MyRedisLimiter.class);
@Autowired
private HttpServletResponse response;
@Autowired
private StringRedisTemplate stringRedisTemplate;
private DefaultRedisScript<List> redisScript;
@PostConstruct
public void init(a){
redisScript = new DefaultRedisScript<List>();
redisScript.setResultType(List.class);
redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource(("limit.lua"))));
}
@Pointcut("execution(public * io.mykit.limiter.controller.*.*(..) )"
public void pointcut(a){}@Around("pointcut()")
public Object process(ProceedingJoinPoint proceedingJoinPoint) throws Throwable{
MethodSignature signature = (MethodSignature) proceedingJoinPoint.getSignature();
// Use reflection to get the MyRedisLimiter annotation
MyRedisLimiter myRedisLimiter = signature.getMethod().getDeclaredAnnotation(MyRedisLimiter.class);
if(myRedisLimiter == null) {// Normal execution method
return proceedingJoinPoint.proceed();
}
// Get the parameter on the annotation to get the configured rate
double value = myRedisLimiter.value();
//List KEYS for Lua [1]
String key = "ip:" + System.currentTimeMillis() / 1000;
List<String> keyList = Lists.newArrayList(key);
//List sets Lua ARGV[1]
List<String> argvList = Lists.newArrayList(String.valueOf(value));
// Call the Lua script and execute it
List result = stringRedisTemplate.execute(redisScript, keyList, String.valueOf(value));
logger.info("Lua script execution result:" + result);
// The Lua script returns 0 to indicate that the traffic size is exceeded, and 1 to indicate that the traffic size is not exceeded.
if("0".equals(result.get(0).toString())){
fullBack();
return null;
}
// Get the token and proceed
return proceedingJoinPoint.proceed();
}
private void fullBack(a) {
response.setHeader("Content-Type" ,"text/html; charset=UTF8");
PrintWriter writer = null;
try{
writer = response.getWriter();
writer.println("Rollback failed, please read later...");
writer.flush();
}catch (Exception e){
e.printStackTrace();
}finally {
if(writer ! =null){ writer.close(); }}}}Copy the codeThe above code reads the limite. lua script file in the project classpath directory to determine whether to perform flow limiting. If the result of calling limit.lua returns 0, the flow limiting logic is performed. Since we need to use Lua scripts in our project, we need to create Lua scripts in our project.
Create limiter. Lua script file
Create the limite. lua script file in your project’s classpath directory, as shown below.
localKey = KEYS[1] -- Limiting key (one per second)local limit= tonumber(ARGV[1]) -- limiting the current sizelocal current = tonumber(redis.call('get', key) or "0")
if current + 1 > limit then-- If the traffic limit is exceededreturn 0
else-- Requests +1 and set expiration to 2 seconds redis. Call ("INCRBY", key, "1")
redis.call("expire", key "2")
return 1
end
Copy the codeThe limite. lua script file is fairly simple and won’t be covered here.
Adding annotations to interfaces
Annotation classes, aspect classes to parse annotations, and Lua script files are all ready. So, next, we add the @MyRedislimiter annotation to the sendMessage2() method in the PayController class and set the limit property to 10, as shown below.
@MyRedisLimiter(limit = 10)
@RequestMapping("/boot/send/message2")
public String sendMessage2(a){
// Record the return interface
String result = "";
boolean flag = messageService.sendMessage("Congratulations on your +1 growth.");
if (flag){
result = "SMS sent successfully!";
return result;
}
result = "Oops, the server is distracted. Please try again.";
return result;
}
Copy the codeHere, we limit the sendMessage2() method to a maximum of 10 requests per second. Then. Next, we’ll test sendMessage2() using JMeter.
Test distributed traffic limiting
At this point, we use JMeter to pressure the interface. In this case, we have configured the number of threads to be 50, which means that 50 threads will access the interface we wrote at the same time. The JMeter configuration is shown below.
Save and run Jemeter, as shown below.
After running, let’s take a look at the JMeter test results, as shown below.
As can be seen from the test results, during the test, some interface visits returned “Oh, the server is distracted, please try again”, indicating that the interface was restricted. Later, some interfaces successfully return the message “SMS sent successfully! The words. This is because we set up the interface to accept a maximum of 10 requests per second. When accessing the interface in the first second, the previous 10 successful requests return “SMS sent successfully!” “, followed by “Oops, the server is distracted, please try again”. The next request returns “SMS sent successfully!” “, indicating that the following request is already the interface called in the second.
We use Redis+Lua script to achieve the flow limiting mode, Java programs can be clustered deployment, this way to achieve the global unified flow limiting, no matter which node the client visits in the cluster, will count the access and achieve the final flow limiting effect.
This idea is a bit like distributed lock, you can pay attention to [Ice Technology] wechat public account reading I wrote a “[high concurrency] high concurrency distributed lock architecture decryption, not all locks are distributed lock!! For an in-depth understanding of how to implement truly thread-safe distributed locks, this article takes a step-by-step in-depth look at the various pits and solutions involved in implementing distributed locks so that you really understand what distributed locks are.
Nginx+Lua implements distributed traffic limiting
Nginx+Lua implements distributed traffic limiting, which is usually used at the entrance of the application, that is, to limit the traffic at the entrance of the system. Here, we also illustrate how to use Nginx+Lua to implement distributed limiting in the form of a practical example.
First, we need to create a Lua script with the contents of the script file shown below.
local locks = require "resty.lock"
local function acquire(a)
local lock =locks:new("locks")
local elapsed, err =lock:lock("limit_key") - the mutex
local limit_counter =ngx.shared.limit_counter - counter
local key = "ip:".os.time(a)local limit = 5 -- Current limiting size
local current =limit_counter:get(key)
if current ~= nil and current + 1> limit then -- If the traffic limit is exceeded
lock:unlock()
return 0
end
if current == nil then
limit_counter:set(key, 1.1) - Set the expiration time for the first time. Set the key value to 1 and the expiration time to 1 second
else
limit_counter:incr(key, 1) Add 1 at the second start
end
lock:unlock()
return 1
end
ngx.print(acquire())
Copy the codeIn the implementation, we need to use the lua-resty-lock mutex module to solve the atomicity problem (consider lock timeouts when using it in real engineering), and use the ngx.shared.DICT shared dictionary to implement the counters. Return 0 if limiting is required, 1 otherwise. To use it, you need to define two shared dictionaries (one for lock and one for counter data).
Next, you need to define the data dictionary in Nginx’s nginx.conf configuration file, as shown below.
HTTP {... lua_shared_dict locks 10m; lua_shared_dict limit_counter 10m; }Copy the codeThe soul of torture
Can Redis or Nginx handle the high number of concurrent applications?
It can be said that Both Redis and Nginx are basically high-performance Internet components that have no problem with the high concurrent traffic of the average Internet company. Why do you say that? Let’s move on.
If your application has really heavy traffic, you can fragment distributed traffic limiting using consistent hashing or degrade traffic limiting to application-level traffic limiting. There are also many solutions, which can be adjusted according to the actual situation. Using Redis+Lua for traffic limiting can reach the traffic limiting of hundreds of millions of high concurrent traffic (the author experienced it personally).
It should be noted that in the face of high concurrency systems, especially those with tens of millions or hundreds of millions of flows, we can not only limit the flow, but also add some other measures.
For distributed traffic limiting, the current scenario is traffic limiting on services rather than traffic limiting on traffic intakes. Traffic limiting at traffic inlets should be done at the access layer.
For the second kill scene, you can limit the flow at the flow entrance, friends can pay attention to [Glacier Technology] wechat public account, to read my “[High concurrency] high concurrency second kill system architecture decryption, not all second kill is second kill!” How to construct a high concurrency SEC kill system
Big welfare
Follow the wechat official account of “Glacier Technology” and reply the keyword “Design Mode” in the background to get the PDF document of “23 Design Modes in Simple Java”. Return keyword “Java8” to obtain the Java8 new features tutorial PDF. “Distributed traffic limiting solutions under 100 million levels of traffic” PDF document, the three PDFS are created by the glacial and collated super core tutorial, interview essential!!
Ok, that’s enough for today! Don’t forget to click a like, to see and forward, let more people see, learn together, progress together!!
Write in the last
If you think glacier wrote good, please search and pay attention to “glacier Technology” wechat public number, learn with glacier high concurrency, distributed, micro services, big data, Internet and cloud native technology, “glacier technology” wechat public number updated a large number of technical topics, each technical article is full of dry goods! Many readers have successfully moved to Dachang by reading articles on the “Glacier Technology” wechat official account; There are also many readers to achieve a technological leap, become the company’s technical backbone! If you also want to like them to improve their ability to achieve a leap in technical ability, into the big factory, promotion and salary, then pay attention to the “Glacier Technology” wechat public account, update the super core technology every day dry goods, so that you no longer confused about how to improve technical ability!
