Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.
This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.
1, the introduction of
Token bucket algorithm is relatively simple, it is like a lottery to buy a house, get the number of people eligible to buy, did not get the number can only wait for the next time (fortunately xiaobian does not need to lottery, because can not afford to buy! . In real development, the system maintains a container for storing tokens and adds tokens to the container at a fixed rate, which is usually a tradeoff against the system’s processing capacity. When a request is made by a client, the request is processed only after the client obtains a token from the token bucket. Otherwise, the request is denied. The key of token bucket flow limiting lies in the token issuing rate and token bucket capacity. \
There are many ways to realize the token bucket traffic limiting, this article describes the Redis based on Redis redis-cell traffic limiting module, which is provided by Redis for distributed systems, efficient, accurate traffic limiting method, is very widely used, and very simple!
2. Installation of Redis-Cell
Redis does not integrate the redis-cell module by default, just like Redis uses bloom filter, we also need to install and integrate the module.
2.1 GitHub source code & installation package
Redis-cell GitHub address:
Github.com/brandur/red…
Redis-cell is developed in Rust. If you don’t want to spend a lot of time in the Rust environment, you can download the installation package for your operating system. \
image.png
Download the corresponding installation package:
Github.com/brandur/red…
If you do not know your server (Linux) version, you can check before downloading the installation package:
Uname -a # Run the following command to check the version of the operating system: cat /proc/versionCopy the code
2.2 Installation & Exception Handling
- In the same directory as the Redis installation directory, create a folder redis-cell, upload the compressed package, and decompress it
Tar - ZXVF redis - cell - v0.2.5 powerpc64 - unknown - Linux - gnu. Tar. GzCopy the code- After decompressing the libredis_cell.so file, copy the directory of the libredis_cell.so file
- Edit the Redis configuration file, Redis. Conf, and save it before exiting
- Restart Redis. If the startup is normal, log in to the Redis client and check whether the mounted modules have Redis-cells by running the Module list
- Test command. If the following information is displayed, the Redis-Cell is successfully integrated
- If the client cannot be connected after the Redis is restarted, it indicates that the Redis startup fails. In this case, you need to view the Redis startup log. If the log file has been configured, you can directly view the log file to locate the fault. Restart the system again and check the error log output from the log file
- /lib64/libc.so.6: version ‘glibc_2.18’ not found
43767:M 08 Sep 2021 21:39:39.643 # Module /usr/local/soft/ redis-cell-0.3.0 /libredis_cell. So failed to load: /lib64/libc.so.6: Version 'glibc_2.18' not found (required by /usr/local/soft/ redis-cell-0.3.0 /libredis_cell.so) 43767:M 08 Sep 2021 21:39:39.643 # Can't load module from /usr/local/soft/redis-cell0.3.0 /libredis_cell.so: server abortingCopy the code- If glibc_2.18 is missing, install it (the last two compilations take a few minutes)
Yum install GCC wget http://ftp.gnu.org/gnu/glibc/glibc-2.18.tar.gz tar ZXF glibc 2.18, tar, gz CD/mkdir glibc 2.18 build cd build/ .. /configure --prefix=/usr make -j4 make installCopy the code- After the installation is complete, restart Redis, test whether the installation is successful, repeat the above process, and analyze the error through the log
CL.THROTTLE
Parameter Description Cl. THROTTLE Indicates the parameter
CL. THROTTLE liziba 10 5 60 1 bring bring bring bring bring | | | | └ ─ ─ ─ ─ ─ the apply 1 token (default if omitted) application a token () | | └ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ 5 Tokens / 60 seconds (60 seconds 5 token is added to the token bucket) | └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ 10 max_burst (the largest sudden request, Is not the maximum capacity of the token bucket └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ the key "liziba" (current-limiting key)Copy the codeOutput Parameter Value Meaning
127.0.0.1:6379> cl. Throttle liziba 10 5 60 11) (integer) 0 # Whether the current request is allowed, 0 indicates yes, 1 indicates no 2) (integer) 11 # Maximum token bucket capacity, Maximum number of tokens in the token bucket 3) (INTEGER) 10 # Current number of tokens in the token bucket 4) (INTEGER) -1 # How long will it take to retry if rejected, or -1 if currently allowed 5) (INTEGER) 12 # How long will it take for the token bucket to be fullCopy the codeThe only possible ambiguity here is max_burst, which is not the maximum size of the token bucket, as explained in the author’s readme.md
The total limit of the key (max_burst + 1). This is equivalent to the common X-RateLimit-Limit HTTP header.
4. Java invokes the Redis-Cell module to achieve stream limiting
4.1 Importing Dependencies
<dependency> <groupId> IO. Lettuce </groupId> <artifactId> crowd-core </artifactId> <version>5.3.4.RELEASE</version> <! > <exclusions> <groupId> io.Net </groupId> <artifactId>netty-buffer</artifactId> </exclusion> <exclusion> <groupId>io.netty</groupId> <artifactId>netty-common</artifactId> </exclusion> <exclusion> <groupId>io.netty</groupId> <artifactId>netty-codec</artifactId> </exclusion> <exclusion> <groupId>io.netty</groupId> <artifactId>netty-transport</artifactId> </exclusion> </exclusions> </dependency>Copy the code4.2 Implementation Code
Redis command interface definition:
package com.lizba.redis.limit.tokenbucket; import io.lettuce.core.dynamic.Commands; import io.lettuce.core.dynamic.annotation.Command; import java.util.List; / * * * < p > * Redis command interface definition * < / p > * * @ Author: Liziba * @ the Date: 2021/9/8 23:50 */ public interface IRedisCommand extends Commands {/** * defines a traffic limiting method ** @param key Traffic limiting key * @param maxBurst Param tokens Tokens and seconds Add tokens in seconds * @param seconds Tokens in seconds Add tokens * @return */ @command (" cl.throttle? 0? 1? 2? 3? 4") List<Object> throttle(String key, long maxBurst, long tokens, long seconds, long apply); }Copy the codeRedis-cell token bucket traffic limiting class definition:
package com.lizba.redis.limit.tokenbucket; import io.lettuce.core.RedisClient; import io.lettuce.core.api.StatefulRedisConnection; import io.lettuce.core.dynamic.RedisCommandFactory; import java.util.List; /** ** <p> * redis-cell token bucket limit * </p> ** @author: Liziba * @date: 2021/9/8 23:47 */ public class TokenBucketRateLimiter { private static final String SUCCESS = "0"; private RedisClient client; private StatefulRedisConnection<String, String> connection; private IRedisCommand command; public TokenBucketRateLimiter(RedisClient client) { this.client = client; this.connection = client.connect(); this.command = new RedisCommandFactory(connection).getCommands(IRedisCommand.class); } /** * Please allow @param key * @param maxBurst * @param tokens * @param seconds * @param apply * @return */ public boolean isActionAllowed(String key, long maxBurst, long tokens, long seconds, long apply) { List<Object> result = command.throttle(key, maxBurst, tokens, seconds, apply); if (result ! = null && result.size() > 0) { return SUCCESS.equals(result.get(0).toString()); } return false; }}Copy the codeTest code:
package com.lizba.redis.limit.tokenbucket; import io.lettuce.core.RedisClient; /** ** <p> * /** * <p> ** @author: liziba * @date: 2021/9/9 0:02 */ public class TestTokenBucketRateLimiter { public static void main(String[] args) { RedisClient client = RedisClient. Create (" redis: / / 192.168.211.108:6379 "); TokenBucketRateLimiter limiter = new TokenBucketRateLimiter(client); // cl.throttle liziba 10 5 60 1 for (int i = 1; i <= 15; i++) { boolean success = limiter.isActionAllowed("liziba", 10, 5, 60, 1); System.out.println(" "+ I +" "+ (success? "Success" : "failure ")); }}}Copy the codeTest results (also indicated here that the token bucket has max_burst + 1 capacity) :
The first request is successful. The second request is successful. The third request is successful. The fifth request is successful. The sixth request is successful Failed the 14th request Failed the 15th request failedCopy the code