The cause of
In the afternoon, I received the operation and maintenance feedback, production Redis had a command request of executing keys that was too slow, it took two or three seconds to respond to the command involved as follows: keys TTL_600 ::findHeadFootData-15349232-*-head
What is the keys command?
Keys official documentation www.redis.cn/commands/ke…
KEYS pattern
Find all keys that match the given pattern (regular expression). The time complexity is O(N), where N is the number of keys in the database. For example, Redis takes 40 milliseconds to execute a query in a database with 1 million keys.
Warning: KEYS are very fast, but using them in a large database can still cause performance problems. If you need to find specific KEYS from a dataset, you are better off using Redis’s collection structure SETS instead.
Supported regular expression modes:
h? Llo matches Hello, hallo, and hXlLO h* lLO matches hlLO and Heeeello H [AE] LLO matches Hello and Hallo, but does not match Hillo h[^e] lLO matches hallo, hBLlo… But does not match Hello H [a-b] lLO matches hallO and HBLLO
The return value
All eligible keys
Redis provides all API operations, relative to the server side are executed one by one, commands are executed one after another, there is no parallel execution.
Although REDIS executes KEYS quickly, with nearly six million KEYS in production, it takes two or three seconds for KEYS to block, which can be disastrous in production.
Redis provides a scan command
Official document of the scan command www.redis.cn/commands/sc…
SCAN cursor [MATCH pattern] [COUNT count]
Cursor [MATCH pattern] String to be matched count Number of scanned keys
The SCAN command and its related SSCAN, HSCAN, and ZSCAN commands are used to incrementally iterate over a collection element.
- The SCAN command is used to iterate over the key set in the current database.
- The SSCAN command is used to iterate over elements in a SET.
- The HSCAN command is used to iterate over key/value pairs in Hash.
- The ZSCAN command is used to iterate over the elements and their scores in the SortSet collection
chestnuts
Redis 127.0.0.1:6379> scan 0 MATCH *11* 1) "288" 2) 1) "key:911" redis 127.0.0.1:6379> Scan 288 MATCH *11* 1) "224" 2) (empty list or set) redis 127.0.0.1:6379> scan 224 MATCH *11* 1) "80" 2) (empty list or set) redis 127.0.0.1:6379> scan 80 MATCH *11* 1) "176" 2) (empty list or set) redis 127.0.0.1:6379> Scan 176 MATCH *11* COUNT 1000 1) "0" 2) 1) "key:611" 2) "key:711" 3) "key:118" 4) "key:117" 5) "key:311" 6) "key:112" 7) "key:111" 8) "key:110" 9) "key:113" 10) "key:211" 11) "key:411" 12) "key:115" 13) "key:116" 14) "key:114" 15) "key:119" 16) "key:811" 17) "key:511" 18) "key:11" Redis 127.0.0.1:6379 >Copy the codeIn the example above, the first iteration uses 0 as a cursor to indicate the start of a new iteration. The second iteration uses the cursor 17 returned from the first iteration as the new iteration parameter.
Obviously, the SCAN command returns an array of two elements, the first array element is the new cursor for the next iteration, and the second array element is an array containing all the iterated elements.
On the second call to SCAN, the command returns a cursor 0, indicating that the iteration has ended and the entire data set has been fully traversed.
Start a new iteration with 0 as the cursor and keep calling the SCAN command until the command returns 0. This process is called a complete walk.
Java code to practice
The keys and scan commands have been described above, so let’s use jedis to test the local code:
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.3</version>
</dependency>
<!--hutool工具类的jar包不是必须的，我是用习惯这个工具类了，你们有其他工具类直接去掉hutool即可-->
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.6.5</version>
</dependency>
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.date.DateUtil;
import cn.hutool.core.date.TimeInterval;
import cn.hutool.core.thread.ExecutorBuilder;
import lombok.extern.slf4j.Slf4j;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.ScanParams;
import redis.clients.jedis.ScanResult;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.atomic.AtomicInteger;
@Slf4j
public class JedisService {
public static Jedis getJedis(){
return new Jedis("localhost",6379,1000000000);
}
volatile static Boolean testCycleSetExitFlag = Boolean.FALSE;
/**
* 初始化数据到redis
*/
public static void initData() throws InterruptedException {
TimeInterval timeInterval = DateUtil.timer();
log.info("initData start");
getJedis().set("111CCCCCCCCCCCCCC1111","1");//先把查找的key设置进去
//初始化数据个数
AtomicInteger count = new AtomicInteger(10000*2000);
//开100个客户端去执行set命令
Integer totalClientNum = 100;
ExecutorService executor = ExecutorBuilder.create()
.setCorePoolSize(10)
.setMaxPoolSize(50)
.setWorkQueue(new LinkedBlockingQueue<>(1024))
.build();
CountDownLatch countDownLatch = new CountDownLatch(totalClientNum);
for (int i = 0; i < totalClientNum; i++) {
executor.submit(()->{
Jedis jedis = getJedis();
while (true){
Integer crrentCount = count.decrementAndGet();
if(crrentCount<=0){
break;
}
if(crrentCount%10000==0){
log.info(" 设置key的数量还剩:{} 已耗时:{}毫秒",crrentCount,timeInterval.interval());
}
jedis.set(UUID.randomUUID().toString().replaceAll("-",""),"1");
}
countDownLatch.countDown();
});
}
countDownLatch.await();
log.info("initData end 总耗时：{}毫秒",timeInterval.interval());
executor.shutdown();
}
public static void main(String[] args) throws Exception {
Jedis jedis = getJedis();
System.out.println("服务正在运行: "+jedis.ping());
// initData();//初始化数据
log.info("现在redis服务有{}个key",jedis.dbSize());
String pattern = "*CCCCCCCCCCCCCC*";
//第一步：先验证一下keys和scan的执行效率
keys(pattern);
scan(pattern);
log.info("----------------我是一条分割线---------------");
//第二步:来验证一下keys和scan命令是否会阻塞其它命令
new Thread(()->{
testCycleSet("1111","11");
}).start();
Thread.sleep(2000);//休眠两秒
keys(pattern);
scan(pattern);
testCycleSetExitFlag = true;
log.info("----------------我还是一条分割线---------------");
//下面测试一下设置不同count的效率
scan(pattern,100);
scan(pattern,200);
scan(pattern,300);
scan(pattern,400);
scan(pattern,500);
scan(pattern,1000);
scan(pattern,2000);
scan(pattern,5000);
scan(pattern,10000);
}
//循环设置一个key值 测试用的
public static void testCycleSet(String key,String value){
Jedis jedis = getJedis();
while (true){
if(testCycleSetExitFlag){
break;
}
Long startTime = System.currentTimeMillis();
log.info("testCycleSet key：{} value:{} start",key,value);
try {
jedis.set(key,value);
}catch (Exception e){
log.error("testCycleSet",e);
}
log.info("testCycleSet key：{} value:{} end 耗时:{}毫秒",key,value,(System.currentTimeMillis() - startTime));
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public static Set<String> keys(String pattern){
Long startTime = System.currentTimeMillis();
log.info("jedis keys start 匹配的key:{}",pattern);
Set<String> keySet = getJedis().keys(pattern);
log.info("jedis keys end 匹配的key:{} 结果集大小:{} 耗时:{}毫秒 ",pattern,keySet.size(),(System.currentTimeMillis() - startTime));
return keySet;
}
public static Set<String> scan(String pattern){
return scan(pattern,300);
}
public static Set<String> scan(String pattern,int count){
Long startTime = System.currentTimeMillis();
Jedis jedis = getJedis();
log.info("jedis scan start 匹配的key:{} 每次遍历{}个key",pattern,count);
String index = "0";
Set<String> keySet = new HashSet<String>();//匹配到的结果集
Integer scanNum = 0;
try {
ScanParams params = new ScanParams();
params.match(pattern);
params.count(count);
while (true){
ScanResult<String> scanResult = jedis.scan(index,params);
index = scanResult.getStringCursor();//下标重新赋值
List<String> result = scanResult.getResult();//扫描到的key值
if(CollUtil.isNotEmpty(result)){
keySet.addAll(result);//扫描到的key放到Set
}
if("0".equals(index)){
break;
}
scanNum++;
}
} catch (Exception e) {
log.info("redis异常" + e.getMessage());
}
log.info("jedis scan end 匹配的key:{} 每次遍历{}个key scan执行了:{}次 结果集大小:{} 总耗时:{}毫秒",pattern,count,scanNum,keySet.size(),(System.currentTimeMillis() - startTime));
return keySet;
}
}
服务正在运行: PONG
00:34:30.776 [main] INFO com.test.JedisService - 现在redis服务有25000000个key
00:34:30.780 [main] INFO com.test.JedisService - jedis keys start 匹配的key:*CCCCCCCCCCCCCC*
00:34:41.493 [main] INFO com.test.JedisService - jedis keys end 匹配的key:*CCCCCCCCCCCCCC* 结果集大小:1 耗时:10713毫秒
00:34:41.493 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key
00:35:09.307 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key scan执行了:83231次 结果集大小:1 总耗时:27814毫秒
00:35:09.307 [main] INFO com.test.JedisService - ----------------我是一条分割线---------------
00:35:09.352 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:09.355 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:3毫秒
00:35:10.358 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:10.359 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:11.356 [main] INFO com.test.JedisService - jedis keys start 匹配的key:*CCCCCCCCCCCCCC*
00:35:11.360 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:22.724 [main] INFO com.test.JedisService - jedis keys end 匹配的key:*CCCCCCCCCCCCCC* 结果集大小:1 耗时:11368毫秒
00:35:22.725 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:11365毫秒
00:35:22.725 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key
00:35:23.728 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:23.729 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:24.734 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:24.735 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:25.739 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:25.740 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:26.744 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:26.744 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:27.747 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:27.747 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:28.751 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:28.751 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:29.755 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:29.757 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:30.761 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:30.761 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:31.766 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:31.767 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:32.767 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:32.768 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:33.772 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:33.773 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:34.776 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:34.776 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:35.779 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:35.780 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:36.784 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:36.785 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:37.788 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:37.788 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:38.793 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:38.793 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:39.796 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:39.796 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:40.797 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:40.797 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:41.800 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:41.801 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:1毫秒
00:35:42.806 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:42.806 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:43.811 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:43.811 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:44.816 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:44.816 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:45.821 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:45.821 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:46.826 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:46.826 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:47.830 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:47.830 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:48.835 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:48.835 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:49.840 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:49.840 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:50.845 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 start
00:35:50.845 [Thread-0] INFO com.test.JedisService - testCycleSet key：1111 value:11 end 耗时:0毫秒
00:35:51.225 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key scan执行了:83231次 结果集大小:1 总耗时:28500毫秒
00:35:51.225 [main] INFO com.test.JedisService - ----------------我还是一条分割线---------------
00:35:51.226 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历100个key
00:36:24.747 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历100个key scan执行了:249070次 结果集大小:1 总耗时:33522毫秒
00:36:24.747 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历200个key
00:36:54.580 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历200个key scan执行了:124768次 结果集大小:1 总耗时:29833毫秒
00:36:54.580 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key
00:37:23.674 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key scan执行了:83231次 结果集大小:1 总耗时:29094毫秒
00:37:23.674 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历400个key
00:37:56.541 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历400个key scan执行了:62441次 结果集大小:1 总耗时:32867毫秒
00:37:56.542 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历500个key
00:38:28.282 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历500个key scan执行了:49962次 结果集大小:1 总耗时:31740毫秒
00:38:28.282 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历1000个key
00:39:00.211 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历1000个key scan执行了:24990次 结果集大小:1 总耗时:31929毫秒
00:39:00.211 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历2000个key
00:39:31.754 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历2000个key scan执行了:12497次 结果集大小:1 总耗时:31543毫秒
00:39:31.754 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历5000个key
00:40:03.548 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历5000个key scan执行了:4999次 结果集大小:1 总耗时:31794毫秒
00:40:03.548 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历10000个key
00:40:36.329 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历10000个key scan执行了:2499次 结果集大小:1 总耗时:32781毫秒
这是500万key 不同count执行的结果
服务正在运行: PONG
01:23:29.449 [main] INFO com.test.JedisService - 现在redis服务有5000000个key
01:23:29.452 [main] INFO com.test.JedisService - jedis keys start 匹配的key:*CCCCCCCCCCCCCC*
01:23:31.172 [main] INFO com.test.JedisService - jedis keys end 匹配的key:*CCCCCCCCCCCCCC* 结果集大小:1 耗时:1720毫秒
01:23:31.172 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key
01:23:36.587 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key scan执行了:16649次 结果集大小:1 总耗时:5415毫秒
01:23:36.587 [main] INFO com.test.JedisService - ----------------我还是一条分割线---------------
01:23:36.587 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历100个key
01:23:43.030 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历100个key scan执行了:49851次 结果集大小:1 总耗时:6443毫秒
01:23:43.031 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历200个key
01:23:48.801 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历200个key scan执行了:24963次 结果集大小:1 总耗时:5769毫秒
01:23:48.801 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key
01:23:54.355 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历300个key scan执行了:16649次 结果集大小:1 总耗时:5554毫秒
01:23:54.355 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历400个key
01:23:59.711 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历400个key scan执行了:12490次 结果集大小:1 总耗时:5356毫秒
01:23:59.711 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历500个key
01:24:05.016 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历500个key scan执行了:9994次 结果集大小:1 总耗时:5305毫秒
01:24:05.016 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历1000个key
01:24:11.594 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历1000个key scan执行了:4998次 结果集大小:1 总耗时:6578毫秒
01:24:11.594 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历2000个key
01:24:18.137 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历2000个key scan执行了:2499次 结果集大小:1 总耗时:6543毫秒
01:24:18.137 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历5000个key
01:24:24.655 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历5000个key scan执行了:999次 结果集大小:1 总耗时:6518毫秒
01:24:24.655 [main] INFO com.test.JedisService - jedis scan start 匹配的key:*CCCCCCCCCCCCCC* 每次遍历10000个key
01:24:31.257 [main] INFO com.test.JedisService - jedis scan end 匹配的key:*CCCCCCCCCCCCCC* 每次遍历10000个key scan执行了:499次 结果集大小:1 总耗时:6602毫秒
Copy the codesummary
Do keys and scan block other commands?
If you look at the log, you can see that the keys command blocks other commands. The scan command is equivalent to iterating all keys in batches, iterating some keys, and returning the location (cursor) to the client. Next time the client holds the cursor returned last time, the scan command continues to iterate until the traversal is complete. So it does not block Redis for long.
The scan command count sets the appropriate size
The number of keys on our Redis server is usually in the range of five to seven million
After the above tests:
When the scan command count is set to 300, a key of 25 million will perform 80,000 scans, which takes 30 seconds to complete. A key of 5 million will perform 16,649 scans per second, which takes 6 seconds in total. Approximately 3000 executions per second and redis QPS on production can easily carry 10W, so 300 would be appropriate.
In the end
Local self-test passed, sent to grayscale environment, test feedback business no abnormality, sent to the production observed a few days of operation and maintenance no bad feedback, problem solved ~