1 investigate knowledge points

There are many reasons for this.

  1. Keys and Scan
  2. Disadvantages of Keys queries
  3. How do I use Scan?
  4. Scan query features

2 Solution

  1. Keys query problem
  2. The use of the Scan
  3. The characteristics of the Scan

3 Keys usage related

1) The Keys are as follows

2) Problems existing in Keys

  1. This command has no paging function, so we can only query all the key values that meet the conditions at one time. If the query result is very large, the output information will be very large.
  2. The keys command is a traversal query, so its query time complexity is O (n), so the larger the amount of data, the longer the query time.

4 Scan Usage related

We first simulate massive data, add 10W data using Pipeline, Java code implementation is as follows:

import redis.clients.jedis.Jedis;
import redis.clients.jedis.Pipeline;
import utils.JedisUtils;

public class ScanExample {
    public static void main(String[] args) {
        // 添加 10w 条数据
        initData();
    }
    public static void initData(){
        Jedis jedis = JedisUtils.getJedis();
        Pipeline pipe = jedis.pipelined();
        for (int i = 1; i < 100001; i++) {
            pipe.set("user_token_" + i, "id"+ i); } // Execute the command pipe.sync(); System.out.println("Data insertion completed"); }}Copy the code

To query the data of user 9999*, use the Scan command as follows:

127.0.0.1:6379> scan 0 match user_token_9999* count 10000
1) "127064"
2) 1) "user_token_99997"
127.0.0.1:6379> scan 127064 match user_token_9999* count 10000
1) "1740"
2) 1) "user_token_9999"
127.0.0.1:6379> scan 1740 match user_token_9999* count 10000
1) "21298"
2) 1) "user_token_99996"127.0.0.1:6379> Scan 21298 match user_token_9999* count 10000"65382"
2) (empty list or set)
127.0.0.1:6379> scan 65382 match user_token_9999* count 10000
1) "78081"
2) 1) "user_token_99998"
   2) "user_token_99992"127.0.0.1:6379> Scan 78081 match user_token_9999* count 10000"3993"
2) 1) "user_token_99994"
   2) "user_token_99993"127.0.0.1:6379> Scan 3993 match user_token_9999* count 10000"13773"
2) 1) "user_token_99995"127.0.0.1:6379> Scan 13773 match user_token_9999* count 10000"47923"
2) (empty list or set)
127.0.0.1:6379> scan 47923 match user_token_9999* count 10000
1) "59751"
2) 1) "user_token_99990"
   2) "user_token_99991"
   3) "user_token_99999"127.0.0.1:6379> Scan 59751 match user_token_9999* count 10000"0"
2) (empty list or set)
Copy the code

From the above results, we can see two problems:

  1. The query result is empty, but the value of the cursor is not 0, indicating that the traversal is not finished.
  2. Count 10000 is set, but the number returned is not 10000 and is not fixed because count only defines the number of dictionary slots that the server traverses in a single pass (approximately equal to), not the value of count that returns the result.

Scan cursor [MATCH pattern] [COUNT COUNT]

Among them:

  • Cursor: integer value, starting from 0 and ending at 0. The query result is empty, but the cursor value is not 0, indicating that the traversal is not finished.
  • Match pattern: indicates the matching field.
  • Count: specifies the number of dictionary slots to be traversed by the server at a time (approximately equal to). This is only a hint of an incremental iteration command. It is not the maximum number of dictionary slots to be returned by the query.

5 Scan code Practice

In this paper, we use Java code to realize Scan query function, the code is as follows:

import redis.clients.jedis.Jedis; import redis.clients.jedis.Pipeline; import redis.clients.jedis.ScanParams; import redis.clients.jedis.ScanResult; import utils.JedisUtils; public class ScanExample { public static void main(String[] args) { Jedis jedis = JedisUtils.getJedis(); // define match and count parameters ScanParams params = new ScanParams(); params.count(10000); params.match("user_token_9999*"); // String cursor ="0";
        while (true) {
            ScanResult<String> res = jedis.scan(cursor, params);
            if (res.getCursor().equals("0") {// indicates the last itembreak; } cursor = res.getCursor(); // Set the cursorfor(String item: res.getresult ()) {// Print the query result.out.println (String item: res.getresult ())"Query result:"+ item); }}}}Copy the code

The execution results of the above procedures are as follows:

Query result: user_token_99997

Query result: user_token_9999

Query result: user_token_99996

Query result: user_token_99998

Query result: user_token_99992

Query result: user_token_99994

Query result: user_token_99993

Query result: user_token_99995

Query result: user_token_99990

Query result: user_token_99991

Query result: user_token_99999

6 summarizes

In this paper, we learned that Scan should be used to query certain data in massive data in Redis, and Scan has the following features:

  1. Scan can realize the matching function of keys.
  2. Scan is queried by cursor and does not cause Redis to fake death;
  3. Scan provides the count parameter, which specifies the number of traversals.
  4. Scan returns the cursor to the client, and the user client continues to traverse the query.
  5. Duplicate data may exist in the Scan result, which requires the client to delete the data.
  6. If a null value is returned and the cursor is not 0, the traversal is not complete.
  7. Scan guarantees that deleted elements will not be queried before retrieval begins;
  8. If any element is modified during iteration, Scan does not guarantee that the related element can be queried.

7 video version

Video content is as follows: www.bilibili.com/video/av880…

Follow the qr code below and subscribe for more exciting content.