1. Delete big key

Bigkey (bigkey) refers to that the value of key is huge, such as Hashes, Sorted Sets, Lists and Sets. After accumulating over a long period of time, it will become very large, maybe tens or hundreds of MB, or even GB.

If you run the del command to delete such large keys, they will block for a long time or even crash.

Because the time complexity of del command is O(M) when deleting set type data, M is the number of elements in the set.

Redis is single-threaded, and if a single command takes too long to execute, it will block other commands, causing an avalanche.

Ali also stipulated in the internal Redis usage specification:

Do not use DEL to delete bigkeys that are not strings. Use HSCAN, SSCAN, or ZSCAN to gradually delete bigkeys

Second, solutions

Reliable scheme:

  • Progressive deletion
  • UNLINK (after Version 4.0)

1. Progressive deletion

Ideas:

Batch deletion: Run the scan command to traverse the large key, obtain a small number of elements each time, delete them, and then obtain and delete the next batch of elements.

Example:

  • Delete the large Hashes

Steps:

(1) The key is changed to a new name, which is equivalent to the logical deletion of the key. No redis command can access the key

(2) Delete small steps and multiple batches

Pseudo code:

# key namenewkey = "gc:hashes:" + redis.INCR( "gc:index" )redis.RENAME("my.hash.key", newkey)Delete 100 elements at a timecursor = 0loop  cursor, hash_keys = redis.HSCAN(newkey, cursor, "COUNT", 100)  if hash_keys count > 0    redis.HDEL(newkey, hash_keys)  end  if cursor == 0    break  endendCopy the code
  • Delete the large Lists

Pseudo code:

# key namenewkey = "gc:hashes:" + redis.INCR("gc:index")redis.RENAME("my.list.key", newkey)# removewhile redis.LLEN(newkey) > 0  redis.LTRIM(newkey, 0, -99)endCopy the code
  • Delete the large Sets.

Pseudo code:

# key namenewkey = "gc:hashes:" + redis.INCR("gc:index")redis.RENAME("my.set.key", newkey)Delete 100 members at a timecursor = 0loop  cursor, members = redis.SSCAN(newkey, cursor, "COUNT", 100)  if size of members > 0    redis.SREM(newkey, members)  end  if cursor == 0    break  endendCopy the code
  • Delete the large Sorted set

Pseudo code:

# key namenewkey = "gc:hashes:" + redis.INCR("gc:index")redis.RENAME("my.zset.key", newkey)# removewhile redis.ZCARD(newkey) > 0  redis.ZREMRANGEBYRANK(newkey, 0, 99)endCopy the code

2. UNLINK

Redis 4.0 introduced an important command UNLINK, to save the del delete large key dilemma.

UNLINK

Delete key from all namespaces and return it immediately without blocking.

(2) The background thread performs the real free space operation.

UNLINK can basically replace DEL, but some scenarios still need DEL. For example, UNLINK is not suitable for space accumulation because it does not immediately release space.

Third, summary

  • usedelDeleting a large key can cause long blocks or even crashes.
  • You can use progressive deletion, rightHashes, Sorted Sets, Lists, SetsDelete the key logically and rename the key so that the client cannot use the original key. Then delete the key in small batches.
  • Available after version 4.0UNLINKCommand, background thread to free space.