This article refers to the Redis interview question

Redis expiration Strategy and implementation principle

Redis expiration policy and implementation principle

describe

When we use Redis, we usually set an expiration time, of course, there is no expiration time, that is, never expire. When we set the expiration time, how Redis determines whether it is expired or not, and what policy is used to delete it.

Redis expiration time setting

grammar

#Setting expiration in seconds is the most common way
EXPIRE KEY time

#String unique way
SETEX KEY_NAME TIMEOUT VALUE
Copy the code

instructions

With the exception of the string’s own methods that set the expiration time, all methods rely on the EXPIRE method to set the time. If no time is set, the cache will never EXPIRE.

If you set an expiration time and then want the cache to never expire, use the Persist KEY.

Three expiration strategies

Time to delete

meaning

When you set the expiration time of a KEY, you can create a timer for the KEY and delete the KEY when the expiration time is up.

advantages

This method can ensure that the memory is released as soon as possible.

disadvantages

If there are a lot of expired keys, deleting these keys will take up a lot of CPU time. In the case of tight CPU time, the CPU cannot spend all of its time doing important things, but also needs to spend time deleting these keys.

Timer creation time. If a timer is created for each KEY whose expiration time is set (a large number of timers are generated), performance is seriously affected.

Lazy delete

meaning

If the KEY expires, the system does not delete it. Each time it obtains a value from the KEY, the system checks whether the value expires. If the value expires, the system deletes it and returns NULL.

advantages

The delete operation only happens when the KEY value is passed, and only the current KEY is deleted, so the CPU time is relatively small, and the deletion is already necessary (if the deletion is not done at this point, we will get the expired KEY).

disadvantages

If a large number of keys are not fetched for a long time after the timeout period, a memory leak (a large amount of memory is occupied by useless garbage) may occur.

Periodically delete

meaning

Delete expired keys at intervals.

advantages

You can limit the duration and frequency of delete operations to reduce CPU usage.

disadvantages

In terms of memory friendliness, it is not as good as “timed delete” (which takes up a certain amount of memory, but not as much memory as lazy delete), and in terms of CPU time friendliness, it is not as good as “timed delete” (which periodically compares and deletes operations, but is better than lazy delete).

The difficulty is to reasonably set the execution duration (how long does each deletion take) and the execution frequency (how long does every deletion take) (this depends on the server running condition). Too long or too high execution frequency will put pressure on CPU. After each periodic deletion operation is performed, it is necessary to record which flag bit the traversal cycle reached, so that the next periodic time comes, the cycle will start from the last position.

instructions

Memcached only uses lazy deletion, whereas Redis uses both lazy deletion and periodic deletion, which is another difference (and one that redis has over Memcached).

For lazy delete, it’s not only when you get the KEY that you check for expiration, it’s also when you set the KEY, like SETNEX, because SETNX is set when the KEY doesn’t exist, because if you don’t do the expiration check, So if you just set it up, it’s going to be the opposite of what we meant.

Timing task

How does a single-threaded Redis know to run a scheduled task?

Redis is a single-threaded thread that processes both scheduled tasks and client requests. Threads cannot block on scheduled tasks or client requests. How does Redis know when to run scheduled tasks?

Redis’s scheduled tasks are recorded in a data structure called the minimal heap. The fastest tasks in the heap are at the top of the heap. During each cycle, Redis immediately processes tasks that have reached a point in the minimum heap. After the processing is complete, record the remaining time for the fastest task. This time is the maximum time for processing client requests. If the time reaches the maximum time, the scheduled task is run instead of processing client requests.

configuration

Redis periodically deletes a unified timer. The default duration of the timer is 10. The specific configuration is as follows:

Increasing its value will take up more CPU, and of course redis will deal with many keys that expire at the same time more quickly and more accurately with timeouts. The Value of Hz ranges from 1 to 500. Generally, it is not recommended to exceed 100. You can increase the value to 100 only when the request latency is very low.

Expiration strategy adopted by Redis

instructions

Redis uses lazy delete + delete regularly.

Lazy delete process

  1. Before performing operations such as GET or SETNX, check whether the KEY is expired.
  2. If the KEY expires, delete it and perform related operations.
  3. If no, directly perform the corresponding operation.

Periodic Deletion process

In simple terms, each library of the specified N libraries is randomly deleted with less than or equal to the specified M expiration keys. The specific process is as follows:

  1. Iterate through each database (the number of “databases” specified in redis.conf, default is 16)
  2. Check the specified number of keys in the current library (default: 20 keys per library, note that the loop is executed 20 times, as described below)
  3. If none of the keys in the current library is set to expire, the next library is traversed randomly
  4. Obtain a KEY whose expiration time is set and check whether the KEY is expired. If so, delete the KEY to check whether the periodic deletion has reached the specified period. If so, exit the periodic deletion.

For periodic delets, there is a global variable current_db in the program to record the number of libraries to be traversed. Suppose there are 16 libraries, and we have traversed 10 of them on this periodic deletion, so the current_DB would be 11. The next periodic deletion would start on the 11th library. Assuming current_db is equal to 15, then the loop starts again at library 0 (where current_db==0).

More and more

Original link: link

Others: Contents

For more articles, you can search the public account: Hi Guest network IT tutorial