Redis can not only save the key/value data object, but also set the expiration time for this key. Have you ever thought about how Redis maintains these keys when setting the expiration time, and what is the difference between ordinary keys without expiration time? How does Redis remove keys when they expire? With these questions, let’s take a look

Redis database

Before we look at the key expiration mechanism, let’s take a look at the database in Redis. When using the Redis visualization tool, you can see many databases when you connect to Redis and switch databases when you execute the select command.

It can be seen that there are 16 databases, and each database is isolated from each other and does not affect each other, which is mainly caused by the design of Redis database.

  • Redis database structure

    struct redisServer {
	/ /...
    // An array that holds all the data groups in the server. Each object in the array is RedisDb
    redisDb *db;
    // The number of databases on the server, 16 by default
    int dbnum;
    / /...
}

// Each database object
typedef struct redisDb {
    / /...
    
    // Database key space, which holds all the key pairs in the database
    dict *dict;
    
    / /...
}
Copy the code

    The dict stored in the RedisDb object is a dictionary, which is called the key space.

    A key in a key space is a key in a database and is a string object.

    A value in a key space is a value in a database, and each value can be a string object, a hash object, a list object, a collection object, or an ordered collection object.

    When executing Redis commands, such as set, lpush, sadd, etc., a key is created in the key space of the database, and the value object of the response is saved in the key space. Therefore, the key/value of Redis is a dictionary in the saved structure of Redis.

Expired key mechanism

How do I set the expiration time of a key

  • The expire, pexpire

    Expire Key 5 // second PEXPIRE key 5000 // millisecondsCopy the code

    The expire command sets the expiration time for the key in the same way, but in different units. The expire command sets the expiration time in seconds and the pexpire command in milliseconds.

  • Expireat, pexpireat

    Expireat Key 1620225122 // second Precision PEXPireat Key 1620225121000 // millisecond precisionCopy the code

    Similar to the preceding command, both sets the expiration time of the key, but the only difference is that the preceding command sets the expiration time interval of the key, while the two commands set the expiration time of the key, when the timestamp is reached, it indicates that the key will be deleted.

The expire, pexpire, and expireat commands are all implemented using the pexpireat command, although there are different forms and time units to set the expiration time. Eventually Redis converts the set expiration time to a timestamp and sets the expiration time through pexpireat

How is the expiration time key saved

In redisDb, there is also a dictionary, the expiration dictionary, which is used to store keys with an expiration date. The expiration dictionary contains the expiration date of all keys in the current database

  • The structure of redisDb

    // Each database object
typedef struct redisDb {
    / /...

    // Expiration key dictionary, save the expiration time of the expiration key
    dict *expires;
    
    / /...
}
Copy the code

    The expired dictionary is also stored in redisDb. The key of the dictionary is the expired key, and the value of the dictionary is the expiration time corresponding to the key.

    Although save the two dictionaries, but there will be no waste of memory, because the overdue dictionary of key is a pointer, the pointer points to one of the key space key object (key) in the database dictionary, the overdue dictionary value is an integer, the integer keeps a key space key corresponding expiration time, is a timestamp.

Expiration key Deletion policy

We already know that the expiration date of database keys is stored in the expiration dictionary. How does Redis remove expired keys when the expiration date is reached? Here’s a look at the deletion strategy for expired keys

Redis offers three different deletion strategies:

  • Scheduled deletion (immediate deletion)

    Set a time limit for each key. When the key expires, the timer is executed immediately to delete the key

    • Advantages: Delete keys immediately and release memory
    • Disadvantages: It consumes a large amount of CPU. When there are many expired keys at the same time, a large number of deletion operations will be performed. If there are many client requests, the request processing speed will be slow

  • Lazy to delete

    When a key expires, the system does not delete it immediately. Instead, when the key is accessed, the system checks whether the key has expired. If it has expired, the system deletes it and returns NULL

    • Advantages: Does not occupy too much CPU, ensuring the execution efficiency of Redis
    • Disadvantages: A large number of expired keys are stored in memory, taking up extra memory, and if an expired key is no longer accessible, it will remain in memory forever, which can be considered a memory leak

  • Periodically delete

    Periodic deletion is a compromise between the first two strategies:

    Periodically delete A certain number of random keys (20 by default) are checked from the database at intervals (1 second by default) and expired keys are deleted. If more than 25% of the expired keys are scanned again immediately.

    Periodic deletion limits the duration and frequency of periodic deletion operations to prevent CPU usage

    • Advantages: Do not occupy too much CPU and prevent memory waste and other problems
    • Disadvantages: If the time and frequency of periodic deletion are not reasonable, periodic deletion will degenerate to periodic deletion, occupying too much CPU time
The delete strategy used by Redis

Based on the above 3 deletion strategies, Redis uses lazy deletion and periodic deletion. By combining these two strategies, the server can strike a balance between CPU and memory usage.

AOF, RDB, master/slave replication processing of expired keys

RDB

  • Generate the RDB file

    When a new RDB snapshot file is created using the SAVE or BGSAVE commands, the program checks the keys in the database, and expired keys are not saved to the RDB file

  • Load the RDB file

    When Redis is started, the RDB file is used to recover the data in memory. When the RDB file is loaded, Redis also checks the saved keys in the file. Unexpired keys are loaded into the database, and expired keys are ignored.

AOF

  • Write AOF

    The AOF file does not change when a key in the database has expired and has not been deleted. A DEL command is appended to the AOF file to indicate that the key has been deleted

  • Load the AOF

    Because the expiration key was appended to the AOF when it was deleted, the expiration key was then removed by the DEL command, although it could be loaded into memory normally.

  • AOF rewrite

    During AOF rewrite, the program checks the keys in the database and only writes the unexpired keys to the AOF file. Expired keys are not written to the AOF file. (Similar to RDB files)

A master-slave replication

To ensure data consistency between the primary and secondary servers, there is no expiration key on the secondary server, and the secondary server does not trigger the deletion policy

  • The main server when saving a expired keys, through the network links should be sent to the command from the server, but when the key expired in from the server, the server will not remove the key, only when the primary server to remove an expired keys, will be sent from the server a DEL instruction, tell from the server to delete the key date.

summary

  • Redis database is an array, stored inRedisServerIn the structure
  • The database is mainly composed ofdictandexpiresTwo dictionaries are composed of whichdictIs responsible for saving key-value pairs,expiresIt is mainly responsible for saving expired keys, and does not cause the memory waste of keys. Both dictionaries share a key object
  • Redis’s policy for deleting expired keys. Redis uses both lazy deletion and periodic deletion
  • When AOF deletes expired keys, a DEL command will be appended if the server deletes them. When AOF overwrites, it does not save expired key data to a file
  • When the RDB is generated and loaded, expired keys are not saved to the RDB file and are checked to determine whether the key is expired or not
  • In primary/secondary replication, the secondary server does not actively delete the expiration key, but sends the DEL command to the secondary server to delete the expiration key from the secondary server.

Did you Get it

Wechat public number fingertips on the code, welcome to pay attention to ~ learn together and progress together

Original is not easy, click a praise and then go ~ welcome to pay attention to, to bring you more wonderful articles!

Your likes and attention are the biggest motivation for writing articles