Redis Basics series 1 – Redis objects and encodings

Redis uses objects to represent key pairs in the database. When a key-value is set to a Redis database, the database creates at least two objects, one for the key (key object) of the key-value pair and one for the value (value object) of the key-value pair. Each object in Redis is represented by a redisObject structure with three properties related to saving data: type, Encoding, and PTR. There is also a RefCount attribute for object reference counting, an LRU attribute that records how long an object has been idling.

typedef struct redisObject {
    / / type
    unsigned type:4;

    / / code
    unsigned encoding:4;

    // A pointer to the underlying implementation data structure
    void *ptr;

    // Reference count
    int refcount;

    // Record the idling duration of the object
    unsigned lru;
}
Copy the code

type

The type attribute of an object records the type of an object. Redis has five object types: string object, list object, hash object, collection object, and ordered collection object. In Redis key-value pairs, the key is always a string object, and the value can be any of five object types.

coding

The Encoding property in the redisObject structure records the encoding of the object, which determines the underlying implementation data structure to which the PTR pointer points. Objects in Redis can be encoded in two or more ways, each of which has a corresponding data structure.

Encoding for different object types

Type check and code check

The commands used to operate keys in Redis can be basically divided into two types. One TYPE of command can execute any TYPE of keys, such as DEL command and TYPE command. The other is for specific object types, such as the GET command for strings and a type error if used for List.

When the server receives a command, it checks its type; It also selects the correct implementation code to execute the command based on how it is encoded. Type checking is performed according to the Type attribute of the redisObject structure, and encoding checking is performed according to the Encoding attribute. Both type checking and code checking are ways to implement polymorphic commands, the former being type-based polymorphism and the latter coding based polymorphism.

Memory reclamation is shared with objects

The whole life cycle of an object can be divided into three stages: creating an object, manipulating an object and releasing an object. The underlying implementation of Redis uses C language, but C language does not have automatic memory reclamation function, so Redis uses reference counting technology in its own object system to achieve the memory reclamation mechanism. The reference count information for an object is stored in the RefCount property of a redisObject. The reference count information for an object changes with the state of use of the object.

• When a new object is created, the reference count value is initialized to 1;
• When an object is used by a new program, its reference-count value is increased by one;
• When an object is no longer in use by a program, its reference count is reduced by one.
• When the reference count of an object becomes zero, the memory occupied by the object is freed.

In addition to being used to implement reference counting reclaim memory, the refcount attribute is also used to share string objects encoded by REDIS_ENCODING_INT. When creating a key-value pair of integer value type, the database checks for the existence of an equal string object, points the key to the existing string object, and increments the refcount of the object by one.

When Redis initializes the server, it creates 10,000 string objects containing all integer values from 0 to 9999. When the server needs these values, the server uses these shared objects.

Redis does not share objects that contain strings because verifying that strings are identical is too complex and takes up too much CPU resources.

Object idle time and data elimination strategy

The LRU attribute in the redisObject structure records the last operation time of the OBJECT. Using the OBJECT IDLETIME command, you can obtain the idling duration of the specified key. The idle time can serve the data obsolescence policy. When the memory occupied by the database is greater than maxMemory and maxmomery-Policy is set to volatile- LRU or allkeys-lRU, the data with the highest idle time will be deleted.

Elimination strategies in Redis

• Volatile – lRU (least recently used): indicates the algorithm that has been used least recently. The algorithm selects the key-value pair with the longest idle time from the key whose expiration time is set to clear it.
• Volatile – LFU (least frequently used): The algorithm that is least frequently used is used recently. From the keys whose expiration time is set, the key-value pair that is least frequently used in a certain period is deleted.
• Volatile – TTL: Deletes the value pair with the earliest expiration from the set key.
• Volatile -random: The key whose expiration time is set is randomly selected for elimination.
• Allkeys-lru: the least recently used algorithm, from all the keys to select the longest idle time key-value pair cleaning;
• Allkeys-lfu: select the least frequently used key pair from allkeys in a certain period of time to remove;
• Allkeys-random: deletes allkeys randomly.
• Any write operations will return errors when Redis memory limit is exceeded. However, all read operations work normally.