【 Redis 】 Redis memory management, elimination mechanism, memory optimization

The article directories

• • Configure redis
  • Maximum memory limit
  • View information about Redis memory: INFO Memory
  • Where did the memory go? It exploded before it could be used properly
  • Memory reclamation policy
  • • Expired keys
    • • How does Redis eliminate expired keys
  • Memory removal control policy
  • • Eight solutions
    • How does the recycle process work
    • Approximate LRU algorithm
  • Memory optimization
  • • Use 32-bit Redis
    • Bit-level and word-level operations
    • Use hash tables whenever possible

Configure redis

If you want to run a memory-efficient Redis database, you first need to understand all the memory-related instructions in the redis.conf configuration file. The redis. Conf file provides rich inline documentation for most instructions, making some complex memory optimization options easy to understand, change, and test. Portal: redis. Conf translation and configuration of the memory management section

Most Redis configuration instructions can be SET at run time using the CONFIG SET command.

Maximum memory limit

For cache scenarios, when maxMemory is exceeded, delete strategies such as LRU are used to free space. Prevents the memory used from exceeding the server's physical memory.Copy the code

View information about Redis memory: INFO Memory

When memfragmentationRatio is greater than 1, it indicates that some memory is not used for data storage but is consumed by memory fragmentation. If the value is large, it indicates that the fragmentation rate is serious. When memfragmentationratio < 1, this situation generally occurs when the operating system swaps the Redis memory to the hard disk. Special attention should be paid to this situation. Because the speed of the hard disk is much slower than that of the memory, the Redis performance will become very poor, or even zombie.

When Redis runs out of available memory, the operating system swaps and writes the old pages to the hard disk. Reading and writing from hard disk is about 5 orders of magnitude slower than reading and writing from memory. The USED_memory metric helps determine whether Redis is at risk of being swapped or has already been swapped.

If there is no swap, Redis will be killed by the Linix Kernel’s OOM Killer due to out of memory if it suddenly needs more memory than is currently available to the operating system. Although Redis performance deteriorates when its data is swapped out, it is better than being killed outright. : Advice from: redis. IO /topics/admi…

Where did the memory go? It exploded before it could be used properly

Memory reclamation policy

There are roughly two mechanisms for Redis to reclaim memory: one is to delete key value objects that reach expiration time; Second, when the memory reaches maxMemory, the memory removal control policy is triggered to forcibly delete the selected key value object.

Expired keys

How does Redis eliminate expired keys

Redis Keys can expire in two ways: passive and active. When some client tries to access it, the key is discovered and actively expires.

Of course, this is not enough because some keys are out of date and will never access them. These keys should expire anyway, so periodic random tests set the expiration time of keys. All expired keys will be removed from the key space. This is exactly what Redis does 10 times per second:

Test 20 random keys for relevant expiration detection. Delete all keys that have expired. If more than 25% keys are out of date, repeat step 1.Copy the code

In order to obtain correct behavior without sacrificing consistency, when a key expires, DEL will be synthesized to all additional slaves along with the AOF word. In the master instance, this method is centralized and there is no chance of consistency errors.

However, slaves do not expire keys independently when connected to master (wait until master executes DEL), they still exist in the data set, so eliminating keys independently when slave is elected master and becomes master.

Memory removal control policy

Eight solutions

The behavior Redis uses when the maxmemory limit is reached is configured by Redis’s maxMemory-policy configuration directive.

There are eight solutions proposed in redis.conf.

volatile-lRU -> Exit using an approximate LRU, using only keys with expired values. Allkeys-lru -> use approximate LRU to expel any key.volatile-lFU -> Uses an approximate LFU for eviction, using only keys with expiration sets. Allkeys-lfu -> Use approximate LFU to expel any key.volatile-random -> Deletes a random key with expiration Settings. Allkeys-random -> Delete a random key, any key.volatile-TTL -> Delete key with last expired time (smaller TTL) Noeviction -> Does not expel anything, just returns an error during write operations.Copy the code

LRU indicates the least recently used LFU indicates the least frequently used LRU, LFU, and volatile- TTL are all implemented using approximate random algorithms.

Note: With any of the above strategies, Redis will return an error on write operations when there is no suitable key for eviction.

General rule of thumb:

Use the Allkeys-LRU strategy: When you want your request to conform to a power law distribution, that is, you want some subset elements to be accessed more than others. If you're not sure what to choose, this is a great option. Use allkeys-random: if you are accessing in a loop, allkeys are scanned consecutively, or you want the request distribution to be normal (all elements are accessed with equal probability). Use volatile- TTL: If you want to set the TTL value when creating cache objects, determine which objects should be expired.Copy the code

The AllKeys-LRU and volatile-random strategies are useful when you want to cache and persist keys in a single instance. But generally running two instances is a better way to solve this problem. Setting expiration times for keys also consumes memory, so using allkeys-lRU is more efficient.

How does the recycle process work

A client runs a new command to add new data. Redis checks memory usage, and if it exceeds the maxMemory limit, it reclaims according to the preset policy. A new command is executed, etc. So we keep crossing the boundary of the memory limit, by constantly reaching the boundary and then constantly reclaiming back below the boundary.Copy the code

Approximate LRU algorithm

Redis LRU algorithm is not a complete implementation. The reason why Redis doesn’t use a real LRU implementation is because it requires too much memory.

Memory optimization

Use 32-bit Redis

With 32-bit Redis, less memory is used for each key because Pointers take up fewer bytes for 32-bit programs. However, the entire redis instance of 32 will be limited to less than 4G of memory. Use the make 32bit command to compile and generate 32-bit Redis. RDB and AOF files are 32-bit and 64-bit (including byte order), so you can use 64-bit Reidis to restore 32-bit RDB backup files, and vice versa.

Bit-level and word-level operations

Use hash tables whenever possible

Small hash tables use very little memory, so you should try to abstract your data model into a hash table as much as possible.

Let’s stop here. It’s time to eat.