Springboot Actual e-commerce project Mall4j (https://gitee.com/gz-yami/mall4j)

Redis persistence and memory optimization

Redis configuration file to carry out some persistent and memory optimization operations, if there is an error welcome guidance.

1. Why persistence

If user data is stored in the memory, the memory data will be cleared when the server is powered off or down. As a result, cache data will be cleared.

2. Process of using persistent files

After we installed Redis, all the configuration was in the redis. Conf file, which saved the various configurations of both RDB and AOF persistence mechanisms.

Run the save command to persist the current data and generate an RDB file. Bgsave: Memory data is persisted once in the background to generate an RDB file. The difference between commands is as follows: save: When the save command is executed, the set operation is blocked. Bgsave: When bgSave is executed, redis is notified that persistence is required, and redis persists based on user usage without blocking, similar to GC (garbage collection). Rules for using persistent files: A persistent file is generated when the program runs normally. If the server is restarted after a breakdown, data is recovered based on the persistent file specified in the configuration file.Copy the code

3. The RDB schema

Description: 3.1

RDB advantage

1. The RDB mode is the default redis persistence policy with compact files and full backup, which is ideal for backup and disaster recovery.

2. When the RDB file is generated, the main Redis process forks () a child process to handle all the save work. The main process does not need to do any disk IO.

3. RDB can recover large data sets faster than AOF.

4. In RDB mode, snapshots of memory data are taken for each operation, and the persistence file is small.

RDB problem:

1. Time consumption, memory consumption, and I/O performance consumption. It takes a certain amount of time to dump all data in the memory to disks. Bgsave mode fork () child processes take up extra memory, and heavy disk reads and writes also consume IO performance.

2. Uncontrolled data loss. During an outage, memory data written after the last snapshot will be lost.

3.2 Changing the name and storage path of the Persistent file

​

3.3 RDB Mode The default snapshot mode is used

Save 900 1 If at least one key changes within 900 seconds, the snapshot is written to the disk. Save 300 10 If at least 10 keys change within 300, the snapshot is written to the disk. Save 60 10000 If at least 10000 keys change within 60 seconds, snapshots are written to disks.Copy the code

4. AOF mode

4.1 What is AOF

Full backup of Redis RDB mode is always time-consuming, and AOF mode is complementary to RDB mode. If the AOF mode is enabled (AOF has a higher priority than RDB mode when both modes are enabled), the redis write command records logs (log files ending with AOF) each time you run the command. When Redis fails, data can be recovered simply by performing log playback.

4.2 Enabling AOF Mode

Modify the Redis configuration file to yes.

4.3 AOF persistence Strategy

Appendonly. Aof #appendfsync everysec # Appendfsync-on-rewrite no # When an aof log file is about to grow to a specified percentage, Whether Redis automatically overwrites AOF log files by calling BGREWRITEAOF.Copy the code

The Everysec policy is usually used, which is also the default policy for AOF.

4.4 AOF log rewrite function

When the AOF log file becomes too large, Redis automatically overwrites the AOF log. The Append mode constantly writes the updated records to the old log file, and Redis creates a new log file to append subsequent records.

Aof rewriting can greatly reduce the size of the final log file. This reduces disk consumption and speeds up data recovery. For example, we have a count service that has a number of auto-increment operations, such as auto-increment the key to 100 million, which is 100 million INCR for AOF files. The Aof rewrite records only one record.

4.5 Two methods of AOF rewriting

• The Bgrewriteaof command triggers AOF rewrite The Redis client sends the Bgrewriteaof command to Redis, and the Redis server derives a child process to complete the AOF rewrite. The AOF rewrite here traces the data in Redis memory back to an AOF file. Rather than rewriting the AOF file to generate a new AOF file to replace.
• AOF overrides configuration
  • Auto-aof -rewrite-min-size: The size required to rewrite the AOF file
  • Auto-aof -rewrite-percentage: indicates that aOF files increase
  • Aof_current_size: Computes the current size of AOF (bytes)
  • Aof_base_size: Size (bytes) of the AOF that was last started and rewritten
• The triggering time of AOF automatic rewriting should simultaneously meet the following two points:
  • aof_current_size > auto-aof-rewrite-min-size
  • aof_current_size – aof_base_size/aof_base_size > auto-aof-rewrite-percentage

5. RDB versus AOF

6. Redis memory optimization strategy

Redis supports a variety of memory expulsion strategies to limit memory usage, based in part on LRU and LFU algorithms.

6.1 LRU algorithm

LRU stands for Least Recently Used. It is a common page replacement algorithm that selects the pages that have not been Used for the most recent time to eliminate the page. The algorithm assigns each page an access field, which is used to record the time T experienced by a page since it was visited last time. When a page needs to be eliminated, the page with the largest T value in the existing page, that is, the least recently used page, is selected to be eliminated.

Time T: indicates the time since the last time.

6.2 LFU algorithm

Note :LFU algorithm is proposed after REDIS5.

LFU (least frequently used (LFU) Page-replacement algorithm). That is, the least frequently used page replacement algorithm requires that the page with the smallest reference count be replaced during page replacement, because frequently used pages should have a larger number of references. However, pages that are used a lot at the beginning but are no longer used will remain in memory for a long time, so the reference count register can be periodically moved one bit to the right, resulting in an exponentially decaying average number of uses.

6.3 Redis memory optimization

5.4 Modifying the Redis Configuration File

Modify the maxmemory-policy attribute to select an appropriate policy.

Springboot Actual e-commerce project Mall4j (https://gitee.com/gz-yami/mall4j)