preface

Because Redis stores data in memory, it can read data quickly, but it can also introduce some new problems. When the redis server goes down, data in the Redis database is lost. To be able to reuse redis data, we need to store the data in memory to disk, called persistence. Redis provides two methods of persistence, one is RDB snapshot and the other is appending file AOF only

RDB

What is the

RDB is to write a snapshot of the data set in the memory to the disk at a specified interval. The snapshot file is directly read into the memory during recovery.

implementation

RDB is implemented in two ways: SAVE and BGSAVE.

  • SAVE is blocking RDB persistence. When executing this command, the main redis process writes a snapshot of the data in memory to the RDB file, and redis does not execute any other commands until the process is complete
  • BGSAVE is non-blocking RDB persistence. Redis creates a child process to write a snapshot of the data in memory to an RDB file, while the main process can execute other commands. (Note: the child process is basically a copy of the parent process, equivalent to two Redis processes running on the system, which will cause a large increase in memory usage)

configuration

# 900 1 300 10 save save save time strategy 60, 10000 # file name dbfilename dump. # RDB file path dir/home/work/app/redis/data / # If persistence fails, does the main process stop writing stop-writing-on-bgsave-error yes # Whether to compress rdbCompression yes # Whether to check rdbChecksum yes when importingCopy the code

Configuration instructions

  • Save 900 1 Indicates that if one write command is executed within 900 seconds, a snapshot is generated, which can be interpreted as a backup
  • Stop-writes-on-bgsave-error yes this configuration is also very important. When the backup process fails, the main process stops accepting new writes. This is to protect persistent data consistency. If your own service has a comprehensive monitoring system, you can disable this configuration. Otherwise, enable it
  • Of course, if you want to disable RDB configuration, it is very easy to just say: save “” on the last line of save.

There is a problem

  • The last persisted data may be lost.
  • Excessive memory consumption (AOF is recommended)

AOF

What is the

Log each write operation, record all write instructions performed in Redis (read operations are not recorded), only file appending but cannot be overwritten, redis will read the file to rebuild data, in other words, When redis is restarted, write instructions are executed from front to back according to the contents of the log file to complete data recovery. AOF holds the appendone.aof file

Realize the principle of

AOF persistence is divided into three steps: command append, file write, and file synchronization.

  1. When Redis has AOF enabled, the server appends this command to the end of a separate AOF cache each time it performs a write operation, such as set, sadd, or rpush.
  2. Write a command from the AOF cache to an AOF file. This needs to start from the system’s file writing mechanism: In order to improve the efficiency of writing files, the current operating system has a write strategy. When you write data to the hard disk, the operating system does not write data to the hard disk in real time, but stores the data temporarily in a memory buffer. Wait until the memory buffer is full or the specified time limit is exceeded before writing data to disk.
  3. When Redis performs the second file write, from the user’s point of view, the AOF cache has been written to the AOF file, but from the system’s point of view, the CONTENTS of the AOF buffer are simply put into another buffer. Redis then needs to perform file synchronization to actually write the data in the memory buffer to the disk (when file synchronization is performed according to the appendfsync configuration).

configuration

Appendfilename "appendone. aof" # synchronization mode appendfsync everysec # aof Whether to synchronize during rewriting No-appendfsync-on-rewrite no # autoofs-rewrite-percentage 100 auto-ofs-rewrite-min-size 64mb # autoofs-rewrite-size 64mb Aof-load-truncated yes # File rewrite policy aof-rewrite-increment-fsync yesCopy the code

Appendfsync has three modes:

  • Always: The server will forcibly write the contents of the AOF buffer to the AOF file on the hard disk at every event. It can be considered as if you execute a REDis write command to the AOF file. This ensures the integrity of data persistence, but is the slowest and safest
  • everysec: Each time a server performs a write operation (such as set, sadd, or rpush), it appends the command to the end of a separate AOF buffer and writes the AOF buffer to an AOF file. File synchronization is performed every second to actually write the AOF cached data in the memory buffer to an AOF file. This mode takes into account both efficiency and data integrity. Even if the server goes down, the redis database changes will only be lost within a second
  • no: Redis database data even if missing you can also accept, it will write command appended to each AOF towards the end of the buffer, then write to the file, but when to file synchronization really write AOF the data file is determined by the system itself, namely when memory buffer space is filled with or exceed the time limit set by the system automatically after synchronization. In this mode, the efficiency is the fastest, but the data is also the most insecure. If the data in Redis is taken out from the background database such as mysql and belongs to the data that can be retrieved at any time or is not important, then this mode can be considered

There is a problem

  • The AOF file is too large, and every write to the Redis database will add one more piece of data to the AOF file. Over time, the file will become a behemoth

Solution: AOF override mechanism (Redis persistence —-AOF override mechanism)

Refer to the article

  • Redis — Persistence section
  • See Redis persistence principle
  • Redis persistence mechanism (RDB&AOF)