Redis is the most widely used NOSQL database. It is an advanced key-value database. It is similar to memcached, but it can be persisted. Today we are going to talk about Redis persistence.

Redis provides two methods of persistence.

  • Aof (APPEND ON FILE) persistence: The principle is to write redis operations into aOF files in the form of commands.
  • Redis DataBase (RDB) memory snapshot persistence is to copy all data in the Redis memory to a disk.

Let’s examine these two types of persistence in detail.

Aof persistence

Aof persistence is to record the commands of REIDS to an AOF file after each successful execution.

After the AOF command is successfully executed, AOF logs are recorded to ensure that the AOF commands are correct each time. Errors are directly returned to the client. How is it recorded in the AOF files? Let’s take a successful redis command as an example:

set test 'hello world'
Copy the code

*3 in the aOF log content indicates that the command consists of three parts: set, test, hello world. Each part starts with a $+ number and is followed by a command, key, or value. The number represents the length (in bytes) of the command, key, or value that follows. For example, “$4 test” indicates that this part has 4 bytes, which is test. As is shown in

Advantages of aOF logging after redis

  • Incorrect commands are filtered out during command execution and will not be recorded in aOF.
  • AOF logs do not block operations because commands are written after successful execution.

Risk of AOF logs

  • If the server breaks down after the command is executed successfully but before the AOF log is written, the data will be lost.
  • Although AOF avoids blocking the current command, the AOF log compass action is still executed in the main thread and will still block the next command if the disk is full and causes a group race.

Redis provides three aOF write back strategies

  1. Always Synchronous write back: Logs are written back to the disk immediately after each command is executed.
  2. Everysec write back per second: After each command is executed, logs are written to the memory buffer of the AOF file and commands in the buffer are written to disk every second
  3. No Operating system-controlled write back: After each write command is executed, logs are first written into the memory buffer of the AOF file. The operating system determines when to write back to the disk.

Let’s examine three write-back strategies.

Always write back occupies the main thread resources, each write back is a slow compass operation, basically can avoid the loss of data, will affect the main thread, if you have very high requirements for the accuracy of redis data, but the write and read ratio is not high, you can use this strategy.

Per second in the back to write a write back using a second strategy, avoid the synchronization to write performance overhead, while reducing the impact on system performance, but if 1 seconds to produce a large number of write operations, aof buffer in the backlog a lot of log, then could write aof outages have taken place in the log file, can cause the loss of data.

Operating system controlled write back, after writing the AOF buffer can continue to execute commands, but if the system goes down, it will also cause data loss.

Which strategy to use depends on your business scenario.

What are the performance issues with AOF logging itself?

  • Aof logs are files that store commands to disk. If the changes are frequent and the Redis database is large, the AOF file logs will be too large.
Excessive documentation may cause problems in the following three aspects:
  • If the Redis server restarts, relying on aOF files to restore the database will take too long, which will affect performance.
  • File systems have size limits on the size of files themselves.
  • The file is too big, and when you add more content to it, it slows down.
How to solve this problem?

Redis provides aOF rewriting mechanism. The AOF log is to record all write commands. If a key pair is changed ten times, it will record ten times.

Although the AOF rewriting mechanism greatly reduces the size of AOF files, aOF rewriting can take a lot of time if the redis database itself is large.

Will AOF overwriting block?

The aOF log is overwritten by the bgreWriteAOF child daemon, again to avoid blocking the main thread and causing data performance degradation.

On rewrite, the main Redis process forks a bgreWriteAof child, and the bgrewriteAof completes the write back without affecting the main process.

What if a command is written to the main process during a write back?

If there is a new write, the main process will write in the old AOF buffer and also in the new AOF file buffer, so that even if the rewrite fails, the old AOF data will not be lost. After the new AOF file is rewritten, the AOF file buffer can be written into the AOF again, and the new AOF file will not lose the file.

RDB log

RDB persistence stores binary data on disk as a memory snapshot ending in.rdb.

Two RDB storage modes

  • Save: Save is required to occupy the main thread resources, will block the main thread.
  • Bgsave: Write using child processes and do not use resources from the main thread (fork blocks the main thread, but copyonwrite is relatively fast at the nanosecond level and aOF is a millisecond level).

Redis creates a full snapshot for each RDB to ensure data reliability.

What if data is written while Redis is performing a snapshot?

Redis uses copy-on-write COW (COPy-on-write COW) in the operating system. The bgSave child process is generated by the fork process and can share the memory of the fork process. If the fork process has a WRITE command, the main process copies the page to the fork first. In this way, write operations of the main process are not affected and original data can be modified, avoiding adverse impact on services.