introduce

Redis is an in-memory database that will lose data once the main process exits. There are two main mechanisms for its persistence, namely AOF logging and RDB snapshot. This article focuses on AOF logging.

process

After executing a write command, Redis appends the write command to the end of the AOF buffer in the form of protocol text, and then decides whether to write & synchronize the contents of the AOF buffer to the AOF log through the synchronization policy.

Synchronization strategies

The AOF mechanism provides three options for the appendfsync configuration item:

  • Always:AOFEverything in the bufferwriteandsynchronoustoAOFThe logs.
  • Everysec:AOFEverything in the bufferwritetoAOFLog if the last synchronizationAOFThe log time is now more than 1 second away, then yesAOFLog onsynchronous.
  • No:AOFEverything in the bufferwriteTo the AOF log, but not to the AOF logsynchronousThe operating system decides when to synchronize.

The advantages and disadvantages of the three strategies are also obvious. Always has low reliability and high performance, while NO has low reliability and high performance. Everysec takes a compromise between the two strategies.

Rewriting mechanism

The AOF log is a file that records all write commands received. As more write commands are received, the file size increases. That means there are some problems.

  • The operating system (OS) has a limit on file size. Files that are too large cannot be saved.
  • Appending data to a large file reduces efficiency.
  • AOFThe log volume is large and the fault recovery process is slow.

To solve the above problem, Redis provides rewriting functionality. With this feature, Redis can create a new AOF log to replace the existing AOF log.

  • Two copies ofAOFThe database where logs are stored is in the same state.
  • newAOFLogs do not contain redundant commands that waste space.
  • newAOFThe log volume is smaller than or equal to the originalAOFThe logs.

Rewrite the process

Why execute in child process?

It does not block the main thread and can continue processing command requests while the child processes the AOF rewrite.

Is there a write command in the rewrite process, which will cause data inconsistency?

No, Redis maintains the AOF rewrite buffer, which is used after the main thread creates the rewrite child. If a write command is requested during the rewrite, Redis appends the AOF buffer and the AOF rewrite buffer. When the child process finishes creating a new AOF log, the main process will append all contents in the AOF rewrite buffer to the new AOF log. At this time, the database state stored in the old and new AOF logs is exactly the same. Finally, the new AOF log overwrites the old AOF log to complete the AOF log rewrite operation.