Redis 的 AOF 和 RDB

Redis is the most popular key-value database. As a high-performance database, Redis stores its own data in memory rather than on disk. As a result, if you do not have a way to save the data stored in memory to disk, Once the server process exits (the service goes down), the data in memory is lost.

To solve this problem, Redis provides two ways to persist data, commonly known as AOF and RDB. The former uses AOF (Append Only File) file to record the Redis data write operation. Once the process exits, the data will automatically recover Redis according to the AOF file write operation. The latter is to use the RDB (Redis Database) binary file to save the Database state, after the process exits the binary file will be read to achieve the purpose of recovering data data, this paper mainly summarizes the two persistence methods and briefly describes their principles.

AOF persistence

In simple terms, AOF persistence refers to recording database state by saving write commands executed by the Redis server, such as:

redis> SET msg "hello"
OK
Copy the code

After using AOF, you can open the file and take a look. It looks something like this:

*2\r\n$6\r\nSELECT\r\nThe $1\r\n0\r\n
*3\r\n$3\r\nSET\r\n$3\r\nmsg\r\nA $5\r\nhello\r\n
Copy the code

In this AOF file, we send commands through the client, except that the SELECT for specifying the database is automatically added by the server. Where *3 indicates that the current command has three parts, each of which begins with a $+ number, followed by a key, value, or command. The numbers here indicate how many bytes the key, value, or command has, for example, $3\r\nSET means that the command has three bytes, and $5\r\nhello means that the value has five bytes.

AOF write and synchronize

The Redis server process is an event loop in which file events receive command requests from clients and send command replies to clients. The flushAppendOnlyFile function is called every time the server terminates an event loop, because the server may execute write commands that append content to the aof_buf buffer. Consider whether you want to write and save the contents of the AOF_BUf buffer to an AOF file. Therefore, AOF does not block file writes.

However, there is a significant problem with AOF: if a server goes down when the contents of the AOF_buf buffer are written to an AOF file, these writes will disappear permanently. To solve this problem, Redis provides three write back strategies:

To summarize the write back strategy of AOF:

Always or Everysec is recommended for reliability, and No is recommended for high performance. Write back strategies always change based on requirements.

AOF file loading and data restoration

Once the process exits, it is necessary to load the command according to AOF and write back to the database to restore the database state:

In the example above, the server first reads and executes SELECT 0, then executes SET MSG “hello”, and the server database is restored to its previous state, very simple.

AOF rewrite

Because AOF persistence records database state by storing write commands that are executed, as AOF files grow in size, so does the size of the files. With the increasing size of AOF files, the restoration time also increases. In order to solve the problem of AOF volume expansion, Redis provides the function of AOF file rewriting. Through AOF rewriting, Redis can create a new AOF file to replace the existing AOF file, and the new AOF file is usually much smaller than the old one. So how does AOF implement file rewriting? Here’s a quick example:

redis> SADD colors "red"  // {red}
(integer) 1

redis> SADD colors "yellow" "blue" // {red, yellow, blue}
(integer) 3

redis> SREM colors "red" // {yellow, blue}
(integer) 1

redis> SADD colors "orange" // {yellow, blue, orange}
(integer) 3
Copy the code

To record such a command, you need to write four times to the AOF file; If the colors value can be read from Redis and replaced with a SADD colors “yellow” “blue” “orange” command, the number of commands required can be reduced from four to one. The AOF override function reads the value of the key from the database and records the key/value pair with a single command instead of multiple commands. At the same time, AOF log rewriting is through the backend process, will not block the main process, students can explore the specific process ~

RDB persistence

RDB is also known as memory snapshot, RDB persistent generated RDB file is a compressed binary file, thanks to the file can be generated when the RDB file database state.

The RDB files are stored on disk, and Redis can use the RDB to restore the database (automatically) as long as the RDB file exists, even if the process exits.

RDB file creation

The SAVE command blocks the Redis server process until the RDB file is created. The server cannot process any command requests while the RDB file is blocked. The BGSAVE command generates a child process that creates the RDB file, and the parent process continues to process the command requests. In addition, if Redis has both AOF persistence and RDB persistence enabled, the server will use AOF files to restore database state preferentially. The RDB file is used to restore the database state only when AOF persistence is disabled

Automatic interval saving

Because the BGSAVE command can be executed without blocking the server process, Redis allows the user to configure the save option so that the server automatically executes the BGSAVE command once in a while, as an example:

save 900 1
save 300 10
save 60 1000
Copy the code

BGSAVE is executed if any of three criteria are met: 1) The server has made at least one change to the database within 900 seconds; 2. The server has modified the database at least 10 times within 300 seconds; 3. The server made at least 1000 changes to the database within 60 seconds.

A mixture of

Redis4.0 supports the use of a mixture of AOF files and RDB to restore Redis. Between two RDB file creation, use AOF file to record the commands written by Redis. In this way, you can avoid data loss caused by two RDB writes, and the problem of AOF file size caused by only using AOF file. “The adult world is” I want it all “.

conclusion

Through Redis AOF persistence and RDB persistence can avoid the process exit caused by data loss problems, of course, what way to use must be combined with the characteristics of the business, the use of Redis new features can also learn from each other, the two ways combined, might as well try

reference

• Redis Getting Started
• Redis Design and Implementation
• Redis Core Technology and Practice