Original author, public number [programmer reading], welcome to pay attention to the public number, reprint the article please indicate the source oh.

In this article, we will continue our knowledge of Redis and take a look at one of the most important aspects of Redis: its persistence mechanism.

What is Redis persistence?

Redis acts as a key-value in-memory database (NoSQL). Data is stored in memory and all operations are performed in memory when processing client requests, as shown below:

What’s wrong with that?

In fact, as long as a little basic computer knowledge knows, data stored in memory, as long as the server shutdown () due to a variety of causes, the data in memory will disappear, not only the server shut opportunity data to disappear, Redis server daemon exits, the same data in memory will disappear.

For projects that only use Redis as a cache, data disappearance may not be a problem, and data can be re-loaded from the data source. However, if the business data submitted by users are directly stored in Redis, Redis is used as a database, and important business data can be stored there. The impact of Redis’s memory data loss could be devastating.

In order to avoid data loss in memory, Redis provides support for persistence. We can choose different ways to save data from memory to hard disk, so that data can be persisted.

Redis provides two different methods of data persistence: RDB and AOF. Let’s take a look at these different methods in detail.

RDB

RDB is a snapshot storage persistence mode. Specifically, the Redis memory data at a certain point is saved to a file on the hard disk. The default file name is dump. RDB.

Enable the RDB persistence mode

Enabling RDB persistence is simple. Clients can send save or BGSave commands to the Redis server to make the server generate an RDB file, or specify RDB trigger conditions from the server configuration file.

1. Save command

The save command is a synchronous operation.

# Synchronize data to disk
> save 
Copy the code

When a client sends a save command to the server for persistence, the server blocks requests from other clients following the save command until data synchronization is complete.

If the amount of data is too large, synchronizing the data will take a long time, during which time the Redis server cannot receive other requests, so it is best not to use the save command in a production environment.

2. bgsave

Unlike the save command, the BGsave command is an asynchronous operation.

Save the data set to disk asynchronously
> bgsave
Copy the code

When the client sends the BGSave command, the main Redis server forks a child to synchronize the data, which exits after saving the data to an RDB file.

So, in contrast to the save command, the Redis server uses sub-threads to write IO to bgSave, and the main process can still receive other requests, but the Forks sub-process is synchronous, so the forks sub-process cannot receive other requests, which means, The BGSave command can still block requests from other customers if it takes too long (usually too fast) to forks a sub-process.

3. The server configuration is triggered automatically

In addition to sending commands from the client, save in the Redis configuration file specifies that a condition has been reached to trigger RDB persistence, such as RDB data synchronization.

For example, we can specify the following options in the redis.conf configuration file:

At least one write command is reached within 900s
save 900 1
At least 10 write commands in # 300s
save 300 10
Reach at least 10000 write commands in 60 seconds
save 60 10000
Copy the code

The configuration file is then loaded when the server is started.

Start the server to load the configuration file
redis-server redis.conf
Copy the code

This is similar to the bgsave command. When the trigger condition is reached, data synchronization will be performed by a forks sub-process. However, it is best not to trigger RDB persistence by using this method because the trigger time is too short and RDB files will be frequently written, affecting server performance. If the time is too long, data will be lost.

RDB file

There are three ways to get the server to generate RDB files, either by the main process or by a child process, as follows:

  1. Generate a temporary RDB file and write data.
  2. Complete the data write and replace the official RDB file with a temporary text.
  3. Delete the original DB file.

The default generated file for RDB is dump. RDB. The configuration file can be used for more detailed configuration. For example, when starting multiple Redis server processes on a single machine, you can configure different RDB names based on port numbers, as shown below:

Whether to compress RDB files
rdbcompression yes

# RDB file name
dbfilename redis-6379.rdb

# RDB file save directory
dir ~/redis/
Copy the code

Several advantages of RDB

  1. Compared with AOF, data recovery from RDB files is faster.
  2. RDB files are very compact and suitable for data backup.
  3. Data backup through RDB has little impact on Redis server performance due to the use of sub-process generation.

RDB has several disadvantages

  1. If the server is down, useRDB“Will cause data loss in a certain period of time. For example, if we set synchronization once in 10 minutes or synchronization once in 5 minutes when 1000 writes are reached, if the server freezes before triggering conditions are reached, the data in this period will be lost.
  2. Using the save command will block the server until the direct data synchronization is complete to receive subsequent requests.
  3. Too much data can block the Forks process using the BGsave command, and the forks process can consume memory.

AOF

After talking about RDB, let’s talk about another Redis persistence method: AOF(appends-only file).

Unlike RDB, which stores snapshots at a certain point in time, THE AOF persistence mode records every write operation command from the client to the server, and stores these write operations to the end of the file with the suffix AOF in Redis protocol. When the Redis server restarts, the AOF file commands are loaded and run to recover data.

Enable AOF persistence

AOF persistence is not enabled by default in Redis. You can enable AOF persistence in the configuration file as shown in the following Redis.

# Enable aOF
appendonly yes

# aof file name
appendfilename "appendonly.aof"

Write policy,always means that every write operation is saved to aof file, everysec or no
appendfsync always

Aof files are not overwritten by default
no-appendfsync-on-rewrite no

# save directory
dir ~/redis/
Copy the code

Three write strategies

In the configuration file above, we can specify the write policy through the appendfsync option, which has three options

appendfsync always
# appendfsync everysec
# appendfsync no
Copy the code
1. always

Each write operation on the client is saved to an AOF file. This policy is secure, but each write has IO operation, so it is slow.

2. everysec

Appendfsync’s default write policy writes to aOF files once per second, so at most 1s of data may be lost.

3. no

The Redis server is not responsible for writing the AOF, but leaves it up to the operating system to handle when the AOF file is written. Faster, but also the least safe option and not recommended.

AOF file overwritten

AOF appends each write operation on the client to the end of the AOF file. For example, when the INCR command is executed on a key multiple times, AOF saves each command to the AOF file, and the AOF file becomes very large.

incr num 1
incr num 2
incr num 3
incr num 4
incr num 5
incr num 6
...
incr num 100000
Copy the code

To solve this problem, Redis supports aOF file rewriting. By rewriting AOF, you can generate a minimum set of commands to restore the current data, such as the number of commands in the above example, which can be rewritten as:

set num 100000
Copy the code

The aOF file is a binary file, and instead of saving each command directly as in the example above, Redis uses its own format, which is just for demonstration purposes.

Two ways to rewrite it

The no-appendfsync-on-rewrite option in the redis.conf configuration file allows you to set whether to enable rewriting. This option overwrites every fsync, affecting server performance, so default is no and is not recommended.

Aof files are not overwritten by default
no-appendfsync-on-rewrite no
Copy the code

The client can also send the bgrewriteaOF command to the server to have the server do AOF rewriting.

Let the server asynchronously override the append aOF file command
> bgrewriteaof
Copy the code

AOF overwriting is also an asynchronous operation, i.e. if an AOF file is written, the main Redis process forks a child process, as shown below:

Benefits of rewriting aOF files

  1. Compress AOF files to reduce disk usage.

  2. Aof commands are compressed into a minimum command set to speed up data recovery.

The AOF file is damaged

If the Redis server is down when writing the AOF log file, the AOF log file will be formatted incorrectly. When the Redis server is restarted, the Redis server will refuse to load the AOF file. You can perform the following steps to repair the AOF and recover the data.

  1. Backup current AOF files, just in case.

  2. Run the redis-check-aof command to repair aOF files. The command format is as follows:

Fix the AOF log file
$ redis-check-aof -fix file.aof
Copy the code
  1. Restart the Redis server, load the repaired AOF file, and restore the data.

The advantages of AOF

AOF simply apends log files, so it has less impact on server performance, is faster than RDB, and consumes less memory.

The disadvantage of AOF

  1. The log files generated in AOF mode are too large. Even if they are overwritten by AFO, the file size is still too large.

  2. Data recovery is slower than RDB.

RDB or AOF?

Through the above introduction, we understand the advantages and disadvantages of RDB and AOF, how to choose?

In fact, if you want to ensure data security, you can enable both methods. However, the IO operation of the two persistent methods at the same time will seriously affect server performance. So sometimes choices have to be made.

When both RDB and AOF are enabled, Redis preferentially uses AOF logs to recover data because AOF holds more complete files than RDB files.

summary

Spoke above a lot of Redis persistence mechanism of knowledge, in fact, if you just put the Redis as a caching server, you can need not consider completely persistence, however, in today’s most server architecture, the Redis only play the role of a cache server, also can be used as a database, to save our business data, At this point, we need to understand the differences and choices about Redis persistence strategy.

Your attention is the biggest encouragement on my writing road!