Persistence mechanism

Although Redis is a memory-based key-value service, data can also be persisted so that the service can be restarted and the data can be reloaded.

To ensure data loss as much as possible, Redis provides several persistence mechanisms:

  • RDB: Redis Database, which mirrors the Redis memory data ata specified interval and saves it to a file.
  • AOF: Appends the server’s write operations to a file, which records all the logical operations.
  • No persistence: The performance is best when no persistence is carried out.
  • RDB + AOF: Combine RDB and AOF to combine their respective advantages.

Let’s take a look at these mechanisms.

RDB configuration

In the redis. Conf file we can set the RDB persistence mode:

Save 900 1 # 900 seconds if 1 key has changed, the action will be persisted. Save 300 10 # 300 seconds if 10 key has changed, the action will be persisted. Save 60 10000 # if 10000 key values change within 60 seconds, Redis will not update the data. Stop -writes-on-bgsave-error yes # Whether to save persistent RDB files rdbcompression yes # Whether to verify persistent RDB files rdbChecksum yes # RDB filename dbfilename example. RDB # Export directory dir /opt/redisdataCopy the code

RDB advantage

  • When Redis persists, it forks a child process and writes data to the child. The parent process does not do disk IO, so RDB performance is very good.

  • If you’re on a Unix system, you can also take advantage of the copy-on-write mechanism. That is, the child and the parent share the same memory page and will only be copied if the child or parent makes changes. This saves the use of physical memory.

  • Because RDB files only store memory data at a certain point in time, there is no logical command, so it can be quickly loaded in the case of restart recovery.

RDB shortcomings

  • Although the RDB fork can make Redis persistence independent, once the data volume is large, it will always occupy CPU, and may affect the parent process.

  • In the previous RDB configuration, we mentioned that persistence is triggered at a certain interval, but because of this interval, there is a certain probability that data will be lost during this time.

AOF configuration

Similarly, we can configure AOF persistence in the redis. Conf file:

Appendonly no # Whether to enable aof appendfilename "appendonly.aof" # aof filename # Sync to disk policy default once per second # appendfsync always # every time Appendfsync everysec # once per second # appendfsync no Default Linux configuration loses a maximum of 30 seconds auto-aof-rewrite-percentage 100 # aof files will be rewritten if the number of aof files exceeds the percentage. Auto-aof -rewrite-min-size 64mb # aof files will be rewritten if the number of aof files exceeds the percentage Aof -load-truncated yes # If the last instruction of AOF is incorrect, it will be ignored when resuming.Copy the code

In the configuration above, we see the configuration for aOF file rewriting. Aof needs to be overridden because the AOF file records logical operations. As the system runs longer, there are more and more operation commands, and the log files become larger, so they need to be overwritten to reduce disk space.

When AOF does a file rewrite, it rearranges the memory data, saves it to a new file, and then replaces the original AOF file with the new file.

AOF advantage

  • AOF allows us to persist at a rate of one second, which greatly reduces data loss.
  • AOF uses appending to write files, so that even if persistence fails, the impact is less and can be repaired using redis-check-aof.

AOF shortcomings

  • As mentioned earlier, logs get bigger and need to be overwritten to reduce disk footprint.

Mixed persistence

To combine the benefits of AOF and RDB, Redis has enabled mixed persistence since 4.0. Its configuration is as follows:

aof-use-rdb-preamble yes
Copy the code

When mixed persistence is enabled, Redis forks the child process to write the memory data to the AOF file in RDB format, followed by the incremental commands to overwrite the buffer to the file in AOF format. In this way, the new AOF file has data in both formats.

When mixed files are used for recovery, the system checks whether the header of AOF file is in RDB format first. If yes, data is loaded in RDB mode first, and then the subsequent AOF data is processed.

This speeds up data recovery and reduces the risk of data loss.

Of course, this hybrid approach makes AOF files complex and unreadable, and they need to be at least version 4.0 to be used.

conclusion

Redis persistence has its own characteristics, so which mechanism should we choose?

  • If our data is allowed to be lost, then we should turn off persistence, so that the performance is good.
  • If we want to keep the data as secure as possible, we should choose hybrid persistence.
  • If we want to ensure that data is not lost, we should set appendfsync to always, but this has a significant impact on performance, so mysql is generally considered directly.

Interested friends can search the public account “Read new technology”, pay attention to more pushed articles. Thank you for your support! Read new technology, read more new knowledge.