What is Redis persistence?

Maybe many people seldom touch this word, always feel that all the data of Redis can be stored permanently. What you may not know, however, is that Redis data is stored in memory, and without a persistence strategy, you may lose all of your data if you shut down Redis or after. Every time we log into Redis to access the previous data, we see the original data, thanks to Redis persistence. Redis persistence simply means storing Redis values in memory to a permanent location (disk, etc.)

Redis persistence scheme

  • RDB Redis DataBase
  • AOF Append Only File

RDB Redis DataBase

RDB is the default Redis persistence scheme. When certain conditions are met, data in the current memory is written to disks and a snapshot file dump. RDB is generated. The Redis restart restores data by loading the dump. RDB file. Dump. RDB is one of our redis files.

How does RDB persist

RDB triggers persistent storage according to rules. In our redis.conf we can see the following configuration:

Save 900 1 # 900 seconds At least one key is changed (including adding) Save 300 10 # 300 seconds At least 10 keys are changed save 60 10000 # 60 seconds at least 10000 keys are changedCopy the code

These configurations do not conflict, and any one of them will be triggered.

  • Advantages of RDB mode
  • RDB is a compact single file that can be easily transported to another remote data center, making it ideal for disaster recovery.
  • Compared to AOF, RDB is faster for recovering large data sets.
  • Disadvantages of RDB
  • Java learning notes sharing address: Redis field notes
  • RDB data cannot be persisted in real time/second level. Because bgSave needs to be run every time

Fork to create a child process.

  • Do backups at regular intervals, so if redis goes down unexpectedly, the last is lost

All changes made after a snapshot (data lost). If the data is relatively important and you want to minimize the loss, you can use AOF for persistence.

Demonstration of RDB persistence

If we follow normal procedures, it is very difficult to see failure sites where there is no persistence or persistence problems. So we need to learn to persist, or we need to manually trigger persistence when we see it. Here we mainly demonstrate two cases, one is normal data backup, the other is data loss, we restore backup data.

Pay attention to

Both of these are dangerous operations, so we need to do a little bit of setup before we do this Redis snapshot, Redis

Two commands are provided:

  • Save: blocks the current Redis server during snapshot generation. Redis cannot process other commands. if

A large amount of data in the memory will cause Redis to block for a long time. This command is not recommended in the production environment. To solve this problem, Redis offers a second way.

  • Bgsave: When bgSave is executed, Redis asynchronously takes snapshots in the background and responds to client requests

O. The Redis process forks to create a copy-on-write sub-process. The RDB persistence process is responsible for the sub-process and ends automatically after it is complete. It does not record subsequent commands after fork. Blocking occurs only during the fork phase, which is usually very short. You can use the lastsave command to view the time when the last snapshot was successfully created.

Use Shutdown for persistence

Let’s start by setting the following values in the Redis library

Set k1 1 set k2 2 set k3 3 set k4 4 set k5 5 # Redis.conf = redis.conf = redis.conf = redis.conf = redis.conf = redis.conf = redis.confCopy the code

After performing the above steps, we can see that our data is all there, indicating that our triggered backup was successful.

Use flushall to simulate data loss

This operation has certain risks. If it is a demonstration exercise, there will be no problems according to the operation, but it should be carefully operated in production. Before doing this, it is important to back up the persistence problem dump.rdb

RDB dump dump. RDB dump. Bak dump dump. View the previously stored Kye keys *Copy the code

When we started looking again, we found that our data was missing

Recover lost data

Dump. RDB rm -rf./dump. RDB RDB mv dump.rdb.bak dump.rdb # Redis-server redis.conf # redis-server redis.conf # redis-server redis.conf # redis-server redis.bak  keys *Copy the code

After that, the data we looked at appears!

AOF Append Only File

AOF: Redis is disabled by default. AOF records each write operation in the form of a log and appends it to a file. When enabled, commands that change Redis data are written to AOF files. When Redis restarts, write instructions are executed from front to back according to the log file contents to complete data recovery.

This mode is disabled by default. You need to modify the following configuration

Is the data persisted to disk in real time?

Due to the caching mechanism of the operating system, AOF data is not actually written to the hard disk, but enters the system’s hard disk cache. When do I write the contents of a buffer to an AOF file?

There are three preservation rules for AOF

AOF persistence policy (hard disk cache to disk), default everysec

  • No indicates that fsync is not executed. Data is synchronized to disks by the operating system, which is fast but insecure.
  • Always indicates that fsync is executed for each write to ensure data synchronization to disks, which is inefficient.
  • Everysec indicates that fsync is performed once per second, which may result in the loss of 1s of data. Usually choose Everysec,

Both safety and efficiency.

What can I do if the file is getting bigger and bigger?

AOF persistence means that Redis continuously records write commands into AOF files. As Redis continues, AOF files will become larger and larger. Larger files will occupy more server memory and require longer AOF recovery time.

This can be overridden using the command bgrewriteaof. AOF file rewriting is not to rearrange the original file, but to directly read the server’s existing key and value pairs, and then replace the previous multiple commands that recorded this key and value pair with a command, and generate a new file to replace the original AOF file.

AOF specifies the size to start rewriting

  • Auto-aof-rewrite-percentage: The default value is 100. The aOF automatic rewrite configuration allows Redis to call bgreWriteAOF to rewrite log files when the current AOF file size exceeds a certain percentage of the size of the last aOF file that was rewritten. When the current AOF file size is twice the AOF file size obtained from the last log rewrite (set to 100), the new log rewrite process is automatically started.
  • Auto-aof -rewrite-min-size: the default value is 64M. Set the minimum AOF file size allowed to be overwritten to avoid overwriting when the agreed percentage is still too small.
  • Advantages of AOF
  • The AOF persistent method provides multiple synchronization frequencies, and even if you use the default synchronization frequency once per second, Redis will lose at most one second of data.
  • Disadvantages of the AOF approach
  • For Redis with the same data, AOF files are generally larger than RDF files (RDB stores snapshots of the data).
  • While AOF offers a variety of synchronization frequencies, a frequency of synchronization per second by default also provides high performance. RDB provides better performance guarantees than AOF in high concurrency cases.

Compare the two schemes

So how do we choose between AOF and RDB persistence?

If you can tolerate the loss of data for a short period of time, there is no doubt that RDB is best. Periodic SNAPSHOTS are very convenient for database backups, and RDB can recover data sets faster than AOF can. Otherwise, rewrite using AOF. However, it is generally recommended not to use either persistence mechanism alone, but to use both. In this case, when Redis restarts, AOF files will be loaded first to restore the original

Because in general, AOF files hold more complete data sets than RDB files do.