This is the 23rd day of my participation in the August More Text Challenge.More challenges in August

Redis persists RDB and AOF

Redis persistence

Redis is an in-memory database. If you do not save the in-memory database to disk, the data will be lost once the server is down or the REDis process exits, and the database status in the server will also be lost

So Redis provides persistence

Redis persistence is divided into RDB and AOF

RDB (Redis DatabBase)

In a master/slave replication, the RDB files are placed on the slave machine as backup

At a specified interval, a snapshot is written to the disk. When a snapshot is restored, the snapshot file is read into the memory.

Redis forks a child process for persistence,

  • The data is first written to a temporary file, which is used to replace the last persistent file after the persistence process ends

During the whole process, the main process does not perform task IO operations, which ensures extremely high performance

If large-scale data recovery is required and data integrity requirements are not sensitive and strict, RDB persistence is better and more efficient than AOF persistence.

Although RDB has high performance, data may be lost after the last persistence. Redis uses the RDB persistence mode by default, and generally does not need to be modified

save 60 3

# The filename where to dump the DB
dbfilename dump.rdb

dir ./
Copy the code

We set the redis operation to persist once if three times in 60 seconds

127.0.0.1:6379> ping
PONG
127.0.0.1:6379> config get dir
1) "dir"
2) "/root"
127.0.0.1:6379> set k1 v1
OK
127.0.0.1:6379> set k2 v2
OK
127.0.0.1:6379> set k3 v3
OK
Copy the code

The dump.db file is generated in the dir directory, which in our case is /root

Dump. RDB = dump. RDB = dump. RDB = dump

root@iZuf66y3tuzn4wp3h02t7pZ:~# rm dump. RDB -rf root@iZuf66y3tuzn4wp3h02t7pZ:~# redis-cli 127.0.0.1:6379> set P1 1 OK 127.0.0.1:6379> set p2 1 OK 127.0.0.1:6379> set p3 3 OK root@iZuf66y3tuzn4wp3h02t7pZ:~# ls dump.rdbCopy the code

Sure enough, it’s normal

Trigger mechanism for persistence

  • When the rules of save are met, persistence will be triggered. For example, the redis operation in 60s mentioned above will trigger one persistence after 3 times
  • When you run the flushall command, persistence is also triggered to generate the dump.db file
  • When exiting Redis, persistence is also triggered to generate the dump.db file

The backup automatically generates a dump.db file

How do I recover persistent files

1. You only need to place the dump.db file in the redis startup directory. When redis starts, the dump.db file will be read into the memory for data recovery

2. Check the redis startup directory to do this

127.0.0.1:6379> config get dir
1) "dir"
2) "/root"
Copy the code

For this part of the configuration, we basically do not need to modify too much, the default configuration is basically enough

The advantage of RDB

  • Suitable for large-scale data recovery
  • Data integrity requirements are not high

RDB disadvantage

  • You need to operate at regular intervals, and if Redis unexpectedly goes down, the last written data will be lost
  • It takes up some space to fork the child process

AOF persistence

What is AOF?

Write down all of our write commands, and execute all of the records in the file when restoring

AOF is another redis persistence method. It records every write operation in the form of a log and records all write operations performed by Redis. Only files can be appended, but files cannot be overwritten

When Redis starts up, it will read the AOF file to rebuild the database. That is to say, when Redis restarts, it will execute write instructions in the write order according to the contents of log files to complete data recovery

Aof holds the appendone.aof file

# Please check https://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

# appendfsync always
appendfsync everysec
# appendfsync no

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
Copy the code

auto-aof-rewrite-min-size 64mb

When the AOF file is larger than 64 MB, a child process is created to write a new AOF file

As for the configuration of AOF, basically the default configuration is used for other things. We just need to turn on the AOF mode

appendonly yes
Copy the code

Appendonly is not enabled by default, after we modify the configuration, redis-server will take effect immediately

Rewriting rule Specification

Aof defaults to unlimited appending of files, which inevitably get bigger and bigger

After redis.conf is changed to the aof mode, restart redis-server and you can see the appendone. aof file

The redis client connects to the server to operate redis, simply setting several values

Root @ iZuf66y3tuzn4wp3h02t7pZ: / # redis - cli 127.0.0.1:6379 > ping PONG 127.0.0.1:6379 > 6379 > set name xiaozhu OK 127.0.0.1: Set age 19 OK 127.0.0.1:6379> Set hobby play OK 127.0.0.1:6379> set k1 v1 OK 127.0.0.1:6379> set k2 v2 OK 127.0.0.1:6379> set K3 v3 OK 127.0.0.1:6379> shutdown not connected>Copy the code

View the appendone.aof file

Appendone.aof holds the record of our write command to redis

At this point, we consider modifying some values in appendone.aof

set
$2
k2
$2
v2
*3
$3
set
$2
k3
$2Ashdkklasdjkkv3 # modifs this lineCopy the code

Start redis-server to check whether data can be restored

Root @ iZuf66y3tuzn4wp3h02t7pZ: / # redis server/usr/local/redis/redis - 6.2.5 / redis. Conf root @ iZuf66y3tuzn4wp3h02t7pZ: / # Redis - CLI Could not connect to redis at 127.0.0.1:6379: Connection refused the not connected > root @ iZuf66y3tuzn4wp3h02t7pZ: # / ps axu | grep redis root 1251 0.0 14436 0.0 1048 PTS / 0 S+ 14:55 0:00 grep --color=auto redis root@iZuf66y3tuzn4wp3h02t7pZ:/#Copy the code

We found that redis-server failed to start because our appendone. aof file has been manually modified, and we need to repair this file. Redis provides tools to modify the aof file, redis-check-aof

Use instructions:

redis-check-aof --fix appendonly.aof

# redis-check-aof --fix appendonly.aof0x ce: Expected \r\n, got: 6864 AOF analyzed: size=223, ok_up_to=181, ok_up_to_line=47, diff=42 This will shrink the AOF from 223 bytes, with 42 bytes, to 181 bytes Continue? [y/N]: Y Successfully truncated AOF root @ iZuf66y3tuzn4wp3h02t7pZ: / # redis server/usr/local/redis/redis - 6.2.5 / redis. Conf Root @ iZuf66y3tuzn4wp3h02t7pZ: / # redis - cli 127.0.0.1:6379 > ping PONG 127.0.0.1:6379 > get k1 "v1"Copy the code

After repairing aOF file, we start redis-server again to restore disk data, recovery success, nice

Advantages and disadvantages of AOF

advantage

  • Every operation of REIDS is recorded with good file integrity
  • If you synchronize once per second, you may lose one second of data
  • Never synchronizing is the most efficient

disadvantage

  • Compared to data files, AOF files are much larger than RDB files, and repair is slower than RDB files
  • Aof runs more slowly than RDB, so redis is configured with RDB persistence by default

summary

Two types of persistence are briefly described

  • RDB persistence enables snapshot storage of data at a specified time interval

  • When the server is restarted or down, the write operation command in these records will be re-executed to recover data. The AOF command appends each write operation to the end of the file using redis protocol. Redis can also rewrite the AOF file in the background. Yes, the size of AOF files is not too large

  • If the requirement is to cache only and only expect the data to exist while the server is running, then persistence is not required

On and off of the two persistence modes

  • You can enable both persistence methods

    • In this case, the REDis reboot will load the AOF file first to recover the data, because aOF files usually hold more complete data sets than RDB
    • RDB data sets are not real-time. When using both methods, the server will only look for AOF files when restarting. Should you use only AOF files?

    The author of Redis suggests not to use only AOF files, because RDB is more suitable for backup database, because AOF is constantly changing, it is not easy to back up, when fast restart, RDB will not have aOF potential bugs, leaving RDB as a backstop mechanism

Performance recommendations

For RDB persistence

  • Since the RDB file is only used to back up data, it is recommended that the RDB file be persisted only on the slavesave 900 1
  • If you turn on AOF persistence, the benefit is that in extreme cases you don’t lose more than 2s of data, and the startup script simply loads its own AOF file, which comes at a cost
    • One of the costs of doing so is the constant I/O operations
    • The second cost is that at the end of the AOF rewrite, the blocking caused by writing new data to new files is inevitable. As long as the disk permits, the frequency of AOF rewrite should be reduced as much as possible

For AOF persistence

  • The basic size value of aOF rewrite is 64 MB, we can set it to more than 5g, the default size is more than 100% of the original size rewrite, this parameter can be set to a reasonable parameter
  • If aOF mode is not turned on, it is possible to achieve high availability with master slave replication alone, which saves a lot of IO consumption and reduces the performance fluctuation of the system due to rewriting, but this still comes at a cost
    • One of the costs is that if both the primary and secondary Redis fail (such as a power outage), the data will be lost for more than ten minutes. The startup script will also compare the primary and secondary RDB files and load the newer one

References:

redis_doc

Welcome to like, follow and favorites

Friends, your support and encouragement, I insist on sharing, improve the quality of the power

All right, that’s it for this time

Technology is open, our mentality, should be more open. Embrace change, live in the sun, and strive to move forward.

I am Nezha, welcome to like, see you next time ~