Hello everyone, I’m Xiao CAI. A man who wants to be a man who talks architecture! If you also want to be the person I want to be, or point a concern to do a companion, let small dishes no longer lonely!

This article focuses on Redis persistence

Refer to it if necessary

If it is helpful, do not forget the Sunday

Wechat public number has been opened, xiao CAI Liang, did not pay attention to the students remember to pay attention to oh!

Redis is definitely a popular interview direction as I am going further and further on my way to interview. How many data structures are there? How to implement delay queuing? What is the elimination mechanism? Almost asked numbness, these questions are often around the brain. A job interview is a very difficult job interview. What is the Redis persistence strategy?

Ask a question at the beginning, I believe that there are not a few students who are asked about Redis persistence, and there are certainly not a few students who answered correctly. Some friends will definitely open their mouth when talking about Redis persistence. After all, there are only two concepts of AOF and RDB, as long as you prepare for the interview, you will not be asked too badly. But do you really get it or are you just memorizing it for the interview? Do you know which words AOF and RDB are short for? Have you ever done it? Do you really think the interviewer can’t tell if you’re memorizing or practicing? If you answered half of these questions correctly, read on to find out if you can answer them with some tips in mind.

Redis persistence

What is Redis persistence?

Let’s not remember to go in the direction of the solution, first understand the meaning of the problem.

Persistence is about making data last forever. So what is Redis persistence? That is to write Redis data stored in memory to disk, in case of service downtime memory data loss problem. Then some partners say, that disk is damaged, how to persist data? Even if multi-point backup can solve the problem of disk corruption, what about multi-point loss? This article is about Redis persistency. Don’t expect to talk about it for half an hour.

The Redis persistence problem is explained in several ways.

It’s a three-point, three-step strategy to solve your persistence problem.

A, RDB

Let’s start with one of the opening questions, what is RDB for? Redis Database Backup File (RDB) – Redis data Backup file, also called Redis data snapshot.

This is used to record all the data in memory to disk, and when the Redis instance fails and restarts, the snapshot file is read from disk to recover the data. The first concept can solve the Redis persistence problem

Before we learn RDB, we will understand the two core concepts fork and cow, which we will explain below.

RDB is the default persistence mechanism in Redis. It saves data in memory to disk as a snapshot for a certain period of time. It generates a special type of file.

Let’s start with two configuration parameters in the configuration file, starting with the Save configuration.

This command is used by the main Redis process to execute the RDB and blocks all commands

We found the configuration for SAVA in the configuration file

1.

dbfilename dump.rdb
Copy the code

This configuration item is used to define the RDB file name (note that this name cannot be defined as a path, only the file name).

After executing the save command, you can see a dump. RDB file in the redis folder

2,

save <seconds> <changes>
Copy the code

The purpose of this configuration item is to define how many changes occur over a period of time before bgSave is executed, or if save “”, RDB is disabled.

Let’s test the save configuration next

Dbfilename dump-test. RDB # dump-test. RDB save 3600 1 # BgSave is executed if a change is made within 3600 secondsCopy the code

We enter the operation through redis- CLI

Then we exit and see the dump-test.rdb file we just generated in the current directory

This indicates that our configuration works, and we directly restart Redis to see if the data we just saved still exists

When you see our data, redis persistence is successful. Then we delete the dump-test. RDB file and restart Redis

Redis starts with.rdb to restore file data. What about bgSave that we’ve been talking about, how does bgSave work?

We talked about fork and cow in the previous two concepts. I don’t know if I still remember, but these two concepts are the key ~!

Bgsave starts by forking the main process to get a new child process that shares the main process’s memory data. The child process writes data to a temporary.rdb file on disk. When the child process finishes writing the temporary file, it replaces the original.rdb file. This is the heart of fork, what is cow? Copy-on-write, cow stands for copy-on-write. When the main process reads data, it accesses the shared memory. When the main process writes data, it copies data and writes data.

The specific process is as follows:

What are the advantages of this persistence approach?

  • For persistence, there’s only onedump.rdbfile
  • Dr Is good and files can be saved to secure disks
  • To maximize performance, fork the child process to complete the write operation and let the main process continue processing commands, maximizing IO and ensuring high performance of Redis

There are drawbacks:

  • Data security is low. RDB is to persist (SAVE) for a period of time. If Redis fails during the persistence, data will be lost
  • The snapshot takes a long time. If a large amount of data is stored, the snapshot takes a long time and occupies disk space

The pros and cons of the same, appropriate use

Second, AOF

AOF stands for Append Only File. The effect is that every write command processed by Redis is recorded in an AOF file, which can be viewed as a command log file.

This feature is turned off by default. You can see the AOF configuration items in the redis.conf file

1.

Appendonly yes # enable AOF logging, disabled by defaultCopy the code

2,

appendfilename "appendonly.aof"# AOF the name of the fileCopy the code

The above two configuration items are used to enable AOF logging, so there is an additional configuration item to know as well

3,

Appendfsync everysec # AOF The frequency recorded by the commandCopy the code

This configuration item has three optional values

Configuration items Brush set time advantages disadvantages
Always Synchronous brush set High reliability, almost no data loss Great performance impact
everysec Brush set per second The performance is moderate A maximum of 1 second of data is lost
no Operating system control The best performance The reliability is poor and a large amount of data may be lost

For those of you familiar with Mysql relay logs, this model is familiar.

Principle: it is to write command add AOF the end of the file, use the AOF persistent need to set up sync option, to ensure that the write command the timing of synchronization to the disk file, this is because to write files will not be immediately synchronized to the disk, memory but stored in the cache area first, and then decide when to synchronize to disk by the operating system.

Let’s turn on the AOF recording function to check:

We can see that every operation has been recorded in the AOF file, we can also get the data just stored by restarting Redis, indicating that the persistence is effective ~

Do we look at the AOF records above and think it’s neat? But more clutter in the online environment is bad, because the file is mainly for the machine to see, not for us, so it’s best to compress it.

To address the growing size of AOF files, you can send the bgrewriteAof command to Redis, which will rewrite the AOF file to make it as small as possible by removing redundant commands from the AOF file. Bgrewriteaof works very similar to how BGSave creates snapshots: Redis creates a child process that is then responsible for rewriting the AOF file. Because AOF file rewrite also requires child processes, the performance and memory footprint problems associated with creating child processes in snapshot persistence also exist in AOF persistence.

Since manual compression exists, automatic compression also exists, which brings us to the two configuration items in the configuration file

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
Copy the code

This configuration item means that Redis will run the bgrewriteaof command when the AOF file size is greater than 64MB and the AOF file size is at least twice (100%) larger than the size since the last rewrite.

To sum up, its advantages are as follows:

  • Data security. AOF persistence can be configuredappendfsyncProperties of thealwaysEach write command operation is recordedAOFOnce in a file
  • Consistency. Write files through the Append model, even if the server goes downredis-check-aofTools to solve data consistency problems

Disadvantages are as follows:

  • AOF files are larger than RDB files and are slower to recover
  • Large data sets are less efficient to start than RDB files

It is equally good and bad, use as appropriate

Three, the difference between the two

Having introduced the two, let’s review the differences between the two.

In terms of RDB AOF
Persistent mode Periodically snapshots are taken for the entire memory Record the commands executed each time
Data integrity Incomplete, lost between backups Relatively complete. Depends on the flush strategy
The file size There will be compression, file size is small Record the command, the file size is very large
Downtime recovery speed soon slow
Data recovery priority Low because data integrity is not as good as AOF High, because data integrity is higher
System Resource Usage High, large CPU and memory consumption Low, mainly disk I/O resources. And AOF rewriting will consume a lot of CPU and memory resources
Usage scenarios Can tolerate several minutes of data loss, pursuit of faster startup speed High requirements on data security

After reading the above, you should have some understanding of both persistence mechanisms. Both have advantages and disadvantages, so how should we choose? Here are some suggestions

  1. If you can tolerate data loss for a short period of time, you can use the RDB mechanism to generate RDB snapshots on a regular basis, and RDB can recover data sets faster than AOF can
  2. However, if only RDB mechanism is used, a lot of data may be lost. Therefore, we need to use AOF and RDB persistence mechanism in a comprehensive way. AOF is used to ensure that data is not lost as the first choice for data recovery. Use RDB for varying degrees of cold backup and for quick data recovery in cases where AOF files are lost or corrupted and unavailable
  3. We can use RDB to quickly recover data and AOF to complete data

Here we are talking about the Redis persistence mechanism configuration, through this article learning, I believe that when the interview will not be so at a loss ~!

Don’t talk, don’t be lazy, and xiao CAI do a blowing bull X do architecture of the program ape ~ point a concern to do a companion, let xiao CAI no longer lonely. See you later!

Today you work harder, tomorrow you will be able to say less words! I am xiao CAI, a man who grows stronger with you. 💋 wechat public number has been opened, xiao CAI Liang, did not pay attention to the students remember to pay attention to oh!