The previous article covered the use and underlying principles of Redis’s simple data structures. In this article, we will discuss how Redis can ensure high availability.

Data persistence

We know that although the performance of the single Redis is very good, the single Redis can withstand 10W QPS, thanks to its fast memory-based read and write operation, so what if Redis suddenly dies at some time? We need a persistence mechanism to hold data in memory, otherwise it will be lost directly.

Redis has two ways to persist data, respectively Redis Database (RDB) and AOF (Append Only File). You can simply think of RDB as a snapshot of data in Redis memory ata certain point in time. AOF is the set of all instructions that record all memory data modification (that is, the set of Redis instructions), and these two methods will generate corresponding files and land on disk to realize data persistence and facilitate the next recovery.

Let’s take a look at each of these persistence scenarios.

RDB

There are two ways to create an RDB snapshot in Redis, one is to use save and the other is to bgSave, but the underlying implementation calls the same function called rdbSave, just in a different way.

Generating methods

save

The save command calls the rdbsave method directly, which blocks the main Redis process until the snapshot file is generated.

void saveCommand(client *c) {
    if(server.rdb_child_pid ! =- 1) {
        addReplyError(c,"Background save already in progress");
        return;
    }
 rdbSaveInfo rsi, *rsiptr;  rsiptr = rdbPopulateSaveInfo(&rsi);  if (rdbSave(server.rdb_filename,rsiptr) == C_OK) {  addReply(c,shared.ok);  } else {  addReply(c,shared.err);  } } Copy the code

bgsave

The bgSave command forks a child process that invokes rdbSave. The parent process continues to respond to read and write requests from the client. After the child process finishes generating the RDB file, it sends a signal to the parent process to inform the parent process that the save is complete.

/* BGSAVE [SCHEDULE] */
void bgsaveCommand(client *c) {
    int schedule = 0;

    /* The SCHEDULE option changes the behavior of BGSAVE when an AOF rewrite  * is in progress. Instead of returning an error a BGSAVE gets scheduled. */  if (c->argc > 1) {  if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"schedule")) {  schedule = 1;  } else {  addReply(c,shared.syntaxerr);  return;  }  }   rdbSaveInfo rsi, *rsiptr;  rsiptr = rdbPopulateSaveInfo(&rsi);   if(server.rdb_child_pid ! =- 1) {  addReplyError(c,"Background save already in progress");  } else if (hasActiveChildProcess()) {  if (schedule) {  server.rdb_bgsave_scheduled = 1;  addReplyStatus(c,"Background saving scheduled");  } else {  addReplyError(c,  "Another child process is active (AOF?) : can't BGSAVE right now. "  "Use BGSAVE SCHEDULE in order to schedule a BGSAVE whenever "  "possible.");  }  } else if (rdbSaveBackground(server.rdb_filename,rsiptr) == C_OK) {  addReplyStatus(c,"Background saving started");  } else {  addReply(c,shared.err);  } } Copy the code

This is why Redis is single-threaded but can provide services while generating RDB files. Fork is the main method of creating processes on Unix systems. It copies all the data of the parent process to the child process, and the parent process shares the memory space.

After the fork, the operating system kernel to the parent of all set to read only memory, only when writing data occurs, page abort will happen, the kernel will make a copy of the corresponding pages, hold each a copy of a parent and child process, so in the process of generating RDB, thanks to the COW, memory dirty pages will gradually from the child process.

Is it possible that during the bgSave call, I can invoke the save command again, which will generate two RDB files?

In fact, when the save command is called, Redis determines whether bgSave is being executed, and if it is executing, the server can no longer call the underlying rDBSave function, thus avoiding resource contention between the two commands.

For example, in the save command, there is the following judgment:

if(server.rdb_child_pid ! =- 1) {
  addReplyError(c,"Background save already in progress");
  return;
}
Copy the code

In BGSave, there are the following judgments:

if(server.rdb_child_pid ! =- 1) {
  addReplyError(c,"Background save already in progress");
} else if (hasActiveChildProcess()) {
.}
Copy the code

It can be seen that they are all judgments of the same variable, as follows:

pid_t rdb_child_pid; /* PID of RDB saving child */
Copy the code

In other words, when the save or BGSave command is invoked, the bgSave command is checked in advance to see if it is still running, and if it is running, the bgSave command is not executed. The save command itself is blocked. If other commands come in at this time, they will be blocked until the save command is executed.

How can I use the RDB file after it is generated?

When Redis starts the server, it calls the rdbLoad function and loads the generated RDB file into memory. During the load, it processes the incoming request once every 1000 keys are loaded. But it only processes publish, subscribe, psubscribe, unsubscribe, and punsubscribe. All other requests return an error until loading is complete.

You play so well, what are the pros and cons of RDB?

advantages

RDB policies can be flexibly configured for cycles, depending on what backup policy you want. Such as:

  • The data generated within the last 24 hours is generated every hour
  • Data for the latest week is generated every day
  • Data for the latest month is generated every day

Based on this policy, you can quickly recover the data of a certain period of time.

Second, RDB is great for cold backup, you can store RDB files and transfer them to other storage media. Even cross-cloud storage, such as OSS and S3 at the same time, cross-cloud storage makes data backup more robust.

Also, rDB-based recovery is faster than AOF, because AOF is a Redis instruction, and RDB is the final look of the data. With a large amount of data, all AOF instructions are played more slowly than RDB.

disadvantages

RDB as a data persistence solution is possible, but if you want to achieve the high availability of Redis through RDB, RDB is not so suitable.

If Redis hangs before the RDB file is generated, the data added since the last RDB file is successfully generated will be lost and cannot be retrieved.

In addition, if the amount of data in memory is very large, even if the RDB is forked, it also needs to occupy the CPU resources of the machine, and many abnormal interrupts may occur, and the whole Redis may stop responding for hundreds of milliseconds.

AOF

As mentioned above, RDB does not satisfy the high availability of Redis. Because in some cases, data is permanently lost over a period of time, let’s talk about another solution, AOF. First we need to understand that the RDB is a snapshot of the data in the current Redis Server, while the AOF is a record of the change order (all fetch operations are not recorded, because the current Redis data is not changed).

But because of this, AOF files are larger than RDB files. Let’s talk about a Redis command request from the client to the AOF file.

AOF recording process

First, the Redis client and server need to communicate, the client sends not the string we write, but the special protocol text. If you’re familiar with Thrift or Protobuf you should be able to understand this protocol.

For example, if you run the SET KEY VALUE command, it will become “*3\r\n$3\r\nSET\r\n$3\r\nKEY\r\n$5\r\nVALUE\r\n” when passed to the server.

The Redis server then selects the appropriate handler for processing based on the content of the protocol text. After the client sends the command to the Redis server, the command will be propagated to the AOF program as long as the command is successfully executed.

Note that the protocol text will not be written to disk immediately after it is propagated to the AOF program, because frequent I/O operations will incur huge overhead and greatly reduce the performance of Redis. The protocol text will be written to the AOF_BUF in the Redis server, also known as the WRITE buffer of AOF.

You wrote all this into the buffer. When will it land?

The flushAppendOnlyFile function is called every time serverCron is executed.

This command calls write to write buffer data to the AOF file, but it still doesn’t actually hit disk. In order to improve the efficiency of file writing, the OS temporarily writes data to the buffer of the OS memory. When the buffer is full or exceeds the specified time, the OS calls fsync or sdatasync to actually write the contents of the buffer to the disk.

But if the machine goes down during that time, the data will still be lost. So if you want to actually save the AOF file to disk, you have to call the two functions mentioned above.

ServerCron

role

Now let’s talk more specifically about the serverCron function, which is mainly used to handle general tasks in Redis.

What are routine tasks?

As mentioned above, AOF writes to the buffer. Each time serverCron executes, the AOF in the buffer is written to a file (of course, the OS writes to its own buffer). The rest is like AOF and RDB persistence operations, master-slave synchronization and cluster-related operations, cleaning up failed clients, expired keys, and so on.

So how often does this CRon run?

Many blogs are directly given the conclusion, 100ms execution once, said without evidence, we directly strove source code. The following is the function definition for serverCron.

/* This is our timer interrupt, called server.hz times per second.
*...* /
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
. server.hz = server.config_hz; } Copy the code

To avoid distracting you, I’ve omitted code and comments that aren’t currently useful to us. Called server.hz times per second. The serverCron function will call server.hz several times per second.

server.hz

I’m sure you all know the Hertz unit, which is the Si unit of frequency, indicating the number of times each periodic event occurs. So, we know that this configuration item is used to control the frequency of periodic events.

The assignment is given in the function above, and you can see that its initial value comes from the redis.conf configuration file. Let’s take a look at the configuration.

# Redis calls an internal function to perform many background tasks, like
# closing connections of clients in timeout, purging expired keys that are
# never requested, and so forth.
#
# Not all tasks are performed with the same frequency, but Redis checks for
# tasks to perform according to the specified "hz" value. # # By default "hz" is set to 10. Raising the value will use more CPU when # Redis is idle, but at the same time will make Redis more responsive when # there are many keys expiring at the same time, and timeouts may be # handled with more precision. # # The range is between 1 and 500, however a value over 100 is usually not # a good idea. Most users should use the default of 10 and raise this up to # 100 only in environments where very low latency is required. hz 10 Copy the code

Simply extracting useful information, Redis will internally call functions to perform many background tasks, and the frequency of calling these functions is determined by this Hz, which defaults to 10. That is, the serverCron function mentioned above executes 10 times a second, which averages out to one call every 100ms (1000ms/10).

Write policy

If the Redis AOF is already in the OS buffer, if it goes down, the AOF data will also be lost.

You can’t, so what’s the point of persistence? How can data not be lost?

There are three strategies for writing AOF logs:

  • Always Each command is written to a file and synchronized to disk
  • Everysec synchronizes data to disk every second
  • Do not force write, wait for OS to decide when to write

It is obvious that always policy is not advisable in real production environment. Every command writes files, which will cause great IO overhead, occupy a lot of resources of Redis server and reduce the service efficiency of Redis.

With Everysec, even if there was a power outage and the machine went down, I only lost data for a second at most.

However, NO is completely scheduled by the operating system, which may lose more data.

🐂🍺, that this AOF file how to use, how to restore?

As mentioned above, the AOF file is a record of all the write commands from the client, so the server only needs to read and replay it to restore the state of Redis.

However, Redis commands can only be executed in the context of the client, so Redis builds a dummy client with no network connection to execute commands until the command is executed.

Old iron, you this not good, in case of AOF log data volume is very large, you this is not to restore a very long time, that service is not unavailable?

Indeed, as the server runs, the volume of AOF data increases and the playback takes longer and longer. So Redis has a mechanism for slimming down AOF files.

Despite the name, you’re actually doing something that has nothing to do with the AOF files you’ve already generated.

Slimming is achieved by reading the current data state of the Redis server, of course, when the server is running properly. You can also think of it as a snapshot, but instead of a real binary file, you can directly set the snapshot value command.

For example, if you have a Redis key called test and its history is 1 -> 3 -> 5 -> 7 -> 9, then a normal AOF file would record 5 Redis instructions. AOF Rewrite would just write test=9.

The previous AOF file is still written as usual and can be replaced when the new AOF file is generated.

Are you fucking kidding me? Rewrite while the server was still processing normal requests. If you changed the state of the server, your slimmed AOF file would be inconsistent.

This does happen, but Redis solves the problem with an AOF override buffer.

When rewrite started, Redis would fork a child to do the downsizing of AOF, leaving the parent to handle requests as normal. The AOF rewrite buffer is used after rewrite has created its children, and the Redis server sends the write instructions to two places at once:

  1. Aof_buf, which is the write buffer for the AOF file mentioned above
  2. AOF overwrites the buffer

You might ask, why record it in two places? As mentioned above, the regular AOF file is still generated when Redis performs the thin operation, so the new Redis instruction must be sent to the write buffer.

The purpose of sending the AOF override buffer is to replace the changes made to the Redis state during the slimming operation, so that the slimming AOF file state is guaranteed to be consistent with the Redis state. In general, this is to ensure that the data state in the slimmed AOF file is consistent with the memory state of Redis at the time.

End

About Redis data persistence, let’s talk about so much, the next phase of the plan should be to talk about Redis high availability related mechanism, interested can wechat search “SH full stack notes” continue to pay attention to, the public account will be pushed first than other platforms.

Past Articles:

  • Redis Basics – Dissects basic data structures and their usage
  • Brief introduction to the JVM and garbage collection
  • Take a quick look at K8S and set up your own cluster
  • WebAssembly is all about getting started – Learn about WASM
  • Understand basic principles of InnoDB

This article is formatted using MDNICE