Redis special topic: learn the principle of master slave replication!

Wechat search public number “mark road”, thank you for your attention!

In distributed environment, data Replica and Replication, as effective means to improve system availability and read/write performance, are widely used in various distributed systems, and Redis is no exception. Redis master-slave replication, a distributed system based on the mode of one master and multiple slaves, is the basis for Redis to build high availability clusters (sentry mode and Cluster mode), providing strong support for fault tolerance and failover. This article will introduce the following:

Master-slave sample

Redis sets up the master-slave structure in three ways:

In redis.confreplicaof <masterip> <masterport>, such asReplicaof 192.168.1.2 instead of 6379Will be created with instances192.168.1.2 instead: 6379The relationship between master and slave; This is equivalent to adding parameters to the Redis startup commandreplicaofParameters.
Add “–replicaof” to the redis-server startup parameter.
To connect to the Redis service using the redis cli client, runreplicaof <masterip> <masterport>Command.Note: If Redis restarts, the master-slave relationship cannot be re-established.

Note: Since Redis 5.0, Replicaof has been replaced by Slaveof. Due to some racist bizarre requirements, Redis has updated its instructions to be compatible with Slaveof.

Set up the master/slave structure

Docker Compose is used in this example to build a master-slave structure with one master and two slaves.

newdocker-compose.ymlAdd the following configuration information.

version: "3"
services:
  redis-master:
    image: Redis: 6.0
    ports: 
    - "6379:6379"
    container_name: "redis-master"
    command: redis-server
    networks:
    - redis-master-slave
  redis-slave-1:
    image: Redis: 6.0
    ports:
    - "6380:6379"
    container_name: "redis-slave-1"
    command: redis-server --slaveof redis-master 6379 
    depends_on: 
    - redis-master
    networks:
    - redis-master-slave
  redis-slave-2:
    image: Redis: 6.0
    ports:
    - "6381:6379"
    container_name: "redis-slave-2"
    command: redis-server --slaveof redis-master 6379 
    depends_on: 
    - redis-master
    networks:
    - redis-master-slave
networks:
  redis-master-slave:
Copy the code

CD Go to the current directory and execute the commanddocker-compose upWhen you start docker, it looks like this:

Connect the client to the command line of the master node (localhost:6379)info replicationViewing the master/slave relationship, you can see that there are two slave nodes.

Testing data synchronization

Execute commands on the master client command lineset name abcdAnd run the command on the slave clientget name, you can see that the data synchronization is successful, as shown in the following figure:

Execute instructions in the master clientdel nameAnd then execute it on the slave clientget name, you can see that the data has been deleted.

From this process, we can see that,redis-master,redis-slave-1,redis-slave-2The three nodes establish the relationship of one master and two slaves. Furthermore, data written by the master node is successfully synchronized to the two slave nodes. The master-slave relationship example is introduced here first. Next, let’s take a look at the principle and process of data synchronization between the master and slave nodes of Redis.

How Replication works

Redis Replication is a simple, easy-to-use master-slave Replication mechanism that enables slave nodes to be identical copies of master nodes. The slave node automatically reconnects every time it is disconnected from the master node, and it always tries to achieve the same state as the master node, no matter what happens to the master node. Redis takes a number of ancillary measures to ensure data security.

There are two versions of the Redis master-slave replication technology: Before version 2.8, each time the slave node disconnects and reconnects, only full synchronization can be performed. It was redesigned after version 2.8 to introduce the concept of partial synchronization. This article will introduce the principle of master-slave replication based on Redis 6.0.

In the old version, the master/slave replication adopts the “full synchronization + command propagation” mechanism to complete data synchronization between the master and slave. In this case, a time-consuming and resource-consuming full synchronization operation is performed to achieve data consistency even if there is only a small amount of data inconsistency between the master and slave. To this end, the Redis team has introduced several mechanisms to ensure a less costly partial synchronization for master/slave replication when small amounts of data are inconsistent. Therefore, the current master-slave replication mechanism consists of three parts: full synchronization, partial synchronization, and command propagation.

For the sake of completeness, the concept of full synchronization and command propagation is introduced first.

Full synchronization: The master node creates a full RDB snapshot file and sends it to the slave node over the network connection. The slave node loads the snapshot file to recover the data and then sends the commands added in the replication backlog buffer to make the data consistent.
Command propagation: If the master-slave node remains connected, the master node continues to send command flows to the slave node to ensure that the changes in the data set of the master node also apply to the data set of the slave node. These commands include: Client write requests, key expiration, data obsolescence, and all the other operations that cause data set changes.

Synchronization principle

In master-slave replication mode, Redis uses a pair of Replicaion ids, offset, to uniquely identify the version of the data set on the Master node. To understand this concept, you need to understand the following three concepts of Redis:

Replication ID: Each master node of Redis uses a randomly generated string to indicate the state of its internal storage data at a certain point in time. “At a certain point” can be understood as the moment it becomes the master.
Offset (replication offset) : In master-slave mode, the master node continuously propagates commands that cause data set changes to the slave node. Offset represents the total number of command bytes that the master node passes to the slave node. It does not exist in isolation and needs to work with replication backlogs.
Backlog (replication backlog buffer) : This is a circular buffer that stores commands passed from the master node to the slave node. The backlog is fixed in size. It can be used for partial synchronization or for command repush during command propagation.

The concepts may be abstract, but the diagram below shows how they relate to each other.

Illustration:

The figure shows that the Redis role is Master, whose replID is XXXX, and whose current replication offset is 1010.

It has a copy backlog buffer with a size of 100 (backlog_size) and an offset of 1000 (backlog_off) from the start of the backlog. The current backlog of command bytes (backlog_hislen) is 11, corresponding to the [1000,1010] offset in the backlog.

Offset is always the same as the offset of the last byte in the backlog.

The above information is maintained by the storage of the master node. The concepts of replication ID and replication offset are also included in the slave node.

Replication ID: If the slave has never replicated with any master, it does not exist. If the connection between the slave and master is normal, the replication ID is stored inserver.masterWithin; If the slave is disconnected from the master, Redis stores it in theserver.cache_master.
Replication offset: The value is the same as the replication ID.

Server. master indicates the information about the master node that is performing the master/slave replication and is working properly. Server.cache_master is the master node information that has been properly replicated from master to slave in preparation for partial synchronization.

So what’s the use of all this stuff? As mentioned earlier, the old version required full synchronization every time the node was connected to the master node, which was inefficient, so partial synchronization was introduced. These components are designed to allow partial synchronization to be used under certain conditions to improve efficiency. So how do they work?

After the secondary node is connected to the primary node, the secondary node needs to run commandsPSYNC <replid> <offset>When a synchronization request is made to the master node, the slave node willserver.cache_masterTake two parameters from the command and send it to the master node. Note: If the slave node is brand new and has never been replicated master-slave with any master node, the special command is used:PSYNC ? - 1.
The master node receives the command to resolve the replication ID and offset in the request, and then to determine whether the current request can use partial synchronization.
To be able to use partial synchronization, two conditions need to be met (The case of multiple replication ids caused by primary/secondary switchover is not considered here) :
- The replication ID is the same as that of the primary node.
- The replication offset must be in the range between backlog_OFF and offset.
Instead of using partial synchronization, you have to use full synchronization.

To make it easier to understand, let’s look at a few examples, as shown in the figure below. There are four slave nodes to initiate master-slave replication with the master node. The status of the master node and slave node has been marked in the figure. Let’s analyze it.

Redis-1: ReplID and offset are the default values, indicating that it has never been synchronized with the master node. Therefore, full synchronization is performed.
Redis-2: replID the master node is the same as the slave node, slave_offset>=backlog_off and slave_offset
Redis-3: replID the primary node is the same as the secondary node. Slave_offset
Redis-4: replID Is the same as the primary and secondary nodes. The replid is not the same as the current node, so full synchronization is performed.

To summarize: Partial synchronization is an incremental synchronization logic based on full synchronization (to get the replicate ID), which uses the cache commands in the replicate backlog to do command replay. However, due to the size of the replicate backlog, it is limited in scope. This is the same as AOF hybrid persistence and the Binlog approach to master-slave replication in mysql.

Command transmission

After the synchronization is complete, the master-slave enters the command propagation phase, and the master-slave data is consistent.

When the maste finishes executing a new write command, the propagator appends the command to the replication backlog and asynchronously sends it to the slave. The slave receives and executes the command, and updates the replication offset maintained by the slave. Command propagation is as follows:

If the slave can receive each propagated command and execute it successfully, it can maintain the same state of data as the master. However, the master does not wait for the return of the slave node. The master communicates with the slave node through the network. Due to network jitter and other factors, the slave cannot be guaranteed to receive the command during the transmission process.

During command transmission, the slave node sends heartbeat information to the master node every second in the format of REPLCONF ACK

.

Offset in the command is the latest replication offset of the slave. The master compares the offset with its own. If the slave node’s data is missing, the master node will push the missing data. If anyone knows, please let me know.

Replication Execution process

According to my understanding, the master/slave replication process is divided into three phases: preparation phase, synchronization phase, and command propagation phase. In the following figure, the overall master/slave replication process is shown on the left, and the slave replication log output is shown on the right after a slave node is added.

Preparation stage

In the preparation stage, the connection with the master is completed. The status check and identity authentication are processed between the master and slave through the command ask and answer, laying a foundation for the following data synchronization.

Set the host and port of the master node: This step is easy to understand. After receiving the Slaveof command, Redis will set the host and port of the master node and set the replication status of the service toREPL_STATE_CONNECT.
Network connection with master: Network connection with master is in periodic functionserverCronIf the service status is found to beREPL_STATE_CONNECT, will call the functionconnectWithMasterCreate a network connection with the master. After the connection is successful, change the replication status toREPL_STATE_CONNECTINGAnd triggers the callback functionsyncWithMasterThe command interaction with the master node starts. If the connection fails, wait for the next retry.
PING: After the connection with the master is created, the slave sends the PING command to the master. You can run the PING command to check whether the read/write status of the network connection is normal and whether the master can process command requests properly. If the slave times out or the master returns an error message, the slave disconnects and reconnects to the server. The server returns to PONG and proceeds to the next step.
Identity authentication: Check whether masterauth is set on the secondary server. If masterauth is set, authentication is performed. If masterauth is not set, skip this step. If the identity authentication fails, the connection is disconnected and reconnected.
Sending slave port information: The slave sends its listening port number to the master server. After receiving the port number, the master records the port number in the status attribute of the slave. You can run the commandclient listLook at it.

Synchronous phase

Synchronization is the most complex phase of master-slave replication. Based on the above working principles, we know that it may be partial synchronization or full synchronization. So, the next step is to determine the synchronization method and then execute the synchronization process in different ways.

Determine the synchronization mode

In this phase, the master and slave cooperate to determine the synchronization mode. In simple terms, there are two steps: the slave sends the PSYNC command, and the master determines the synchronization mode and replies.

Slave Sends the PSYNC command. The PSYNC command assigns values to runId and offset. There are two cases that need to be distinguished:

If the slave is replicating with the current master for the first time, the default value is used:runId=? ; offset=-1;
If the slave has performed full replication with the master before, the value is assigned based on the cached master information and the replication offset.

How do I know if the slave is replicating for the first time?

The Redis core structure server’s cached_master holds information about the master node, and is null unless there has been a master/slave replication. The following code snippet from slaveTryPartialResynchronization () function:

File: / / code replication. C function: slaveTryPartialResynchronization
// Whether there is a cache of master node information
// If yes, execute if
if (server.cached_master) {
    // 这里的replid，复制id就是master的runId
    psync_replid = server.cached_master->replid;
    // Get the replication offset
    snprintf(psync_offset,sizeof(psync_offset),"%lld", server.cached_master->reploff+1);
    serverLog(LL_NOTICE,"Trying a partial resynchronization (request %s:%s).", psync_replid, psync_offset);
} else {
    // No: go else
    serverLog(LL_NOTICE,"Partial resynchronization not possible (no cached master)");
    // Set the default value:? - 1
    psync_replid = "?";
    memcpy(psync_offset,"1".3);
}

/* Send the PSYNC command to master */
reply = sendSynchronousCommand(SYNC_CMD_WRITE,conn,"PSYNC",psync_replid,psync_offset,NULL);
Copy the code

Master Determines the synchronization mode. Master receive commands by syncCommand function after treatment, and then call masterTryPartialResynchronization function judge synchronous way, from the code analysis shows that there are several ways:

Offset resolution failed, and full synchronization was performed.
The replication ids of runId and master are different, and full synchronization is performed.
Offset is not in the master backlog buffer. Full synchronization is performed.
The others meet the partial synchronization conditions and perform partial synchronization;

Next, take a detailed look at the execution process of full synchronization and partial synchronization from the master and slave perspectives.

Full synchronization process

Look at the master.After confirming that full synchronization is required, the master directly enters the processing flow. It should be mentioned here that Redis drives the whole process by relying on the status of slave in the process of full synchronization. I will first describe the process of full synchronization and the status flow of slave, and then explain:

Example Change the replication status of the slave toSLAVE_STATE_WAIT_BGSAVE_STARTWait for the BGSave operation to start and add it to the slave list;
If the current slave is the first slave, you need to generate a new Replication ID, clear Replication ID2, and initialize the Replication backlog buffer.
When the BGSave starts, the replication status of the slave is changed toSLAVE_STATE_WAIT_BGSAVE_END, the following three situations need to be considered:
- If a child process is already running and is in disk mode, reuse the current BGSave and save the buffer state. Full synchronization is then returned to the slave, offset being the replication offset when bgSave is being started.
- If a child process is running but in diskless mode, do not execute bgSave until periodic check is triggered.
- If no child process is running: enable BGSave; Then full synchronization is restored to the slave. Offset is the replication offset of the current master.
The master executes the BGSave and then processes another request, which is triggered in serverCron on successful executionbackgroundSaveDoneHandler, and callupdateSlavesWaitingBgsave, and all it does is copy all the states in turnSLAVE_STATE_WAIT_BGSAVE_ENDThe slave transmits RDB files. The slave replication status was changed toSLAVE_STATE_SEND_BULKAfter the transmission is complete, the status is changed toSLAVE_STATE_ONLINE.

On the slave side. In the preparation phase, the slave sends the psync command and waits for the reply from the master. After receiving the reply from the master, the slave starts the full synchronization process. The process is as follows:

If there are cascaded slaves, disconnect all network links with them and empty the replication backlog buffer;
Create RDB temporary file, receive file stream from master and write to it;
Stop ongoing RDB persistence and AOF persistence processes.
Rename temporary files to official RDB files and perform data loading;
Create a slave client based on the current network connection with the master and use the master as the slave client.
Set the replication ID of the slave and create a replication backlog buffer.
Enter the command propagation phase.

Partial synchronization process

Partial synchronization is much simpler than full synchronization.

The master determines that partial synchronization can be used and performs the following process:

Example Change the slave status toSLAVE_STATE_ONLINEAnd add the slave to the slave list.
Reply partial synchronization commands to slave, “+CONTINUE ReplID “;
Commands are extracted from the replication backlog buffer and sent to the slave at the offset requested by the slave.

After receiving a partial synchronization reply, the slave performs the following process:

If the master replicate ID is changed, update the replicate ID and transfer the original replicate ID to replicate ID2. If there are cascaded slaves, disconnect them and let them reconnect.
Create a slave client based on the current connection with the master, ready to receive commands.
Receive and execute commands transmitted by master;
The command transmission phase is displayed.

Command propagation phase

I’m not going to write it again.

The full text summary

This paper introduces the way of master-slave structure construction and data synchronization effect through examples, and then introduces the working principle of Redis master-slave replication in detail, that is, “full copy + partial copy + command propagation”, and then introduces the execution process of master-slave replication in detail with the source code, and finally introduces some other problems related to master-slave replication. Through these questions, it is generally introduced the master slave synchronization principle of Redis, hoping to bring help to you.

I spent almost four days writing this article, and I overturned it three times in the course, mainly trying to find a relatively clear idea to explain this problem clearly. Poor ability, still need more practice.

If you feel useful, please forward to more friends in need, thank you for reading! Code word is not easy, thank you for your support!

reference

Replication: redis. IO /topics/repl…
The Redis replication process, rounding: mp.weixin.qq.com/s/0VVYTyAI1…
Redis Series: Master/Slave Replication: juejin.cn/post/684490…
Redis Master-slave Replication: juejin.cn/post/684490…
About Master/slave Replication of Redis: juejin.cn/post/685457…
Redis High Availability – Master/Slave Replication: juejin.cn/post/684490…
Redis master/slave replication steps: juejin.cn/post/692718…
Redis master/slave replication: juejin.cn/post/684490…
The Redis source analysis of master-slave replication (4) : segmentfault.com/a/119000002…

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Redis special topic: learn the principle of master slave replication!