Redis master/slave synchronization (I) prior to version 2.8

Recently, one of our services had to solve the problem of synchronizing data from a master node to a slave node, so WE looked at how Redis implemented the master slave synchronization, to summarize

Today, how was it implemented prior to version 2.8

There are two main ways to implement master-slave replication in Redis

1. Set the Slaveof option in the configuration file

• Configure the Slaveof option in the slave server’s configuration file
• The configuration is as follows:

slaveof <masterip> <masterport>
Copy the code

• Masterip is the IP address of the primary redis server
• Masterport is the Redis service port for the master redis service

2. Run the slaveof command on the client from the server

• Using the client, connect to the slave server
• Send the slaveof command on the slave server
• The command is as follows:

slaveof masterip masterport
Copy the code

• Masterip is the IP address of the primary redis server
• Masterport is the Redis service port for the master redis service

The principle of redis master-slave synchronization

• The master/slave synchronization feature of Redis is divided into two main operations
• Sync operation
• Command propagate for both operations
• The sync operation is used to update the status of the slave server to that of the master server. That is, all data on the master server is synchronized to the slave server
• After the completion of the sync operation, the master server passes all the commands that are executed to the slave server to keep the master and slave consistent

Let’s start with the sync operation

• When the slave server sends the slaveof command to the master server to synchronize the master service, the slave server performs the sync operation first. The sync operation updates the status of the slave server to that of the master server. That is, all data on the master server is synchronized to the slave server
• The flow of sync is as follows

Sync operation flow

• 1. The secondary server sends the slaveof command to the primary server
• 2. The primary server receives the slaveof command
• 3. Run the BGSAVE command
• 4. Generate an RDB file in the background
• 5. Use a buffer to record all commands executed since the beginning
• 6. The BGSAVE command is executed on the primary server
• 7. The primary server sends the RDB file generated by the BGSAVE command to the secondary server
• 8. Accept the RDB file from the server and load it
• 9. Update your database status to the database status when the master server runs the BGSAVE command
• 10. The master server sends all write commands recorded in the buffer to the slave server
• 11. Run these commands from the server to complete the synchronization

Now for the command propagate

• After the sync operation is complete, the database status of the primary and secondary servers is consistent
• Then the master server will send the write command that it executes to the slave server, and the slave server will execute the command that the master sends to achieve synchronization, which is the command propagate operation

Here’s an example of the overall process

Defects in master/slave synchronization prior to version 2.8

• Prior to version 2.8, master/slave synchronization had some drawbacks
• Follow the table above
• If it is the first master/slave synchronization, there is no problem
• If K1 and K2 data are synchronized after the sync operation is completed, at this time, the command propagation stage is in between the master and slave servers, the K3 and K4 commands are transmitted, and then the connection between the master and slave servers is disconnected
• After the connection between the primary and secondary servers is re-established, the synchronization between the primary and secondary servers starts again
• During the sync operation at this time, the master server generates an RDB file. The RDB file contains keys starting from K1 in the first synchronization, including K1, K2, K3, and K4
• While the sync operation can still be done, there are many keys that do not need to be synchronized again
• Since the main server needs to generate RDB files during the sync operation, this operation takes up a lot of resources of the server: CPU, memory, IO
• The transfer of RDB files between the master and slave servers consumes server bandwidth
• Loading RDB files from the server also consumes a large amount of server resources: CPU, memory, AND IO
• Therefore, no unnecessary sync operation can consume server resources and degrade service performance

Today, I talked about the mode before 2.8 in master/slave synchronization of Redis, and finally I talked about some defects in the mode before 2.8. Next time, I will talk about the mode after 2.8. Welcome everyone to communicate and point out some mistakes in the article, so that I can deepen my understanding, I hope you have no bugs, thank you!