Recently, one of our services had to solve the problem of synchronizing data from a master node to a slave node, so WE looked at how Redis implemented the master slave synchronization, to summarize
Today, how was it implemented prior to version 2.8
There are two main ways to implement master-slave replication in Redis
1. Set the Slaveof option in the configuration file
- Configure the Slaveof option in the slave server’s configuration file
- The configuration is as follows:
slaveof <masterip> <masterport>
Copy the code- Masterip is the IP address of the primary redis server
- Masterport is the Redis service port for the master redis service
2. Run the slaveof command on the client from the server
- Using the client, connect to the slave server
- Send the slaveof command on the slave server
- The command is as follows:
slaveof masterip masterport
Copy the code- Masterip is the IP address of the primary redis server
- Masterport is the Redis service port for the master redis service
The principle of redis master-slave synchronization
- The master/slave synchronization feature of Redis is divided into two main operations
- Sync operation
- Command propagate for both operations
- The sync operation is used to update the status of the slave server to that of the master server. That is, all data on the master server is synchronized to the slave server
- After the completion of the sync operation, the master server passes all the commands that are executed to the slave server to keep the master and slave consistent
Let’s start with the sync operation
- When the slave server sends the slaveof command to the master server to synchronize the master service, the slave server performs the sync operation first. The sync operation updates the status of the slave server to that of the master server. That is, all data on the master server is synchronized to the slave server
- The flow of sync is as follows
Sync operation flow
- 1. The secondary server sends the slaveof command to the primary server
- 2. The primary server receives the slaveof command
- 3. Run the BGSAVE command
- 4. Generate an RDB file in the background
- 5. Use a buffer to record all commands executed since the beginning
- 6. The BGSAVE command is executed on the primary server
- 7. The primary server sends the RDB file generated by the BGSAVE command to the secondary server
- 8. Accept the RDB file from the server and load it
- 9. Update your database status to the database status when the master server runs the BGSAVE command
- 10. The master server sends all write commands recorded in the buffer to the slave server
- 11. Run these commands from the server to complete the synchronization
Now for the command propagate
- After the sync operation is complete, the database status of the primary and secondary servers is consistent
- Then the master server will send the write command that it executes to the slave server, and the slave server will execute the command that the master sends to achieve synchronization, which is the command propagate operation
Here’s an example of the overall process
| time | The primary server | From the server |
|---|---|---|
| T0 | Run the SET K1 V1 command | |
| T1 | Run the SET K2 V2 command | |
| T2 | Send the slaveof command to the primary server | |
| T3 | 1. After receiving the slaveof command, the primary server starts to run BGSAVE 3. Generate an RDB file in the background with K1, K2 4. Use a buffer to record all commands executed from the start | |
| T4 | SET K3 V3 and write the command to the buffer | |
| T5 | SET K4 V4 and write the command to the buffer | |
| T6 | When the BGSAVE command completes, the RDB file is sent to the slave server | |
| T7 | Received and loaded the RDB file sent by the master server, and obtained K1 and K2 data | |
| T8 | Send SET K3 V3, SET K4 V4 in the buffer to the slave server | |
| T9 | Received and executed two commands from the master server, and executed the SET K3 V3 and SET K4 V4 commands | |
| T10 | Synchronization (the sync operation is complete), and there are four keys K1, K2, K3, and K4 in the primary server | Synchronization (sync operation is complete). At this time, there are four keys K1, K2, K3, and K4 in the master server, and the database status is synchronized consistently |
| T11 | Execute the SET K5 V5 command and propagate the command to the slave service | Execute the SET K5 V5 command passed from the master server |
| T12 | Execute the SET K6 V6 command and propagate the command to the slave service | Execute the SET K6 V6 command from the main server |
| T13 | The primary and secondary servers are disconnected and reconnected successfully | The primary and secondary servers are disconnected and reconnected successfully |
| T14 | Restart the sync operation and repeat the process |
Defects in master/slave synchronization prior to version 2.8
- Prior to version 2.8, master/slave synchronization had some drawbacks
- Follow the table above
- If it is the first master/slave synchronization, there is no problem
- If K1 and K2 data are synchronized after the sync operation is completed, at this time, the command propagation stage is in between the master and slave servers, the K3 and K4 commands are transmitted, and then the connection between the master and slave servers is disconnected
- After the connection between the primary and secondary servers is re-established, the synchronization between the primary and secondary servers starts again
- During the sync operation at this time, the master server generates an RDB file. The RDB file contains keys starting from K1 in the first synchronization, including K1, K2, K3, and K4
- While the sync operation can still be done, there are many keys that do not need to be synchronized again
- Since the main server needs to generate RDB files during the sync operation, this operation takes up a lot of resources of the server: CPU, memory, IO
- The transfer of RDB files between the master and slave servers consumes server bandwidth
- Loading RDB files from the server also consumes a large amount of server resources: CPU, memory, AND IO
- Therefore, no unnecessary sync operation can consume server resources and degrade service performance
Today, I talked about the mode before 2.8 in master/slave synchronization of Redis, and finally I talked about some defects in the mode before 2.8. Next time, I will talk about the mode after 2.8. Welcome everyone to communicate and point out some mistakes in the article, so that I can deepen my understanding, I hope you have no bugs, thank you!