This article requires knowledge of [Redis] persistent-digging (juejin. Cn).

Replication implemented by the SYNC command

As you can see here, RadisSYNCThe replication can be divided into six steps. The first five steps can be regarded asData synchronizationAnd, finally,Command transmission.

  1. The client sends a message to the serverSLAVEOFCommand to ask the server to replicate the target server (to form a master-slave relationship). Note that the SLAVEOF command is asynchronous and the server will return OK to the client after receiving the replication command. Then the actual copy action will take place.
  2. The slave server sends messages to the master serverSYNCCommand to request a secondary server as the primary server. The slave server associates the socket connecting to the master with a file event handler dedicated to replication for subsequent RDB file reception, propagation command reception, and so on. The master server treats the slave server as one of its clients. The master and slave servers then send each otherPingCommand to test the network and authenticate. After passing, the actual copy action will be carried out.
  3. Master server executionBGSAVECommand,fockThe child process generates the RDB file and at the same time begins writing requests after that point in time to the buffer.
  4. After the RDB file is generated, it is sent to the slave server, which receives and loads the RDB file.
  5. The master server sends the incremental request for the buffer to the slave server, which executes the command after receiving it, and finally keeps the data consistent with the master server.
  6. The master server propagates the write command sent by the client to the slave server to ensure that the slave server keeps consistent with the master server (ultimate consistency guarantee).

The replication mechanism before Radis 2.8 adopts the above process, but in this way, every time the secondary server is connected to the primary server, the primary server needs to regenerate the RDB file. In the case of disconnection and reconnection, resource consumption is large and unnecessary. The slave server holds most of the master server’s data. Other disadvantages include resource consumption and performance issues for RDB file generation mentioned in persistence, network consumption for transferring RDB files, and so on.

Replication implemented by the PSYNC command

The PSYNC command supports both full and partial resynchronization modes.

Full resynchronization

Full resynchronization is actually the sameSYNCThe order is similar, can beSYNCTo understand. The slave server sends messages to the master serverPSYNC ? - 1Request full resynchronization.

Partial resynchronization

Partial resynchronization relies on three parts: replication offset, replication backlog buffer, and server run ID

  • Both master and slave servers maintain their own replication offsets. The master propagates its own replication offset +N each time N bytes of data are propagated to the slave. The slave server increases its replication offset +N each time it receives N bytes of data from the master server. The master/slave information interaction sends its own offset information.
  • The replication backlog buffer is a first-in, first-out queue of 1M by default. The master server also writes commands to the replication backlog when propagating to the slave server. Send from the serverPSYNC <runid> <offset>When the command requires partial resynchronization, the master server is given its own replication offset information, known as offset. The master server determines whether it can recover from the replication backlog buffer. If so, return+CONTINUEPartial resynchronization. Otherwise, return+FULLREYNC <runid> <offset>Runid is the running ID of the primary server, offset is the replication offset of the primary server. Returns if the primary server version is lower than 2.8-ERRUnable to processPSYNCCommand.
  • Both the master and slave servers have their own run ids, which are generated automatically when the service is started. The secondary server records the RUNID of the primary server during synchronization. If the secondary server requires partial resynchronization with the primary server, the secondary server carries the RUNID of the primary server. The primary server determines that the RUNID is its own rUNID and can be recovered based on the replication backlog buffer.

Reproduction guarantee

In the command propagation phase, the slave server sends REPLCONF ACK to the master server at a default interval of one second. This command carries the replication offset of the slave server and has three functions:

  • Serves as heartbeat detection to check the network status of the primary and secondary servers.
  • Assist to implement Min-Slaves command.min-salves-to-rightandmin-slaves-max-logConfiguration reduces the ability of the primary server to perform unsafe writes. For example, if the primary server has two secondary servers whose replication delay is greater than 5, the write operation is rejected.
  • The detection command is lost. The master server can use this command to know the current replication status of the slave server and decide to send subsequent commands.