Redis uses PSYNC for primary/secondary replication. The PSYNC command has two modes: full and partial resynchronization

  • Full resynchronization is used to handle the initial replication: synchronization is done by having the master create and send the RDB file and sending commands to the slave stored in the cache
  • Some heavy synchronization To deal with repeat system after disconnection: from the server after the break line to connect the main server, if the condition allows, the primary server can connect the slave servers disconnect during the execution of write command is sent to and from the server from the server just receive and execute these commands written, the database can be updated to the current state of the main server.

The implementation principle of partial resynchronization

The replication offsets of the master and slave servers

  • Each time the master propagates N bytes of data to the slave, it adds N to the value of its own replication offset
  • The slave server adds N to its own replication offset each time it receives N bytes of data from the master server

Replication backlog buffer for the primary server

  • The premise

    • A fixed-length FIFO queue maintained by the primary server, with a default size of 1 MB. The difference from a normal queue: when the number of queued elements is greater than the queue length, the first queued element is ejected and new elements are put into the queue
    • When the master server propagates data, it writes it to the replication backlog buffer.

  • operation

    • If the data after offset is still in the replication backlog, the master server performs partial resynchronization on the slave server
    • If the data after offset does not exist in the replication backlog, the master server fully resynchronizes the slave server

  • Adjusting and optimizing

    The formula for the minimum size of the copy backlog buffer is second * write_size_per_second

    • Second: average time required to reconnect to the primary server after the secondary server is disconnected
    • Write_size_per_second: amount of write command data generated by the primary server per second

    For security purposes, this is generally set to 2 * second * write_size_per_second

Running ID of the server

  • Each Redis server, both primary and secondary, will have its own run ID. The run ID is automatically generated when the server starts and consists of 40 random hexadecimal characters.
  • When the slave makes the first copy to the master, the master passes its run ID to the slave, and the slave saves the ID. When the slave server disconnects and connects to a master server, the slave server sends the previously saved run ID to the currently connected master server.

  • Impact on replication mode

    • If the secondary server has the same running ID as the currently connected primary server, the secondary server replicates the currently connected primary server before disconnecting, and the primary server can continue to attempt partial resynchronization
    • If the running ID stored on the secondary server is different from that of the currently connected primary server, it indicates that the secondary server was not copied to the currently connected primary server before disconnecting, and the primary server can continue to try to perform full resynchronization

PSYNC command

Calling from the server

  • If the slave server has not replicated any master servers before, or SLAVEOF no one has been executed before, the slave server will send PSYNC to the master server when starting a new replication? -1 command, actively request the master server for complete replication.

  • If a master server has been replicated from the slave server, the slave server sends the PSYNC


    command to the master server when starting a new replicate.

    • Runid: indicates the running ID of the primary server that was last replicated
    • Offset: indicates the current offset from the server

Master server reply

  • The primary server replies with + FULLRESYNC


    , indicating that the primary server will perform a full resynchronization with the secondary server

    • Runid: indicates the running ID of the primary server, which is saved by the secondary server
    • Offset: the primary server’s current offset. The secondary server will use this value as its initial offset

  • The primary server returns a + CONTINUE reply, which indicates that the primary server will partially resynchronize with the secondary server. The slave server simply waits for the master to send its missing parts

  • The primary server returns a -err response, indicating that the primary server version is earlier than Redis2.8 and cannot recognize the PSYNC command. The secondary server sends the SYNC command and fully synchronizes with the primary server

Implementation of replication

By sending the SLAVEOF


command to the slave server.

  • Set the address and port of the primary server

  • Establish a socket connection

  • Sending the ping command

    role

    • Check whether the read and write status of the socket connection is normal
    • Check whether the primary server can process command requests properly

    The results of

    • If the master server returns a command reply to the slave server, but the slave server cannot read the reply within the specified time, it indicates that the network connection between the master and slave servers is poor. When this happens, the secondary server is disconnected and the socket connected to the primary server is recreated.
    • If the master server returns an error to the slave server, it indicates that the master server is temporarily unable to process the command request from the slave server. When this happens, the secondary server is disconnected and the socket connected to the primary server is recreated.
    • If you read from the serverPONG“, the network connection between the primary server and secondary server is normal, and the primary server can process command requests from the secondary server.

  • The authentication

  • Sending port Information

  • synchronous

    The slave server is always the client of the master server until the synchronization operation is performed, but the master server also becomes the client of the slave server after the synchronization operation is performed.

  • Command transmission

The heartbeat check

During command propagation, the secondary server sends the REPLCONF ACK

command to the primary server once every second. Replication_offset Specifies the current replication offset of the secondary server

  • Check the network connection status of the primary and secondary servers

    If the master server does not send a command from the slave server for more than a second, the master server knows that there is a problem with the connection between the master and slave servers

    By sending the INFO replication command to the primary server, you can view the lag that determines the interval between the last command sent from the secondary server to the primary server

  • Assist to implement the Min-Slaves option

    Redis’ min-rabes-to-write and Min-Rabes-max-lag options prevent the master server from executing write commands in unsafe situations

  • Detection command loss