copy

Users can set the SLAVEOF option to make one server replicate another. The replicated servers are called master servers, and the replicated servers are called slave servers.
The primary and secondary servers must keep the same data in their databases, which is called consistency.

For example, if the primary server runs the following command: SET MSG “VMS”, the secondary server must run the same command to ensure data consistency.

If a server becomes a slave of another server, then

The host is responsible for writing, the slave machine can only be responsible for reading;
Host all data from the opportunity to automatically save

Run slaveof on server 6380 to make it slaveof server 6379:127.0.0.1:6380> slaveof 127.0.0.1 6379 # make yourself slaveof server 6379

At this point, if host 6579 performs a write operation

Data can be read from the slave machine:

The slave machine cannot write data. If the slave machine writes data, an error is reported:

Implementation of replication function

The Redis replication feature is divided into two steps:

Synchronization: Updates the state of the server to that of the master server
Master and slave server role is spread: sync status is consistent, if this time the main server to perform write operations lead to a change in its own state, to the state of the master-slave inconsistencies, will spread the main server just to perform write operations command it spread from the server, the master-slave server database back to a consistent state.

synchronous

The time of synchronization

When the server executes the SLAVEOF command, requiring the slave server to replicate the master server, the slave server performs synchronization first. The purpose is to update the state of the slave server to the current state of the master server.

Specific steps

The slave server sends messages to the master serverPSYNCThe command
The primary server that receives the command executesBGSAVEOperation (if you do not understand this command, please refer to itJuejin. Cn/post / 692684…), an RDB file is generated in the background and a buffer is used to record the start of executionBGSAVESo write the command after.
When the master server completesBGSAVEAfter command, the master server will generate the RDB file sent to the slave server, the slave server received after loading the RDB file, set its database state to the master server to executeBGSAVEStatus of the command.
The master server sends all write commands of the buffer set in step 2 to the slave server, and the slave server executes these commands (the slave server is blocked at this time, and cannot execute other commands while loading the RDB file). After the slave server executes these commands, the state of the master and slave servers is consistent.

spread

The time of transmission

After the synchronization is complete, the status of the primary server is the same as that of the secondary server. If the write command is executed on the primary server, the status of the primary server changes and the status of the primary server is inconsistent with that of the secondary server.

In order for the master and slave servers to reach the state always situation again, the master server needs to perform command propagation on the slave server:

The master server sends the write command it just executed to the slave server
Execute this command when received from the server
After the command is executed, the status of the primary and secondary servers is the same again

Full and partial resynchronization

Full resynchronization is used at the time of the initial replication, the synchronization phase above, by having the primary server send RDB files and write commands to the buffers in the primary server.
Partial resynchronization is used when reconnecting to the primary server after the secondary server has gone offline. When dropped from the server, the server may make some write operation, after the launch from the server, if the condition allows, the primary server will disconnects from the server after the write command sent from from the server, the server as long as the write command, carried out by the state of the slave servers are consistent.

Implementation of partial resynchronization

Partial resynchronization consists of the following three parts:

The replication offsets of the master and slave servers
Replication backlog buffer for the primary server
Running ID of the server

Copy offset

Both master and slave servers maintain a replication offset

Each time the master propagates N bytes of data to the slave, it adds N to its own replication offset
Each N bytes of data received from the server to the master server is added to its own replication offset.

Unknown:

If the primary and secondary servers are in the same state, the replication offsets are always the same
If the two replication offsets are different, the two states must be inconsistent

If copy of the current main server A offset is 1000, from the server copy offset is 950 B, B dropped at this time, when B on reconnecting A send A, B PSYNC command to report on his own copy offset is 950, if at this time in A still holds 950 after execution of command, you can send B from the 950 position later Write operation,B can perform synchronization. What if the command data of A after 950 is no longer available?

Replication backlogs

The replication backlog buffer is a fixed-length fifO queue maintained by the primary server, with a default size of 1MB.

When the master propagates commands, it not only sends its own write commands to all slave servers, but also queues the write commands into the replication backlog buffer. As shown in figure:

At this point,The replication backlog buffer on the primary server holds a portion of the recently propagated write commandsThe copy backlog buffer records the corresponding byte for each byte in the queueCopy offset.

When reconnecting from the slave server to the master server, the PSYNC command sends its replication offset to the master server. The master server uses this offset to decide what to do:

If the data after offset (starting with offset+1) is still in the replication backlog, only a partial copy is required.
If the data is no longer available, you need to perform full resynchronization.

Server Running ID

Each server, master or slave, has its own running ID.
The run ID is automatically generated and is a set of 40 random hexadecimal numbers.

On the first replication from the slave server, the master server sends its run ID to the slave server, which saves the ID. When the slave server reconnects to the primary server, the slave server sends the previously saved runtime ID to the currently connected primary server:

If the current ID of the primary server is the same as the ID sent by the secondary server, the primary server is connected to the same primary server, and the primary server tries to perform partial resynchronization.
If no, the current primary server is not the same as the original primary server. In this case, complete resynchronization is performed.

Implementation of the PSYNC command

Command call timing

If the slave server has not replicated any master servers, or SLAVEOF NO ONE has been executed before, when a new replication starts from the slave server, PSYNC will be sent to the master server? -1: initiates a full resynchronization request.
If the slave has replicated the master before, the slave sends PSYNC

when it starts a new replication operation.

:runid represents the id of the primary server when the last replication was performed. Offset represents the current replication offset of the secondary server. The primary server determines the operation based on these two parameters:

If the primary server returns+FULLRESYNC <runid> <offset>, it indicates that the primary and secondary servers will perform full resynchronization.runidIs the running ID of the primary server, which is saved by the secondary server.offsetIs the current replication offset of the master server, which the slave server uses as its own initialization offset.
If the return+CONTINUE, indicating that partial resynchronization will be performed. The slave server only needs to execute the missing part sent by the master server.
If the return-ERR“, indicating a version problem.

Complete steps for replication

Set the address and port of the primary server

SLAVEOF IP Port: SLAVEOF 127.0.0.1 6379: Save the IP address to masterHost and the port to masterPort.
SLAVEOF is an asynchronous command. After the masterhost and masterport properties are set, the slave server returns OK to the command line indicating that the replication command has been received, but the actual replication will not take place until OK is returned.

Establish a socket connection

After executing the SLAVEOF directive, the slave server creates a socket connection to the master server based on the masterHost and masterPort properties.

If the socket created by the slave server successfully connects to the master server, the slave server creates a file event handler for replication, which takes care of the subsequent replication, such as accepting RDB files and propagating commands from the master server.
Once the master server accepts a socket connection from the slave server, it creates the corresponding client state for that socket, treating the slave server as a client, where the slave server is both a client (of the master) and a server (connected to its own Redis-CLI). The slave server can send commands to the master server, and the master server replies with commands to the slave server.

Send the PING

After the secondary server becomes the client of the primary server, it sends a PING message to the primary server.

PING is used to check whether the read and write status of the socket is normal
Check that the primary server can handle requests properly

The authentication

If masterAuth is set on the slave server, authentication takes place. If masterAuth is not set on the slave server, authentication is performed.

Primary server Settingsrequirepass, set from the servermasterauth: If the password is correct, proceed to the next step. Otherwise return an error
The primary server is not setrequirepass, set from the servermasterauth: Return error
Primary server SettingsrequirepassThe slave server is not setmasterauth: Return error
The primary server is not setrequirepassThe slave server is not setmasterauth: Proceed to the next step

Sending port Information

After authentication, the slave server sends the slave server’s port number to the master server. After receiving the information, the primary server records the port number so that it can display the port number of the secondary server that it owns during Info Replication.