Master-slave architecture can be said to be the necessary architecture of the Internet, the first is to ensure the high availability of services, the second is to achieve read and write separation, you may be familiar with our commonly used MySQL database master-slave architecture, for our Redis is not surprising, redis database also has a variety of master-slave architecture, In master-slave architecture, data synchronization between master nodes and slave nodes is involved. This data synchronization process is called replication in Redis. In this article, we will talk about redis replication technology and master-slave architecture in detail.

  • Setting up master/slave architecture environment
    • How the master-slave architecture is established
    • Disconnection of master and slave architectures
  • The principle of replication technology
    • Data Synchronization Process
    • The heartbeat detection
  • Master/slave topology architecture
    • A master from
    • From more than a master
    • Tree structure

Setting up the master/slave environment

Redis instances are all master nodes by default, so we need to modify some configuration to set up the master/slave architecture. The master/slave architecture of Redis is relatively simple. Redis provides three ways to set up the master/slave architecture, which we will introduce later. In the master-slave architecture, there is one master node and at least one slave node, and data replication is one-way, only from the master node to the slave node, not from the slave node to the master node.

How the master-slave architecture is established

The master-slave architecture can be established in the following three ways:

  • Add the slaveof {masterHost} {masterPort} command to the Redis. Conf configuration file. The command takes effect when the Redis instance is started
  • Add –slaveof {masterHost} {masterPort} after the redis-server startup command
  • Slaveof {masterHost} {masterPort}

The above three methods can be used to set up the master/slave architecture of Redis, we will use the first method to demonstrate, the other two methods to try their own, because it is demo, so we will start two instances of Redis locally, not on multiple machines, we will prepare a master node instance of port 6379, Prepare an instance of a slave node on port 6480. The redis instance configuration file on port 6480 is named 6480.conf and add a slaveof statement to it

slaveof 127.00.1.6379
Copy the code

Start two instances of Redis respectively, and they will automatically establish the master-slave relationship. We will talk about the principle behind this in detail later, first to verify whether our master-slave architecture is successfully set up. We will add a new data on the 6379 master node:

Then fetch the data from 6480 slave:

We can see that we have successfully obtained the new value on the slave node and the new value on the master node, indicating that the master node has been set up successfully. We can use the info replication command to check the information on the two nodes, starting with the master node

Port 6379 instance role is master, there is an instance that is connecting, and other information is running

You can see that the two nodes record object information with each other, which will be used during data replication. It is important to note that by default the slave node is read-only and does not support writing, nor is it recommended to enable writing. We can verify this by writing a piece of data to the 6480 instance

127.0.0.1:6480> set x 3
(error) READONLY You can't write against a read only replica.
127.0.0.1:6480> 
Copy the code

Read -only yes The configuration item controls the read-only of the secondary server. Why is it read-only? As we know, replication is one-way and data can only be transferred from master to slave node. If writing is enabled on the SALve node, the master node will not be aware of the modification of data on the slave node. Data on the slave node cannot be copied to the master node. As a result, data inconsistency occurs. Therefore, it is recommended that the slave node be read-only.

Disconnection of master and slave architectures

To disconnect the master and slave architecture, run the Slaveof no one command on the slave node to disconnect the follower relationship with the master node. We run the Slaveof no one command on 6480 node

127.0.0.1:6480> slaveof no one OK 127.0.0.1:6480> info replication # Replication role:master Connected_Slaves :0 master_replid:a54f3ba841c67762d6c1e33456c97b94c62f6ac0 master_replid2:e5c1ab2a68064690aebef4bd2bd4f3ddfba9cc27 master_repl_offset:4367 second_repl_offset:4368 repl_backlog_active:1 repl_backlog_size:1048576 Repl_backlog_histlen repl_backlog_first_byte_offset: 1:4367, 6480:127.0.0.1 >Copy the code

After executing slaveof no one, the role of the 6480 node is restored to master immediately. If the 6480 node is still connected to 6379, we will add a key-value to the 6379 node

127.00.1:6379> set y 3
OK
Copy the code

Get y on node 6480

127.00.1:6480> get y
(nil)127.0.0.1:6480 >Copy the code

6480 is disconnected from 6379. The slaveof command not only disconnects from 6480, but also switches to the master server. Using slaveof {newMasterIp} {newMasterPort}, we made 6379 the slaveof 6480 and slaveof 127.0.0.1 6480 on 6379. Let’s take a look at Info Replication for 6379

127.0.0.1:6379> info replication # replication role:slave master_host:127.0.0.1 master_port:6480 master_link_status:up master_last_io_seconds_ago:2 master_sync_in_progress:0 slave_repl_offset:4367 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:99624d4b402b5091552b9cb3dd9a793a3005e2ea master_replid2:0000000000000000000000000000000000000000 master_repl_offset:4367 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:4368 repl_backlog_histlen:0 127.0.0.1:6379 >Copy the code

The role of node 6379 is already slave, and the master node is 6480, so we can look at Info Replication on node 6480

127.0.0.1:6480> info replication # Replication role: Master Connected_Slaves :1 Slave0: IP = 127.0.0.1 port = 6379, state = online, offset = 4479, lag = 1 master_replid: 99624 d4b402b5091552b9cb3dd9a793a3005e2ea master_replid2:a54f3ba841c67762d6c1e33456c97b94c62f6ac0 master_repl_offset:4479 second_repl_offset:4368 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:4479 127.0.0.1:6480 >Copy the code

There is 6379 on node 6480. From the information of node 6480, we can see that the Slaveof command has completed the master server switch for us.

The principle of replication technology

Redis master-slave architecture seems to be very simple, we carried out a command is successfully constructed the master-slave architecture, and the data copy is no problem, really easy to use, but behind this redis did help us a lot of things, such as data synchronization between master-slave server, master-slave server status detection, etc., How does Redis achieve this? Let’s take a look

Data Replication Principles

After executing the Slaveof command, our master/slave relationship is established. In this process, the master server and slave server need to go through several steps, as shown in the following figure:

Slaveof command, master and slave servers go through seven steps, including permission verification step is not necessary, in order to better understand these steps, we will use our above redis example to talk about each step in detail.

1. Save the information about the active node

When the 6480 client sends slaveof 127.0.0.1 6379 to the 6480 node server, we immediately get an OK

127.00.1:6480> slaveof 127.00.1.6379
OK
127.00.1:6480> 
Copy the code

The data replication does not start until OK is returned. What the 6480 slave node does is save the given master server IP address 127.0.0.1 and port 6379 to the masterHost and MasterPort properties of the server state

2. Establish a socket connection

After the Slaveof command is executed, the slave server will set up a socket connection with the master server according to the IP address and port specified by the command. If the slave server can successfully set up a socket connection with the master server, The slave server will associate the socket with a file event handler dedicated to handling replication, which will be responsible for subsequent replication, such as receiving fully replicated RDB files and writing commands from the server. Similarly, the master server will create a client state for the socket after receiving the socket connection from the slave server. At this time, the slave server has two identities: server and client. The slave server can send command requests to the master server and the master server will return command replies to the slave server.

3. Run the ping command

After the secondary server is successfully connected to the primary server, the first thing to do is to send a ping command to the primary server for the following purposes:

  • Checks whether the master – slave network socket is available
  • Checks whether the master node currently accepts processing commands

After the ping command is sent, the master server will normally return the Pong command and proceed to the next step after receiving the pong reply from the master server. If there is no pong reply from the master node or timeout, such as network timeout or the master node is blocked and cannot respond to the command, The secondary server disconnects the replication connection and waits for the next scheduled task.

4. Authentication

After the slave server receives the pong reply from the master server, the next thing to do is to determine whether authentication is required based on the configuration information:

  • If the masterauth parameter is set on the slave server, authentication takes place
  • If the masterauth parameter is not set on the slave server, authentication is not performed

In case authentication is required, the slave server will send an auth command to the master server. The command parameter is the value of the masterauth option on the slave server. For example, if the masterauth parameter is set to: 123456, then the slave server will send auth 123456 command to the master server. The authentication process is not smooth, and may encounter the following conditions:

  • If the password sent by the secondary server via the auth command is the same as the requirepass parameter value of the primary server, the subsequent operations will continue. If the passwords are not the same, the primary service will return an Invalid Password error
  • If the primary server does not set the requirepass parameter, the primary server will return a no password is set error

All error conditions cause the slave server to abort the current replication and restart the replication process from socket establishment until authentication is passed or replication is abandoned by the slave server

5. Send port information

After authentication, the slave server executes the REPLCONF listening command to send the master server the slave server’s listening port number, such as 6480 in our example, The slave server will send the REPLCONF listening 6480 command to the master server. After receiving this command, the master server will record the port number in the Slave_listening_port property of the slave server’s corresponding client state. This is the port value we saw in Info Replication on the Master server.

6. Data replication

The slave server sends a psync command to the master server for data synchronization. Prior to redis 2.8, the sync command was used. In addition to the different commands, there are also great differences in the way of replication. Before Redis 2.8, full replication was used, which caused a lot of overhead to the master node and the network. After Redis 2.8, data synchronization will be divided into full synchronization and partial synchronization.

  • Full copy: Generally used for initial replication scenario, both old and new version of redis in from the server connected to the main service for the first time will be a full copy, it will all the data master node one-time grant from node, when the data is bigger, can cause a lot of to the master node and the network overhead, redis replication early version only supports the whole quantity, This is not an efficient way to replicate data

  • Partial replication: Deals with data loss caused by intermittent network disconnection during the primary/secondary replication. When the secondary node is connected to the primary node again, the primary node sends the lost data to the secondary node if conditions permit. Because the reissued data is much smaller than the full data, the high cost of full replication can be effectively avoided. Partial replication is a major optimization of the original replication and effectively avoids unnecessary full replication operations

Redis supports full and partial replication mainly because of the optimization of sync command. After Redis 2.8, a new psync command is used. The command format is psync {runId} {offset}.

  • RunId: indicates the running ID of the primary node
  • Offset: offset of the current data replicated from the node

You may be unfamiliar with the above runid and offset. It doesn’t matter. Let’s take a look at the following three concepts:

1. Replication offset

The master and slave nodes that participate in replication maintain their own replication offsets: the master server adds N to its own offset each time it propagates N bytes of data to the slave server, and the slave server adds N to its own offset each time it receives N bytes of data propagated by the master server. Through comparing the master-slave server replication offsets, the master-slave server is known data are consistent, if the slave servers offset is always the same, then the master-slave data, on the contrary, if the slave servers two offset is not the same, then the master slave server is not in a consistent state data, for example, when have more than one from the server, A server is offline during transmission, as shown in the following figure:

During data transmission, secondary server A was disconnected due to network reasons, and the offset was inconsistent with that of the primary server. When secondary server A reboots and successfully connects to the primary server, it sends the psync command to the primary server again. In this case, should full or partial data replication be performed? If partial replication is performed, how does the master compensate slave A for the lost data during the disconnection? The answers to these questions are in the replication backlog

2. Copy the backlogs

The replication backlog buffer is a fixed length queue stored on the master node. The default size is 1MB. It is created when the master node has connected slave nodes. When the master node responds to a write command, it not only sends the command to the slave node, but also writes the replication backlog buffer, as shown below:

Therefore, the replication backlog buffer on the master server will hold a portion of the recently propagated write commands, and the replication backlog buffer will record the corresponding replication offset for each byte in the queue. Therefore, when the slave server reconnects to the master server, the slave server sends its replication offset to the master server through the psync command. The master server will decide which data synchronization operation to perform on the slave server based on the replication offset:

  • If data after the replication offset of the slave server still exists in the replication backlog buffer, the master server performs partial replication of the slave server
  • If the data after the replication offset of the slave server does not exist in the replication backlog buffer, the master performs a full copy operation on the slave server

3. Server running ID

Each Redis node is dynamically assigned a 40-bit hexadecimal string as a run ID after startup. The run ID is used to uniquely identify the Redis node, which can be viewed using the info server command

127.0.0.1:6379> info server # server redis_version:5.0.5 redis_git_sha1:00000000 redis_git_dirty:0 Redis_build_id :2ef1d58592147923 redis_mode:standalone OS :Linux 3.10.0-957.27.2.el7.x86_64 x86_64 ARCH_bits :64 Multiplexing_api: epoll atomicvar_api: atomic - builtin gcc_version: 4.8.5 process_id: 25214 run_id:7b987673dfb4dfc10dd8d65b9a198e239d20d2b1 tcp_port:6379 uptime_in_seconds:14382 uptime_in_days:0 hz:10 Configured_hz: 10 lru_clock: 14554933 the executable: / usr/local/redis - 5.0.5 / SRC /. / redis server. - Config_file: / usr/local/redis - 5.0.5 / redis. Conf 127.0.0.1:6379 >Copy the code

There is a run_id field which is the ID of the server running

With these concepts in mind, let’s take a look at the flow of the psync command. The following figure shows the flow of the psync command:

The logic of the psync command is relatively simple. The whole process is divided into two steps:

1. The secondary node sends the psync command to the primary node. Parameter runId is the running ID of the primary node saved by the secondary node, and parameter offset is the replication offset saved by the secondary node.

2. After receiving the psync command, the master node will return one of the following three responses to the slave server:

  • FULLRESYNC {runId} {offset} : Indicates that the primary server will perform a full replication operation with the secondary server, where runid is the running ID of the primary server, the secondary server will save this ID to be used when sending the psync command next time, and offset is the current replication offset of the primary server. The slave server will use this value as its own initialization offset
  • Reply +CONTINUE: this means that the master and slave will perform a partial copy, and the slave will simply wait for the master to send the missing part of the data
  • Reply +ERR: The primary server is earlier than Redis 2.8 and cannot recognize the psync command. The secondary server will send the sync command to the primary server and perform full replication with the primary server

7. Run the continuous replication command

When the master node synchronizes the current data to the slave node, the replication process is completed. However, the primary and secondary servers are not disconnected because the primary node continues to send write commands to the secondary node to ensure data consistency between the primary and secondary nodes.

Full replication is the RDB file produced by the master node, sent to the slave server, and then loaded from the local disk. In this case, when the file is too large, the network overhead is very large. Otherwise, the slow data transmission will lead to a long delay between the master and slave. Partial replication means that the master server directly sends the write command to copy the backlog buffer to the slave server.

The heartbeat detection

Heartbeat detection occurs when the primary and secondary nodes maintain a long connection and send heartbeat commands to each other after replication is established, and then continuously send write commands. The heartbeat detection of the primary and secondary nodes is shown as follows:

The primary and secondary nodes have a heartbeat detection mechanism for each other and communicate with each other’s clients. The rules for heartbeat detection are as follows:

  • By default, the primary node sends the ping command to the secondary node every 10 seconds to check whether the secondary node is alive and connected. The sending frequency can be controlled by modifying the parameter rep-ping-replica-period in the redis.conf configuration file
  • The secondary node sends the replconf ack {offset} command every second in the main thread to report its current replication offset to the primary node. This command detects the network between the primary and secondary nodes and sends the replication offset to ensure data consistency between the primary and secondary nodes

The master node determines the lag time of the slave node based on the replconf command, which is reflected in the LAG information in the Info Replication statistics. We run the info replication command on the master server:

127.0.0.1:6379> Info replication # Replication role: Master Connected_Slaves :1 Slave0: IP = 127.0.0.1 port = 6480, state = online, offset = 25774, lag = 0 master_replid: c62b6621e3acac55d122556a94f92d8679d93ea0 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:25774 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:25774 127.0.0.1:6379 >Copy the code

Lag indicates the number of seconds after the last communication with the slave node. The normal delay is between 0 and 1. If the value specified by repl-timeout is exceeded (60 seconds by default), the node is judged offline and the replication client is disconnected. If the node is recovered again, heartbeat detection continues.

Master/slave topology architecture

The master-slave topology of Redis can support single-layer or multi-layer replication. According to the complexity of the topology, Redis can be divided into the following three types: one master-slave, one master-slave, and tree-like master-slave

One master and one slave structure

The one master, one slave structure is the simplest replication topology. The one master, one slave architecture we built above is as shown in the figure below:

The master-slave architecture provides failover support for the slave node when the master node is down. When the concurrency of write commands is high and persistence is required, AOF can only be enabled on the slave node, which not only ensures data security but also avoids the interference of persistence to the performance of the master node. There is a catch, however, that you should be aware of: when the primary node turns off persistence, avoid automatic restart if the primary node goes offline. The data set is empty after the primary node is restarted because the persistence function is not enabled. If the secondary node continues to replicate the primary node, the data on the secondary node will also be cleared, thus losing the significance of persistence. It is safe to avoid this problem by executing Slaveof no one on the slave node to disconnect the replication relationship from the master node and then restarting the master node

One master, many slaves architecture

The one master and many slaves architecture is also known as the star topology. The one master and many slaves architecture is shown in the following figure:

The single-master multi-slave architecture can realize read and write separation to reduce the pressure on the master server. For scenarios with a large read load, the read command can be sent to the slave node to share the pressure on the master node. In daily development, time-consuming read commands, such as keys and sort, can be executed on one of the nodes to prevent slow query from blocking the primary node and affecting the stability of online services. In a scenario with high write concurrency, multiple secondary nodes will cause the primary node to send write commands multiple times, consuming network bandwidth, increasing the load on the primary node and affecting service stability.

Tree-like master-slave architecture

The tree-like master-slave architecture is also called the tree-like topology architecture. The tree-like master-slave architecture is shown as follows:

The tree-like master-slave architecture allows the slave node to not only replicate the master node data, but also continue to replicate downstream as the master node of other slave nodes. By introducing the replication intermediate layer, it can effectively reduce the load of the master node and the amount of data that needs to be transmitted to the slave node. As shown in the architecture diagram, data written into node A will be synchronized to nodes B and C, and then to nodes D and E. Data is replicated layer by layer downwards. When the primary node needs to mount multiple secondary nodes, the tree-like primary and secondary structure can be adopted to reduce the pressure of the primary node to avoid performance interference of the primary node.

The last

At present, many Internet masters have Redis series tutorials, if there is the same, please bear with me. The original is not easy, the code word is not easy, but also hope you support. If there are mistakes in the article, please also put forward, thank you.

Welcome to scan the code to follow the wechat public number: “Technology blog of Brother Flathead”. Brother Flathead can learn together and make progress together.