This is the 7th day of my participation in the August More Text Challenge

Although Redis has very high performance, in the actual production environment, the use of single-machine mode is still a lot of problems, such as prone to single machine failures, capacity bottlenecks, and QPS bottlenecks. In general, master-slave replication, sentinel mode and Redis Cluster are three common solutions. This paper will demonstrate how to build a Redis master-slave replication environment through examples and analyze its principles.

1. Build a master/slave replication architecture

1. Create 3 directories redis8000, redis8001, and redis8002. Copy the default configuration file redis.conf to redis8000, specify redis8000 as the host, and modify the following parameters:

Bind 0.0.0.0 port 8000 pidfile /var/run/redis_8000.pid logfile "redis8000.log"#To save performance, turn off RDB persistence and comment the following configuration
#save 900 1
#save 300 10
#save 60 10000
dbfilename dump8000.rdb
dir /home/hydra/files/redis/slave/redis8000/
requirepass 123456
Copy the code

2. Copy the modified redis.conf file to redis8001 and redis8002, and replace port 8000 in the configuration file with your own port.

%s/8000/8001/g
Copy the code

Modifying a configuration file:

Replicaof 127.0.0.1 8000 Masterauth 123456#Enable AOF persistence on the slave machine
appendonly yes 
Copy the code

3. Start three redis instances respectively

. / redis - 5.0.4 / SRC/redis - for server/slave/redis8000 / redis. Conf. / redis - 5.0.4 / SRC/redis - for server/slave/redis8001 / redis. Conf . / redis - 5.0.4 / SRC/redis - for server/slave/redis8002 / redis. ConfCopy the code

The process is started successfully.

4. Connect to host Redis8000 via redis client:

/redis-5.0.4/ SRC /redis-cli -p 8000-a 123456Copy the code

After a successful login, use the command to view the master/slave architecture:

info replication
Copy the code

It can be seen that host 8000 has two slave machines, and slave machines 8001 and 8002 are successfully connected.

5. Connect the slave reDIS80001 through the Redis client, and check the master/slave status through the command:

You can view the role of 8001 as slave and view information about host 8001.

6. In addition, master/slave can be dynamically assigned by mode of instruction. Copy the redis8000 configuration file to redis8003, change the port number to 8003, and leave other configurations unchanged. After logging in to 8003 using redis client, enter the command to specify the host:

Slaveof 127.0.0.1 8000Copy the code

After the host is dynamically specified, if the password is set for the host, you need to run the following command to configure the host password:

config set masterauth 123456
Copy the code

After the configuration is complete, check the slave machine status of 8003:

Check 8000 host status:

The newly added slave 8003 has been added to slave 8000.

Note that the master/slave status dynamically specified by the command will become invalid after the slave machine restarts. Run the kill command to kill process 8003 and check the primary/secondary status.

It can be found that there are only two slave machines left now, 8001 and 8002. Then restart 8003 and check the status again:

There are still two slave machines 8001 and 8002, proving that the instruction specified master and slave will be invalid after restart.

7, read and write test, first test host, read and write can be normal:

The slave machine was tested and found that it could read data normally, but failed to write data:

This is because in master-slave replication, only the host can write data, and the slave is in read-only mode. This is specified in the configuration file. After Redis2.6, the slave machine is read-only by default:

replica-read-only yes
Copy the code

Note that this configuration cannot be changed to no, because the host does not listen to write data events from the slave machine, causing data inconsistency between the master and slave machines.

Second, full copy

For initial replication or other cases where partial replication is not possible, all data from the master node is sent to the slave node. When the amount of data is too large, the network overhead will be high. The process is as follows:

1. Send from machine:

psync ? - 1Copy the code

The “?” The slave machine does not know the runId of the host. -1 indicates full replication

2. The host sends the command to pass its rUNId and offset to the slave:

Fullresync {runid, offset}Copy the code

You can view these two parameters by running the following command:

#You can check out rUNId
info server 
#You can look at offset
info replication
Copy the code

The slave machine reports its offset to the host. If the offset of the host is different from that of the slave machine, data is inconsistent.

3. Save the host data from the slave machine:

save master info
Copy the code

4. When the host executes bgSave, full replication triggers RDB persistence.

bgsave
Copy the code

When the host generates the RDB file, new data may be written. Redis then writes the newly written data to a buffer, repl_back_buffer, of a default size of 1M. The buffer size can be set with repl-backlog-size

5. Host sends RDB to slave:

send rdb
Copy the code

6. Host sends buffer data to slave:

send buffer
Copy the code

7. Clear the data on the slave machine:

flush old data
Copy the code

8. Load RDB and buffer data sent from the host computer:

load rdb&buffer
Copy the code

In full replication, the time consumed includes:

  • Time to persist bgSave
  • RDB file network transfer time
  • Request request data time from the node
  • Time to load the RDB from the machine
  • If AOF persistence is enabled from the node, the possible time for AOF rewriting

Third, partial replication

Partial replication is primarily Redis’s optimization for the high cost of full replication. Redis hopes to minimize the damage in the event of host jitter or disconnection through partial replication.

The specific process is as follows:

  1. Network jitter occurs and the connection is disconnected
  2. The host continues to write the replication buffer repl_back_buffer
  3. The slave machine continues trying to connect to the host
  4. The slave transmits its current RUNID and offset to the master and executes the pysNC command to synchronize the data
  5. If the master finds that the offset is within the range of the buffer, the continue command is returned
  6. Offset is synchronized, so partial replication is based on offset.

So how does Redis decide full or partial copying under normal circumstances? After the slave sends its offset to the host, the host determines whether to perform partial replication based on the offset and buffer size:

  • If all data after offset is still in the replication backlog, a partial copy is performed
  • If the data after offset is no longer in the replication backlog, a full copy is performed

Disadvantages of master-slave replication architecture

1. All write operations are performed on the host first and then synchronized to the slave. Therefore, the synchronization process has a certain delay. This problem is exacerbated when the number of slave machines increases.

2. If the host is down, write operations cannot be performed on the slave host. You need to manually upgrade the slave host to the host, and specify a new host on the slave host.

Manually upgrade a slave machine to a host by executing the following command:

slave of no one
Copy the code

Execute the slave of command on another slave machine to make itself a slave of the new host:

Slave of 192.168.0.1 800 xCopy the code

It can be seen that in this case, when the host goes down, the subsequent repair process is manually operated, which is very troublesome. Therefore, In this case, Redis introduces the Sentinel mode to complete the automatic failover after the host goes down. Later, we will talk about the Sentinel mode in detail.

The last

If you feel helpful, friends can like, forward it, thank you very much for the public number nongshen, to add a friend, do a like friend