This is the 25th day of my participation in the August Genwen Challenge.More challenges in August

Redis learning 10, Redis cluster building and master/slave replication

Master slave replication theory

What is master-slave replication

Master-slave replication refers to the replication of data from one Redis server to other Redis servers. The former is the master node and the latter is the slave node

Data replication is one-way and can only be performed from the host to the slave. Data is mainly written to the master and read to the slave

By default, every Redis server is a master node, and a master node can have multiple slave nodes or no slave nodes, but conversely, a slave node can only have one master node

The role of master-slave replication:

  • Data redundancy

Master-slave replication implements hot backup of data and is a data redundancy method other than persistence

  • Fault recovery

When the primary node is abnormal, the secondary node can replace the primary node to provide services to achieve rapid fault recovery, which is actually a kind of service redundancy

  • Load balancing

On the basis of master-slave replication and read/write separation, the master node can provide the write service, and the slave node can provide the read service (that is, the master server is connected to write, and the slave server is connected to read), which can share the pressure of the server

Especially in the case of more reads and less writes, the concurrency of redis server can be greatly provided by sharing the load with multiple slave nodes

  • Is the foundation of high availability

Master-slave replication is the basis on which the Sentinel pattern and clustering can be implemented, and so master-slave replication is the cornerstone of Redis high availability

In our actual work projects, it is absolutely impossible to have only one Redis server in the production environment. It must be clustered mode, and at least one master server and two slave servers

  • Structurally, it is too difficult for a single REIDS server to have a single point of failure and for a single Redis server to do all the reading and writing
  • From the perspective of redis server capacity, the capacity of a single Redis server is limited. Generally speaking, the maximum memory used by a single Redis server should not exceed 20 GB

For example, e-commerce platforms, which are based on more read and less write, will use the architecture shown above. Master/slave replication and read/write separation are carried out in most cases, while write operations are carried out in a few cases. In this way, the pressure on the server can be reduced

Environment configuration

Let’s start 3 Redis-servers on one server to simulate the cluster of REids, which is a pseudo cluster

View redis master node information,

127.0.0.1:6379 > info replication# Replicationrole:master connected_slaves:0 master_failover_state:no-failover master_replid:87d84e1268d878bd3b1e7d5ea7f5060d4f71471e master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0  repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0Copy the code

Conf file 6379.conf, 6380.conf, 6381.conf

Root @ iZuf66y3tuzn4wp3h02t7pZ: / usr/local/redis/redis - 6.2.5 # cp redis. Conf 6379. Conf Root @ iZuf66y3tuzn4wp3h02t7pZ: / usr/local/redis/redis - 6.2.5 # cp redis. Conf 6380. Conf Root @ iZuf66y3tuzn4wp3h02t7pZ: / usr/local/redis/redis - 6.2.5 # cp redis. Conf 6381. ConfCopy the code

Modify the following four locations of the above three files:

  • Port number of redis Serverport 6379
  • The name of the REDis server PID running in the background,pidfile /var/run/redis_6379.pid
  • Log files,logfile "6379.log"
  • RDB persistence file,dbfilename dump6379.rdb

After the three files are modified, start the three redis-servers

Root @ iZuf66y3tuzn4wp3h02t7pZ: ~ # redis server/usr/local/redis/redis - 6.2.5/6379. The conf root @ iZuf66y3tuzn4wp3h02t7pZ: ~ # Redis server/usr/local/redis/redis - 6.2.5/6380. The conf root @ iZuf66y3tuzn4wp3h02t7pZ: ~ # redis - server / usr/local/redis/redis - 6.2.5/6381. ConfCopy the code

After the redis-server background is started, let’s check whether it is started successfully

ps aux | grep reids

Root @ iZuf66y3tuzn4wp3h02t7pZ: ~ # ps aux | grep redis root 4345 62776 3864 0.0 0.1? Ssl 08:50 0:00 Redis-server 0.0.0.0:6379 root 4351 0.0 0.1 62776 3860? Ssl 08:50 0:00 Redis-server 0.0.0.0:6380 root 4359 0.1 0.1 62776 3856? Ssl 08:50 0:00 redis-server 0.0.0.0:6381 root 4369 0.0 0.0 14436 1016 PTS /0 S+ 08:50 0:00 grep --color=auto redisCopy the code

Use the redis-CLI to connect to the redis-server of each port and check the master/slave information. It is found that each Redis-server is a host, as follows:

127.0.0.1:6379 > info replication# ReplicationRole :master # Host Connected_SLAVES :0 # Slave master_failover_state:no-failover master_replid:38cbcdac7d7f6b1c154643f94d6742111826cccb master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0Copy the code

Start setting primary and secondary (one primary and two secondary)

By default, the Redis server is the master node by default, we only need to configure the slave node

We set 6379 as master, 6380 as slave, and 6381 as slave

6380 and 6381 are both set as the slave nodes of 6379. Now you can view the configuration information of 6379 as follows: it is the master node with two slaves

If one of the Redis servers is restarted, it will also become master. Therefore, we need to set the persistent Settings in the configuration file

Test reading and writing keys on the host and on the slave

root@iZuf66y3tuzn4wp3h02t7pZ:~# redis-cli -p 6379
127.0.0.1:6379>
127.0.0.1:6379> set k1 v1
OK
127.0.0.1:6379> get k1
"v1"
127.0.0.1:6379> set k2 v2
OK
127.0.0.1:6379> set k3 v3
OK
127.0.0.1:6379>
root@iZuf66y3tuzn4wp3h02t7pZ:~# redis-cli -p 6380
127.0.0.1:6380> get k2
"v2"
127.0.0.1:6380> set k4 v4
(error) READONLY You can't write against a read only replica.
127.0.0.1:6380>
root@iZuf66y3tuzn4wp3h02t7pZ:~# redis-cli -p 6381
127.0.0.1:6381> get k3
"v3"
127.0.0.1:6381> set k5 v5
(error) READONLY You can't write against a read only replica.
Copy the code

On the mainframe, you can read, you can write

From the machine, you can only read, not write

Test disconnects the host

root@iZuf66y3tuzn4wp3h02t7pZ:~# redis-cli -p 6379
127.0.0.1:6379> shutdown
not connected>
Copy the code

root@iZuf66y3tuzn4wp3h02t7pZ:~# redis-cli -p 6381
127.0.0.1:6381> keys *
1) "k3"
2) "k2"
3) "k1"
Copy the code

Host down, from the machine can still read data, but still can not write, the host up, the host can still read and write, write operation is back, our happiness is back again

Principle of replication

After connecting to the master successfully, the Slave sends the sync command

After receiving the command, the master starts the background saving process and collects all received commands for modifying the database set. After the background process finishes executing the commands, the master sends the entire data file to the slave and completes a synchronization

Full amount of copy

After receiving the data from the master, the slave saves the data to a disk and loads it to the memory

Incremental replication

The master passes all the newly collected commands to the slave to complete the synchronization

Once the master node is reconnected, a full full synchronization is performed

References:

redis_doc

Welcome to like, follow and favorites

Friends, your support and encouragement, I insist on sharing, improve the quality of the power

All right, that’s it for this time

Technology is open, our mentality, should be more open. Embrace change, live in the sun, and strive to move forward.

I am Nezha, welcome to like, see you next time ~