This is the 10th day of my participation in the August More text Challenge. For details, see: August More Text Challenge

concept

Master-slave replication refers to the replication of data from one Redis server to another Redis server.

The former is called the master node (master/leader) and the latter is called the slave node (slave/follower). Data replication is one-way and can only be carried out from the master node to the slave node.

Master mainly writes and Slave mainly reads.

By default, each Redis server is a master node, and a master node can have zero or more slave nodes, but a slave node can only have one master node.

Master/slave replication, read/write separation.

role

  1. Data redundancy: Master/slave replication implements hot backup of data, which is a kind of data redundancy besides persistence.
  2. Fault recovery: When a fault occurs on the primary node, the secondary node can provide services for rapid fault recovery. It’s actually redundancy of services;
  3. Load balancing: On the basis of primary and secondary replication, the primary node provides write services and the secondary node provides read services (that is, the application connects to the primary node when Redis data is written, and the application connects to the secondary node when Redis data is read) to share server loads. Especially in the scenario of less write and more read, the concurrency of Redis server can be greatly improved by sharing the read load among multiple nodes.
  4. High Availability (cluster) cornerstone: In addition to the above functions, master-slave replication is also the basis on which sentinel and cluster can be implemented, so master-slave replication is the basis for Redis high availability.

Generally speaking, to use Redis in a project, just one Redis should never be used (down) for the following reasons:

  • From the structure, a single Redis server will have a single point of failure, and a server needs to handle all the request load, the pressure is large;

  • In terms of capacity, the memory capacity of a single Redis server is limited. Even if the memory capacity of a Redis server is 256GB, it cannot be used as Redis storage memory.

    In general, the maximum memory usage of a single Redis should not exceed 20GB.

Environment configuration

Configure only the slave database, and do not configure the master database

127.0.0.1:6379> info replication # replication role:master Master connected_slaves: 0 # not from machine master_replid: 0 d064ffd4a54a85fdd85637328e88d4abd05d8ac master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 Repl_backlog_histlen repl_backlog_size: 1048576 repl_backlog_first_byte_offset: 0:0 127.0.0.1:6379 >Copy the code

Since I’m using only one server here, let’s configure a pseudo cluster environment and modify the corresponding configuration information

  • Port: indicates the number of the port occupied by the process.
  • Pidfile: records the ID of the process. The file has a lock, which can prevent multiple startup of the program;
  • Logfile: specifies the logfile location.
  • Dbfilename: indicates the location of the persistent file

After modifying the configuration information, start three Redis services based on the configuration file

A master from 2

By default (before configuration), each Redis server is the primary node;

Configure only the slave database, and do not configure the master database

Master: one master (6379) and two slaves (6380, 6381)

The host can read and write, but the slave can only read and write

Important!!

If the primary server has a password, the secondary server must be added to the configuration file

Masterauthxxx (Master node connection password)Copy the code

Such as:

Conf file, search for the masterauth parameter, and change it to

 masterauth 6379
Copy the code

Before the configuration:

127.0.0.1:6380> INFO Replication # Replication Role :master connected_slaves:0 master_replid:f89b7907749c026069c80059a9d2cd41fae23e39 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0Copy the code

Configure the slave server by running commands

Host port 127.0.0.1:6380> SLAVEOF 127.0.0.1 6379 OKCopy the code

Information after slave server configuration:

127.0.0.1:6380> INFO Replication # Replication Role :slave master_host:127.0.0.1 master_port:6379 master_link_status:up master_last_io_seconds_ago:10 master_sync_in_progress:0 slave_repl_offset:14 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:2382336ee0bd6abe27c267b35c827bcecbb3fa91 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:14 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:14Copy the code

Host information:

127.0.0.1:6379> INFO Replication # Replication role:master connected_Slaves :1 # Slave0: IP = 127.0.0.1 port = 6380, state = online, offset = 798, lag = 1 master_replid: 2382336 ee0bd6abe27c267b35c827bcecbb3fa91 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:798 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:798Copy the code

The configuration file configures the slave server

The true slave master configuration should be configured in the configuration file to be permanent. We are using a command here to configure, is temporary.

View the configuration file as a secondary node:

  1. After the IP address and port number of the host are configured, the server is a secondary node.
  2. Host password

The host disconnected

The host server is down, but the slave server still exists, but the connection status of master_link_status becomes Down.

When the host server is started again, the primary/secondary relationship still exists.

Disconnect from the machine

After slave machine 1 is disconnected and restarted, slave machine 1 becomes a new host (no host information is configured in the configuration file, and the master/slave relationship is established by using commands).

After slave machine 1 establishes the master/slave relationship with the host again, slave machine 1 can still query the data written to the host during the disconnection period.

The principle

After the Slave is successfully started and connected to the master, it sends a sync command

After receiving the command, the Master starts the background saving process and collects all the received commands for modifying the data set. After the background process finishes executing, the Master transfers the entire data file to the slave and completes a full synchronization.

Full replication: The Slave service saves the database file data and loads it into the memory after receiving it.

Incremental replication: The Master continues to transfer all the new modification commands to the Slave at one time to complete the synchronization.

But as soon as the Master is reconnected, a full synchronization (full replication) will be automatically performed!

Tanghulu pattern

A: Host;

B: Slave machine;

C: slave machine;

A serves as the host of B, and B serves as the host of C. At this point, B is in a sense both a slave and a host, but using the info replication command, you can see that B is still a slave and can only read but not write.

The problem with master-slave replication is that when the host is down, all of its slaves are meaningless. They can only read, not write, and cannot store new data.

So can, when the host down, in its from the inside of the machine to choose one out as a host?

Seeks the usurper

manual

When the host does not exist, we select a slave and manually enter the command:

 SLAVEOF no one
Copy the code

The slave is changed to the host, and then the other slave is manually connected to the latest master node.

 SLAVEOF 127.0.0.1 6388
Copy the code

The guard mode

The method of the master/slave switchover technology is as follows: When the primary server is down, a secondary server needs to be manually switched over to the primary server. This requires manual intervention, time and effort, and causes services to be unavailable for a period of time. This is not a recommended approach, more often than not, we prioritize sentry mode. Redis has officially provided the Sentinel architecture since 2.8 to address this issue.

An automatic version of schemausurp, which can monitor whether the host is faulty in the background and automatically convert the slave to master based on the number of votes.

The principle of

Sentinel mode is a special mode, first Redis provides sentinel command, sentinel is a separate process, as a process, it will run independently.

The idea is that sentinel monitors multiple running Redis instances by sending commands and waiting for the Redis server to respond.

The sentinels here serve two purposes:

  • By sending a command, let the Redis server back to monitor its running status, including the primary server and secondary server;
  • When the sentinel detects that the Master is down, it automatically switches a Slave to Master and passes throughRelease subscriptionNotify the other slave servers and modify their configuration files to switch hosts.

Multiple sentry mode

The Redis server can be monitored by a sentinel process and problems may arise. For this reason, we can use multiple sentinel processes for monitoring. The sentinels also monitor each other, creating a multi-sentry pattern.

  1. Suppose there is a host outage, and Sentinel 1 detects it first, but it doesn’t happen immediatelyfailoverThe (failover) process is simply sentinel 1’s subjective belief that the primary server is not availableSubjective offline;
  2. When the number of sentinels behind the sentinel also detects that the primary server is unavailable and reaches a certain value, a vote will be conducted between the sentinels. The vote is initiated by a sentinelfailover(failover) operations;
  3. When the switch is successful, the sentinels are asked to switch the slave servers they monitor to the host in a publish-and-subscribe mode, a process known as objective logging out.

configuration

  1. Create the sentinel.conf file

  2. Configuration information

    # Sentinel Monitor name monitor host IP address port number fixed value (when how many sentinel monitor host subjective offline, Sentinel monitor myredis 127.0.0.1 6379 1 # If the host has a login password, you need to configure the following information: # Sentinel auth-pass Since the sentry cannot set separate passwords for master and slave, the passwords for master and slave should be the same.) sentinel auth-pass myredis 243600Copy the code

  3. Start the redis – sentinel

     [root@wrkq1njujislh7mx redis]# ./bin/redis-sentinel sentinel.conf
Copy the code

Simulate host downtime

  1. Now we will shut down the mainframe

    127.0.0.1:6379> shutdown not Connected > exit [root@wrkq1njujislh7mx redis]#Copy the code

  2. Look at the status of its two slave machines (try to be quick here, otherwise the sentry will have changed the main engine)

  1. Wait a moment, we can see the sentinel’s log file change:

  2. At this point, the sentinel has selected the new Redis server, port 6380, we check the status of server 6380 and 6381 again:

  3. Now that the new master/slave system has been re-established, what if the previous host 6379 returns?

    [root@wrkq1njujislh7mx redis]# ./bin/redis-server redis79.conf [root@wrkq1njujislh7mx redis]# ./bin/redis-cli -p 6379 -a  243600 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> Ping PONGCopy the code

    Check the server status (I entered the command slowly, faster you can see that the server is still Master, wait until the sentry responds, the server will become slave)

    At this point the server automatically changed to 6380 slave machine, sentinel mode rules.

    127.0.0.1:6379> INFO Replication # Replication Role :slave master_host:127.0.0.1 master_port:6380 master_link_status: Down  master_last_io_seconds_ago:-1 master_sync_in_progress:0 slave_repl_offset:1 master_link_down_since_seconds:1628239308 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:6de2947f7c68080c6bce977a0b4fca378ceb982b master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0  repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0Copy the code

    View the Sentinel log:

advantages

  1. Sentinel cluster, based on master/slave replication, has all the benefits of a master/slave configuration;
  2. The primary and secondary can be switched over, and the failure can be transferred, and the system availability will be better.
  3. Sentinel mode is an upgrade of the master-slave replication mode, manual to automatic, and the overall architecture is more robust.

disadvantages

  1. Redis cannot be expanded online. Once the cluster capacity reaches the upper limit, it is very troublesome to expand the cluster online.
  2. The configuration to implement Sentry mode is complicated (the above configuration information is a simple configuration to make Sentry mode work).