Redis- master slave replication

Based on analysis of

What is master-slave replication

  • Redis master-slave replication actually copies data from one Redis server to another Redis server.
  • Each server is a master node
  • A master node can have multiple slave nodes, and a slave node has only one parent node
  • The primary node is written, and the secondary node is read

Here’s another picture to give you a better picture of master slave replication

Why implement master-slave replication

  • Master/slave replication can be used to back up data, which is an alternative to persistence
  • To quickly rectify faults, multiple servers are required for primary/secondary replication. Once the primary node fails, the secondary node can replace the primary node to provide read services
  • Primary/secondary replication and read/write separation can share server load. If read operations are much larger than write operations in the production environment, multiple secondary nodes can share load to improve the concurrency of Redis
  • Master-slave replication is also the basis for Redis cluster building and sentinel mechanisms
  • The memory capacity of a single server is limited. Implementing master/slave replication reduces the load and makes the server less prone to downtime.

Redis implements master/slave replication

Redis implements master/slave replication, requiring at least one master and two servants. Therefore, we need to start three Redis-CLI clients and change the configuration files of these three clients

Modify the configuration file

  1. First, we need to copy three configuration files

  1. We select one of them as the master node file and modify the other two configuration files
  • Changing the port Number
  • Example Change the pidfile file name
  • Example Change the logfile file name
  • Change the file name of the persistent RDB file dump. RDB

Start the Redis service corresponding to the three configuration files

We discovered through the info replication command that each server is a master node

In this case, suppose we choose the service of port 6379 as the primary server, and the others as the secondary server. We are going to configure these servant servers using the slaveof host IP address host port number command

  • Note: we set it by command. After the service is shut down, it is no longer slave. To permanently make it slave, you need to configure it in the configuration file.

Now that we have a cluster environment for the master-slave replication, let’s test the master-slave replication

Simple tests for master-slave replication

  1. We put some data into the mainframe

  1. We’re reading data from the machine

The simple primary/secondary replication ~~ is successfully implemented

Then if we go to the host to change the data at this time, can we also read the modified value from the machine?

The answer is yes

So if we shut down the mainframe service at this point, can we still get data from the slave machine?

The answer is yes ~~~ and at this time you go to see whether they are still slave machine, you will find that they are still slave machine, the realization of the host down, users can also read operations, not all services collapse.

So what happens if we try to write again on the slave machine?

The answer is: an error is reported, and the slave machine can only perform read operations

So if the slave machine goes down, if the slave machine goes down, the host does some write operations, and the slave machine that goes down reconnects, what data can it read?

Simulated outage:

After the simulation outage, the host performs write operations

Reconnect the simulated slave machine

We’ll find that we’ve gone from a machine to a mainframe, with the data that was there before the crash

And what happens if we turn it into a slave machine?

We found that it had all the data of the host. Why is that?

This is where the principle of replication comes in

If a Slave specifies a server as a host, the host will send a synchronous called sync command, after using the SLAVEOF commands, the host received, will start the background inventory process, started collecting the command, modify the data set in the inventory after the process has been completed, the data file will be sent to the Slave, Achieve a full synchronization

Notice the two replicates

  • Full replication: After receiving data files from the host, the Slave files are saved to disks and loaded to the memory
  • Incremental replication: If the host and Slave have synchronized, if the host continues to modify the data set, the command is collected again and sent to the Slave

As long as the slave server is reconnected to the host, after the connection will automatically perform full replication!!

conclusion

1, the host accidentally hung up, so the rest of the machine in the host has not arrived before standby, this time the host dropped, it means that lost the write operation. And after the host is online again, it still maintains the original identity, and the information written by the host can still be obtained from the computer

2, if this time is from the machine to hang out, is this from the machine has been disconnected, waiting for online again, don’t connect the original host, it has now become a host, but the original data remains, if its to reconnect again, then the host of the newly added data will also be Shared

At this point, the big guy might say, well, if the host goes down, and the slave machine has all the data on the host, why can’t you turn one of the slave machines into the host for writing? Instead of doing nothing and wasting resources.

Before sentry mode comes out, we can manually convert the slave machine to the host using the Slaveof no one command, and then manually connect the original slave machine to this host.

Note: when the original host is reconnected, the slave machine has become the host or the host, the original host does not have the slave machine that originally belongs to it, it becomes the commander of the light rod

  • You can think of it as plotting to usurp the throne

But manual, I’m sure big guys won’t like it, so sentinel mode came out

The guard mode

Introduction to the

What is sentinel mode?

In plain English, Sentry mode means that Redis creates its own Sentry process to monitor the working status of hosts in the Redis cluster. If a host fails unexpectedly, Sentry mode can switch between the host and the slave machine.

The working principle of

At the beginning of understanding:

Sentry monitors the running and slave machines by sending commands and waiting for the Redis server to respond. Once the host is down, you found by the sentinel surveillance (normal set up multiple sentinel, a sentry found, not immediately to failover, will be, and many other sentinel surveillance to execute), will vote algorithm (for normal is set up multiple sentinel in case only under the condition of the guard, the guard didn’t also the fully completed). Elects a new host from the machine. (Note: at this time of the downtime of the host and then connected back, he can only be the new host after the election of the slave machine, forced to be the younger brother)

To understand:

  1. Each Sentinel process sends a PING command to the Master, Slave, and other Sentinel processes in the cluster at a frequency of once per second.
  2. The Sentinel process marks an instance as SDOWN if it has taken longer than the value specified in the down-after-milliseconds option for the last effective PING response.
  3. If a Master server is marked as a subjective offline (SDOWN), all Sentinel processes that are monitoring the Master server confirm once per second that the Master server is indeed in a subjective offline state.
  4. Master servers are marked as objective offline (ODOWN) when a sufficient number of Sentinel processes (greater than or equal to the value specified in the configuration file) confirm that the Master server has gone subjective offline (SDOWN) within the specified time range.
  5. In general, each Sentinel process sends INFO commands to all Master and Slave servers in the cluster every 10 seconds.
  6. When the Master server is marked as ODOWN by the Sentinel process, the Sentinel process sends INFO commands to all slaves of the offline Master server once every 10 seconds instead of once every second.
  7. The objective offline status of the Master Master server is removed if a sufficient number of Sentinel processes do not agree to take the Master offline. If the Master server reping the Sentinel process and returns a valid response, the subjective offline status of the Master server will be removed.

Here’s another diagram. (Master, slave)

The role of sentinels

  • Sentry sends commands to Redis servers to monitor their operating status, both master and slave
  • When the sentry detects that the host is down, it automatically switches the elected slave to the host, and then notifies the other slave servers in a public-subscribe mode to modify the configuration file and let them switch to the host

Redis implements the Sentinel mode

Redis implements the sentinel mode configuration

  • Sentinel mode of service –redis-sentinel
  • Sentinel mode core filesentinel.conf
  • Sentinel mode core file core configuration –Monitor IP address monitor port 2 (when two sentinels think the host is down, they objectively think the host is down)

Implementation steps

  1. Build test environment: one master, two slave, one sentry (minimum configuration)
  2. One master and two subordinate environment inA master-slave replicationSet the
  3. Build a sentinel
    • In their own writingredis.confWrite in the same directorysentinel.conffile

Core base configuration (I put the other configuration last)

  1. throughsudo redis-sentinel myconfig/sentinel.confStart sentinel mode for observation

  1. Shutdown the host to simulate the host downtime, and then observe that 6381 has become a host

  1. Imagine what would happen if the mainframe reconnected now?

The results showed that the 6379 became a slave machine

Additional configuration of the Sentinel mode profile

# Example sentinel.conf

# *** IMPORTANT ***
# Bind IP address
# bind 127.0.0.1 192.168.1.1
Protected mode (whether external links are prohibited, except for bound IP addresses)
# protected-mode no

# port <sentinel-port>
# Port on which this Sentinel instance is running
port 26379

Redis Sentinel does not run as a daemon by default. Set to yes if required.
daemonize no

Redis will write a pid file to /var/run/redis-sentinel.pid
pidfile /var/run/redis-sentinel.pid

# Specify the log file name. If the value is null, Sentinel log standard output is forced. Under the daemon, if logging is done using standard output, the log is sent to /dev/null
logfile ""

# sentinel announce-ip <ip>
# sentinel announce-port <port>
#
The above two configuration instructions are very useful in the environment because NAT can access Sentinel externally from a non-local address.
#
# When providing announce-IP, Sentinel will declare the specified IP address in the communication instead of automatically detecting the local address as usual.
#
# Similarly, Sentinel will announce the specified TCP port when providing nose-port is valid and non-zero.
#
# These two options do not need to be used together. If only announce-IP is provided, Sentinel will announce the specified IP and the server port specified by the "port" option.
# If only the announce-port is provided, Sentinel will notify automatically detected local IP and specified port.
#
# Example:
#
IP # sentinel announce - 2

# dir <working-directory>
Every long-running process should have a clearly defined working directory. For Redis Sentinel, / TMP is its working directory.
dir /tmp

# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#
# tell Sentinel to specify the primary node and determine its O_DOWN status only if at least the 
      
        sentinels agree.
      
#
#
# Replicas are automatically discovered, so you do not need to specify replicas.
# Sentinel itself will override this configuration file to add copies using other configuration options. Also note that when a copy is upgraded to a master copy, the configuration file is overwritten.
#
The master name cannot contain special characters or Spaces.
The # valid characters can be a-z 0-9 and the three characters ".-_".Sentinel Monitor MyMaster 127.0.0.1 6379 2If redis is configured with a password, you must configure authentication, otherwise it cannot be automatically switched
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd

# sentinel down-after-milliseconds <master-name> <milliseconds>
#
If the primary node or replica does not reply to PING within a specified period of time, it is considered to be in S_DOWN state.
#
The default is 30 seconds
sentinel down-after-milliseconds mymaster 30000

# sentinel parallel-syncs <master-name> <numreplicas>
#
# How many replica nodes are synchronized during failover
sentinel parallel-syncs mymaster 1

# sentinel failover-timeout <master-name> <milliseconds>
#
# specify failover timeout in milliseconds. It is used in a number of ways:
#
# - The time required to restart the failover after the previous failover has been attempted by the given Sentinel against the same primary server, which is twice the failover timeout.
#
# - Start time counting when a slave synchronizes data from an incorrect master. Until the slave is corrected to synchronize data to the correct master.
#
# - Time required to cancel a failover that is already in progress but has not generated any configuration changes
#
# - Maximum time required to configure all Slaves to point to the new master when performing failover.
# Slaves will still be configured correctly to point to master even after this timeout.
#
Default 3 minutes
sentinel failover-timeout mymaster 180000

# script execution
#
# sentinel notification-script and sentinel reconfig-script are used to configure scripts called to notify system administrators or to reconfigure clients after a failover.
# The script executes using the following rules for error handling:
#
# If the script exits with "1", the execution will be retried later (the maximum number of retries is the currently set 10).
#
If the script exits with a value of "2" (or higher), execution will not be retried.
#
If the script terminates because of a signal, the behavior is the same as exit code 1.
#
The maximum running time of the script is 60 seconds. When this limit is reached, the script terminates with SIGKILL and retry execution.

Notification script
#
# sentinel notification-script <master-name> <script-path>
#
Call the specified notification scripts (e.g. -sdown, -odown, etc.) for any Sentinel events generated for the warning level.
# This script should notify the system administrator of a problem with the Redis system monitored by email, SMS or any other messaging system.
#
Call the script with two parameters: the first is the event type and the second is the event description.
#
The script must exist and be executable in order to start sentinel when this option is provided.
#
#, for instance:
#
# sentinel notification-script mymaster /var/redis/notify.sh

Customer reconfigure the script
#
# sentinel client-reconfig-script <master-name> <script-path>
#
When the primary server changes due to failover, scripts can be called to perform application-specific tasks to notify the client that the configuration has changed and the primary server address has changed.
#
The following arguments will be passed to the script:
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# 
      
        Currently always fails over" failover"
      
# 
      
        is a leader or observer
      
#
The from-ip, from-port, to-ip, to-port parameters are used to pass the old address of the primary server and the new address of the selected copy.
#
#, for instance:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh

# safe
# Avoid script resets, default yes
By default, SENTINEL SET will not be able to change notification-script and client-reconfig-script at run time.
# This avoids a simple security issue where the client can set the script to anything and trigger a failover to execute the program.
sentinel deny-scripts-reconfig yes

The # REDIS command is renamed
#
#
# In this case, Sentinel can be told to use a different command name than the normal command name.
# For example, if the master "mymaster" and the associated copy "CONFIG" were all renamed to "GUESSME", I could use:
#
# SENTINEL rename-command mymaster CONFIG GUESSME
#
Every time Sentinel uses CONFIG, it will use GUESSME. Note that there is no actual need to respect the command case, so it is the same to write "config Guessme" in the example above.
#
# SENTINEL SET can also be used to perform this configuration at run time.
#
To set the command back to its original name (undo renaming), rename the command to itself:
#
# SENTINEL rename-command mymaster CONFIG CONFIG

Copy the code