Redis Cluster Cluster

Redis – cluster architecture diagram

Architectural details

  • All The Redis nodes are connected to each other (the PING PONG mechanism), internally using binary protocols to optimize transmission speed and bandwidth

  • Node fail takes effect only after more than half of the nodes in the cluster fail

  • The client is directly connected to the Redis node without the need for an intermediate proxy layer. The client does not need to connect to all nodes in the cluster. The client can connect to any available node in the cluster

  • Redis-cluster maps all physical nodes to slots [0-16383]. Cluster is responsible for maintenance

There are 16,384 hash slots in the Redis cluster. When a key-value needs to be placed in the Redis cluster, Redis first uses the CRC16 algorithm to calculate a result for the key, and then calculates the remainder of the result for 16384. In this way, each key will correspond to a hash slot numbered 0 to 16383, and Redis will map the hash slots to different nodes roughly equally based on the number of nodes

Redis-cluster voting: fault tolerant

  • Node failure judgment: All the master nodes in the cluster vote. If more than half of the master nodes communicate with one master node more than once (cluster-nod-timeout), the master node is considered to be down

  • Cluster failure judgment: When the entire cluster is unavailable (cluster_state:fail)

    • If any master of the cluster fails and the current master has no slave, the cluster enters the Fail state. It can also be interpreted as that the cluster enters the Fail state when the mapping of slot [0-16383] is incomplete

    • If more than half of the masters of the cluster fail, the cluster enters the Fail state regardless of whether there are slaves in the cluster

RedisCluster Creates a cluster

You need to enable or disable the firewall

service iptables stop

  • Create a node

    A Redis cluster requires a minimum of three primary servers and three secondary servers

    The port number ranges from 8001 to 8006

    • Step 1: Create an instance of 8001 and edit the redis.conf file, changing port to 8001

      Note: When creating an instance, that is, copying the single-server installation, the bin directory generated is directory 8001

    • Step 2: Modify the redis.conf configuration file to enable cluster-enable yes

    • Step 3: Copy 8001, create instances 8002 to 8006, and pay attention to port modification

      cp 8001/ 8002 -r

    • Step 4: Start all instances

    • Step 5: Create a Redis cluster

      ./redis-cli --cluster create 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006 ƕ cluster replicas - 1Copy the code

  • Create the cluster

    /redis-cli -h 127.0.0.1 -p 8001 -c

  • Viewing Cluster Status

    • Viewing Cluster Status

      cluster info

    • View the nodes in the cluster

      cluster nodes

Cluster By default, the secondary node is unwritable, unreadable and only used for backup

The working principle of

  • The client is directly connected to the Redis node without an intermediate Proxy layer and directly connects to any Master node

  • According to the formula HASH_SLOT=CRC16(key) mod 16384, which fragment is mapped to is calculated, and then Redis will go to the corresponding node for operation

    advantages

    • If the Master fails, the Redis Cluster automatically switches from Slave to Master

    • You can expand the capacity horizontally

    • Support automatic migration, when there is a Slave down, then only the Master, this time the high availability is not a good guarantee, in case the Master also down, what to do? In this case, if other masters have redundant slaves, the cluster automatically migrates the redundant slaves to the Master without a Slave

    disadvantages

    • Batch operations are a pit

    • Resources are poorly isolated and easily affect each other

Master-slave switch

If a node in a cluster is found to be in the Fail state through the error detection mechanism, a primary/secondary switchover is performed. Redis also provides a manual switchover by running the cluster failover command

  • Automatic switch

    The switchover process is as follows (assuming that the primary node to be switched is M and the secondary node to perform the switchover is S)

    • S first updates its state and will declare itself as the master node. And remove S from M

    • Since S needs to be switched to the primary node, the information related to synchronization data of S is cleared (that is, data is no longer synchronized from M).

    • Declare all the slots that M provides services in S

    • Send a PONG packet to notify other nodes in the cluster of status updates

  • The manual switch

    When a node receives the cluster failover command, the manual switchover is performed. The process is as follows

    • The slave node first sends a mfStart package to the master node. Notify the primary node to start the manual switchover from the secondary node

    • The primary node blocks the execution of all client instructions. After that, the primary node sends ping packets in the periodic function clusterCron with a special mark in the header

    • When the secondary node receives a ping packet from the primary node and detects a special mark, the replication offset of the primary node is obtained from the packet header

    • The secondary node uses the periodic function clusterCron to check whether the replication offset currently processed is equal to that of the primary node. If the offset is equal to that of the primary node, the switchover process starts

    • After the switchover is complete, the primary node will redirect all blocked client commands to the new primary node by sending the +MOVED command

    As you can see from the process, no data is lost when the master/slave switchover process is executed manually, and no execution commands are lost, except for a temporary pause during the switchover

A copy of the drift

Assuming that A is faulty, A1 of primary A will perform A switchover. After the switchover, A1 becomes primary A1. In this case, primary A1 will have A single point of failure

The periodic scheduling function clusterCron periodically checks the following conditions

- Whether a single primary node exists, that is, the primary node does not have any available secondary nodes. - Whether a primary node has two or more available secondary nodesCopy the code

If both of the above conditions are met, copy drift is performed on a slave node from among the nodes with the most available. The selection criteria is to select the first node from the smallest node name to the largest node name

Specific drift process

- Remove C1 from C's records - Change the primary node recorded by C1 to A1 - Add a slave node C1 to A1 - Set the data synchronization source for C1 to A1Copy the code

The drift process is to change the information recorded by some nodes and then synchronize the information to all cluster nodes through heartbeat packets

Add a node

If a cluster needs to be expanded, add new nodes to the cluster

Let’s start by creating a new Instance of Redis at port 8007. The other configurations are the same as above

Start node 7007 and add the new node to the cluster using redis -CLI

/redis-cli --cluster add-node IP address of a new node: port IP address of an existing cluster: port

/redis-cli --cluster add-node 192.168.133.22:8007 192.168.133.22:8001

Distribution of slot.

We can see that the new node has been added to the cluster, but the new node does not have any slots on it, which means that no data can be placed on this node. So we need to move some of the slots to the new node

/redis-cli --cluster reshard 192.168.133.22:8001 --cluster-from f3ab5f5c53b82b8a9df24bfd4a8191224cf63597,faa53cbb6bae288d7f3a19fa55f18d7e77c9 7f6d,d33eab99055e05440a6ba1dd314efb0a25274fb1 --cluster-to 90dcd8e79fd753d24c35ef6ec63e65d79f56600a 3000

  • Cluster-from: indicates the node ID of the node where the slot resides. Use commas (,) to separate multiple ids

  • Clusterto: indicates the node ID of the new node to be assigned (seemingly only one at a time)

  • Cluster-slots: indicates the number of slots allocated

Adding a slave Node

We have added a new node 7007 to the cluster, and if there is only one master node, the high availability will decrease. Then we should add a backup node to this node to improve availability. We add another node 7008, configure it as we did before, and start it

/redis-cli --cluster add-node 192.168.133.22:8008 192.168.133.22:8007 -- cluster-slave --cluster-master-id 90dcd8e79fd753d24c35ef6ec63e65d79f56600a

  • Add-node: The add-node is followed by the new slave and the master corresponding to the slave

  • Cluster-slave: indicates that the slave node is added

  • Cluster-master-id: indicates the node ID of the master corresponding to slave

Contraction of the cluster

  • First delete the slave corresponding to the master

    . / redis - cli - cluster del -node 192.168.133.22:8008 5 e8924bef3cfb0a07838d45da77e9a2e0e61d7b9

    Del-node is followed by the IP address and port number of any node in the cluster (primarily to connect to the cluster, IP :port) and the node ID

  • Clear the master slot

    . / redis - cli - cluster reshard 192.168.133.22:8007 - cluster - from 90 dcd8e79fd753d24c35ef6ec63e65d79f56600a - cluster - the to F3ab5f5c53b82b8a9df24bfd4a8191224cf63597 ƕ cluster - 3000 slots ƕ cluster - yes

    Since our cluster has four primary nodes and the reshard can only write to one destination at a time, the above command needs to be executed three times (ƕcluster-to for different destination nodes).

    –cluster-yes: indicates that the slot to be migrated is not displayed

  • A node is offline (deleted)

    . / redis - cli - cluster del -node 192.168.133.22:8007 90 dcd8e79fd753d24c35ef6ec63e65d79f56600