Redis cluster introduction

Redis began to support Cluster mode after 3.0.

At present, Redis cluster supports automatic node discovery, master-slave election and fault tolerance, online sharding (Reshard) and so on.

Automatic node discovery: Dynamically add a node to a cluster. The node can be automatically added to the cluster.

Master-slave election and fault tolerance: When a Master node fails, a Slave node is selected to replace the original Master node as the new Master node.

Online sharding: see below.

Redis cluster architecture diagram

In a Redis cluster, nodes are connected to each other and check each other’s heartbeat to determine whether each other is down. Heartbeat detection is about communication between nodes, which occupies a large amount of Intranet bandwidth. Therefore Redis optimized the transmission speed and bandwidth of the original communication of the heartbeat mechanism, ping-pang mechanism.

Ping-pang is a binary protocol that is used by Redis to detect heartbeat between nodes. Node1 sends a PING to Node2. If Node2 returns a PANG to Node1, then Node1 determines that Node2 is still healthy.

If more than half of the nodes in the cluster believe that a node is in an unhealthy state and may break down, the cluster considers the node to be down. Therefore, it is recommended to have an odd number of nodes in the Redis cluster.

Clients can connect to any node in the Redis cluster directly without the need for an intermediate Proxy layer. (Tomcat cluster requires Nginx for middle layer and MySQL cluster requires MyCat for middle layer)

Redis clustering principle

First, understand the meanings of a few nouns

  • Physical node: a node that provides external services, namely, the Master node.
  • Slot: Chinese translation is called “Hash Slot “. The Redis cluster is designed with 16,384 Hash slots ranging from 0 to 16,383. When Redis is created, the 16384 Hash slots are evenly distributed among all physical nodes. (For example, there are three Master nodes in the cluster. The first Master node has Slot 0 to 5461, the second Master node has Slot 5462 to 10922, and the third Master node has Slot 10922 to 16383.)
  • CRC16: a Hash consistency algorithm. In cluster mode, the CRC16 algorithm computes all keys sent by clients to the cluster, and the calculation result is always between 0 and 16,383. The results of CRC16 calculation for the same Key for multiple times are always the same. The results of CRC16 calculation for different keys must be repeated (the results for different keys are consistent).

The cluster maintains physical nodes, and each physical node must have at least one Slot. Because there are 16,384 slots at most, the cluster will fail to be created if there are 16,385 physical nodes. Therefore, the maximum number of physical nodes in the Redis cluster is 16,384.

The Slot maintains a key-value. Suppose the client sends a set name John to the Redis cluster. Although the client can connect to any physical node in the cluster, the command does not directly connect to the physical node as in single-machine mode. Place the key-value on the physical node in the Slot range. For example, if the Value of Key is 12, place the key-value on the first Master node because the Slot range of the Master node is 0 to 5461. When a client sends a GET name to a Redis cluster, the CRC16 algorithm is used to calculate the result and then the client retrieves the key-value on the physical nodes in the corresponding Slot range.

Through CRC16 algorithm, each physical node in Redis cluster can hold certain data, so as to solve the problem of physical upper limit of single node. Because each instruction of the client will be computed by CRC16 algorithm, and the connection will be redirected to the corresponding physical node for operation, multiple physical nodes in Redis cluster provide services externally, thus solving the problem of single node pressure.

Because every master-slave architecture in the cluster is recommended as a sentry mechanism architecture, automatic failover is also addressed. (During failover, Slot is moved along with it)

Reassigning slots is called Reshard, which translates as “online resharding”. For example, if there are three physical nodes in the cluster and one physical node needs to be added, the Slot allocated to the three nodes needs to be Reshard. If the Slot is not allocated to the new node, the new node will never provide service because the Slot maintains a key-value. The principle for Slot allocation is specified by the developer. When a Slot is allocated, the key-value of the Slot is also assigned.

Redis cluster setup

Baidu by yourself.