Earlier we talked about the 13 Catch-22 of Redis Performance Optimization! , one of the most important is to use Redis clustering capabilities, so in this article we will see how to create a Redis cluster in 1s minutes.

Redis Cluster is a Redis Cluster solution released by Redis 3.0. It distributes data in different service areas to reduce the dependence of the system on a single master node and greatly improve the read and write performance of Redis service.

Redis divides all the data into 16384 slots. Each node is responsible for some of the slots. When a Redis client connects to a cluster, it gets a list of the cluster’s slots so that it can send requests directly to the corresponding node for processing.

Redis Cluster is a decentralized operation mode without agent mode. The vast majority of commands sent by the client will be directly handed over to the relevant nodes for execution, so that the request and response can be completed without forwarding or only forwarding once in most cases. Therefore, the performance of a single Cluster node is very close to that of a single Redis server. Therefore, in theory, when the horizontal expansion of the master node is doubled, the performance of request processing is also doubled, so the performance of Redis Cluster is very high.

The Redis Cluster architecture diagram is shown below:

Building Redis Cluster

There are two ways to build a Redis Cluster, one is to use the create-cluster tool provided by Redis source code to quickly build a Redis Cluster environment, the other is to manually create a Redis Cluster environment configuration file.

1. Quickly set up Redis Cluster

The create-cluster tool is in the utils/create-cluster directory, as shown in the following figure:

./create-cluster start 

$ ./create-cluster start Create cluster
Starting 30001
Starting 30002
Starting 30003
Starting 30004
Starting 30005
Starting 30006
Copy the code

Next, we need to create a cluster of the above six nodes by using the create command as follows:

[@ iZ2ze0nc5n41zomzyqtksmZ: create - cluster] $. / the create - cluster create # form a cluster>>> Performing hash slots allocation on 6 nodes...Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:30005 to 127.0.0.1:30001 Adding Replica 127.0.0.1:30006 to 127.0.0.1:30002 Adding Replica 127.0.0.1:30004 to 127.0.0.1:30005 to 127.0.0.1:30001 Adding Replica 127.0.0.1:30006 to 127.0.0.1:30002 Adding Replica 127.0.0.1:30004 to 127.0.0.1:30003>>> Trying to optimize slaves allocation for anti-affinity[WARNING] Some slaves are in the same host as their master M: 445 f2a86fe36d397613839d8cc1ae6702c976593 127.0.0.1:30001 slots: [0-5460] (5461 slots) master M: 63 bb14023c0bf58926738cbf857ea304bff8eb50 127.0.0.1:30002 slots: [5461-10922] (5462 slots) master M: 864 d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 127.0.0.1:30003 slots: [10923-16383] (5461 slots) master S: 64828 ab44566fc5ad656e831fd33de87be1387a0 127.0.0.1:30004 replicates 445 f2a86fe36d397613839d8cc1ae6702c976593 S: 0 b17b00542706343583aa73149ec5ff63419f140 127.0.0.1:30005 replicates 63 bb14023c0bf58926738cbf857ea304bff8eb50 S: E35f06ca9b700073472d72001a39ea4dfcb541cd 127.0.0.1:30006 864 d4dfe32e3e0b81a64cec8b393bbd26a65cbcc replicates Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 127.0.0.1:30001)M: 445 f2a86fe36d397613839d8cc1ae6702c976593 127.0.0.1:30001 slots: [0-5460] (5461 slots) master 1 additional up (s) M: 864 d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 127.0.0.1:30003 slots: [10923-16383] (5461 slots) master 1 additional up (s) S: e35f06ca9b700073472d72001a39ea4dfcb541cd 127.0.0.1:30006 slots: (0 slots) slave replicates 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc S: 0 b17b00542706343583aa73149ec5ff63419f140 127.0.0.1:30005 slots: (0 slots) slave replicates 63bb14023c0bf58926738cbf857ea304bff8eb50 M: 63 bb14023c0bf58926738cbf857ea304bff8eb50 127.0.0.1:30002 slots: [5461-10922] (5462 slots) master 1 additional up (s) S: 64828 ab44566fc5ad656e831fd33de87be1387a0 127.0.0.1:30004 slots: (0 slots) slave replicates 445f2a86fe36d397613839d8cc1ae6702c976593 [OK] All nodes agree about slots configuration.>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Copy the code

During the execution, it will ask you whether to set 30001, 30002, and 30003 as the primary node and 30004, 30005, and 30006 as their secondary nodes. After entering yes, the execution will complete.

We can first connect to the cluster using redis-cli command as follows:

$ redis-cli -c -p 30001
Copy the code

Run the nodes command to view information about nodes in the cluster:

127.0.0.1:30001 > cluster nodes 864 d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 127.0.0.1:30003 @ 40003 master - 0 1585125835078 3 Connected to 10923-16383 e35f06ca9b700073472d72001a39ea4dfcb541cd 127.0.0.1:30006 @ 40006 slave 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 0 1585125835078 6 connected 0b17b00542706343583aa73149ec5ff63419f140 127.0.0.1:30005 @ 40005 slave bb14023c0bf58926738cbf857ea304bff8eb50 0 1585125835078 5 63 connected 63 bb14023c0bf58926738cbf857ea304bff8eb50 127.0.0.1:30002 @ 40002 master - 0 1585125834175 2 connected. 5461-10922 445 f2a86fe36d397613839d8cc1ae6702c976593 127.0.0.1:30001 @ 40001 myself, master 1585125835000 connected 1 0 0-5460 64828 ab44566fc5ad656e831fd33de87be1387a0 127.0.0.1:30004 @ 40004 slave f2a86fe36d397613839d8cc1ae6702c976593 0 445 1585125835000 4 connectedCopy the code

You can see that 30001, 30002, and 30003 are active nodes. Slot 30001 is 0-5460, slot 30002 is 5461-10922, and slot 30003 is 10923-16383. A total of 16,384 slots (0 to 16,383) are available.

Create-cluster is fast, but it can only be used in test environments because the number of primary and secondary nodes in the cluster is fixed and the slot allocation mode is fixed. Create-cluster is installed on the same server.

After we have finished testing, we can use the following command to shut down and clean up the cluster:

$ ./create-cluster stop # shutdown cluster
Stopping 30001
Stopping 30002
Stopping 30003
Stopping 30004
Stopping 30005
Stopping 30006
$ ./create-cluster clean # Clear the cluster
Copy the code

2. Manually set up the Redis Cluster

Due to the limitations of create-cluster, in a production environment we need to manually add the configuration to build the Redis cluster. To do this, we need to copy the Redis installation package to node1 to node6 files, because we need to install 6 nodes, 3 master nodes and 3 slave nodes. As shown below:

1 Set the configuration file

We need to modify the redis. Conf file in each node, set cluster-enabled yes to enable the cluster mode, and modify the respective ports. We continue to use 30001 to 30006, through port 3000X Settings.

② Start each node

After redis. Conf is configured, we can start all the nodes with the following command:

cd /usr/local/soft/mycluster/node1 
./src/redis-server redis.conf
Copy the code

3 Create a cluster and allocate slots

We have started 6 nodes before, but these nodes are not interconnected within their respective clusters. Therefore, we need to connect these nodes into a cluster and assign corresponding slots to them. Run the following command:

Redis-cli --cluster create 127.0.0.1:30001 127.0.0.1:30002 127.0.0.1:30003 127.0.0.1:30004 127.0.0.1:30005 127.0.0.1:30006-1 - cluster - replicasCopy the code

Create is followed by multiple nodes. Cluster-replicas specifies the number of slave nodes for the primary node in the cluster. 1 indicates that each primary node has one slave node.

After executing the create command, the system will specify the role and slot allocation plan for the node, as shown below:

>>> Performing hash slots allocation on 6 nodes...Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:30005 to 127.0.0.1:30001 Adding Replica 127.0.0.1:30006 to 127.0.0.1:30002 Adding Replica 127.0.0.1:30004 to 127.0.0.1:30005 to 127.0.0.1:30001 Adding Replica 127.0.0.1:30006 to 127.0.0.1:30002 Adding Replica 127.0.0.1:30004 to 127.0.0.1:30003>>> Trying to optimize slaves allocation for anti-affinity[WARNING] Some slaves are in the same host as their master M: Bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30001 slots: [0-5460] (5461 slots) master M: Bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30002 slots: [5461-10922] (5462 slots) master M: Bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30003 slots: [10923-16383] (5461 slots) master S: Bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30004 replicates bdd1c913f87eacbdfeabc71befd0d06c913c891c S: Bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30005 replicates bdd1c913f87eacbdfeabc71befd0d06c913c891c S: Bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30006 replicates bdd1c913f87eacbdfeabc71befd0d06c913c891c Can I set the above configuration? (type 'yes' to accept):Copy the code

As can be seen from the above information, Redis intends to set 30001, 30002, and 30003 as the primary nodes, and assign slots to them. Slots corresponding to 30001 are 0-5460, and slots corresponding to 30002 are 5461-10922. The slot 30003 is 10923-16383, and the secondary node 30005 is set to 30001, the secondary node 30006 is set to 30002, and the secondary node 30004 is set to 30003. You only need to enter yes to confirm the assignment, as shown below:

Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 127.0.0.1:30001)M: 887397 e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001 slots: [0-5460] (5461 slots) master 1 additional up (s) s: Abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005 slots: (0 slots) slave replicates 887397e6fefe8ad19ea7569e99f5eb8a803e3785 S: 1 a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 slots: (0 slots) slave replicates f5958382af41d4e1f5b0217c1413fe19f390b55f S: Dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006 slots: (0 slots) slave replicates 3da35c40c43b457a113b539259f17e7ed616d13d M: 3 da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002 slots: [5461-10922] (5462 slots) master 1 additional up (s) M: F5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003 slots: [10923-16383] (5461 slots) master 1 additional up (s)  [OK] All nodes agree about slots configuration.>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Copy the code

If OK is displayed, the entire cluster has been successfully started.

Next, we use redis-CLI to connect and test the running status of the cluster. The code is as follows:

$ redis-cli -c -p 30001 Connect to the cluster127.0.0.1:30001> cluster info # Check cluster information Cluster_State: OK cluster_SLOts_assigned :16384 # Slot cluster_SLOts_OK :16384 Cluster_slots_pfail :0 Cluster_SLOts_FAIL :0 Cluster_Known_nodes :6 # Number of nodes in the cluster Cluster_size :3 # Number of primary nodes in the cluster cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:130 cluster_stats_messages_pong_sent:127 cluster_stats_messages_sent:257 cluster_stats_messages_ping_received:122 cluster_stats_messages_pong_received:130 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:257Copy the code

Descriptions of related fields are already identified in the above code and will not be repeated here.

Dynamically add and delete nodes

In some cases, you need to dynamically add or remove nodes from an existing cluster based on actual service conditions. In this case, you need to perform the following operations.

1. Add an active node

Add method 1: Cluster meet

To add a node to a cluster, run the cluster meet IP :port command:

127.0.0.1:30001 > cluster meet 127.0.0.1 30007 OK 127.0.0.1:30001 > cluster nodes dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006 @ 40006 slave 3 da35c40c43b457a113b539259f17e7ed616d13d 0 1585142916000 6 connected Df0190853a53d8e078205d0e2fa56046f20362a7 127.0.0.1:30007 @ 40007 master - 0, 1585142917740 connected F5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003 @ 40003 master - 0 1585142916738 3 connected. 10923-16383 3 da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002 @ 40002 master - 0 1585142913000 2 connected. 5461-10922 Abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005 @ 40005 slave e6fefe8ad19ea7569e99f5eb8a803e3785 0 887397 1585142917000 5 connected 887397 e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001 @ 40001 myself, master 1585142915000-0 1 connected 0-5460-1 a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 @ 40004 slave f5958382af41d4e1f5b0217c1413fe19f390b55f 0 1585142916000 4 connectedCopy the code

You can see that the node with port 30007 is added to the cluster and set as the primary node.

Add method 2: Add-node

Run redis-cli –cluster add-node to add a node IP address :port IP address of a node in the cluster :port You can also add a node to the cluster by running the following command:

$Redis -cli --cluster add-node 127.0.0.1:30008 127.0.0.1:30001
>>> Adding node 127.0.0.1:30008 to cluster 127.0.0.1:30001
>>> Performing Cluster Check (using node 127.0.0.1:30001)M: 887397 e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001 slots: [0-5460] (5461 slots) master 1 additional up (s) s: Dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006 slots: (0 slots) slave replicates 3da35c40c43b457a113b539259f17e7ed616d13d M: Df0190853a53d8e078205d0e2fa56046f20362a7 127.0.0.1:30007 slots: (0 slots) master M: F5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003 slots: [10923-16383] (5461 slots) master 1 additional up (s) M: 1 d09d26fd755298709efe60278457eaa09cefc26 127.0.0.1:30008 slots: (0 slots) master M: 3 da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002 slots: [5461-10922] (5462 slots) master 1 additional up (s) S: abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005 slots: (0 slots) slave replicates 887397e6fefe8ad19ea7569e99f5eb8a803e3785 S: 1 a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 slots: (0 slots) slave replicates f5958382af41d4e1f5b0217c1413fe19f390b55f [OK] All nodes agree about slots configuration.>>> Check for open slots...
>>> Check slots coverage...[OK] All 16384 slots covered. [ERR] Node 127.0.0.1:30008 is not empty. Either the Node already knows other Nodes (Check) with CLUSTER NODES) or contains some key in database 0.Copy the code

From the above results, you can see that 30008 node is also set as the primary node.

2. Add a secondary node

To set the current node to the secondary node of the target node, run the cluster replicate nodeId command.

127.0.0.1:30008 > cluster replicate df0190853a53d8e078205d0e2fa56046f20362a7 OK 127.0.0.1:30008 > cluster nodes Df0190853a53d8e078205d0e2fa56046f20362a7 127.0.0.1:30007 @ 40007 master - 0, 1585147827000 connected Abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005 @ 40005 slave e6fefe8ad19ea7569e99f5eb8a803e3785 0 887397 1585147827000 1 connected a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 @ 40004 slave f5958382af41d4e1f5b0217c1413fe19f390b55f 0 1585147823000 3 connected 887397e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1: master 30001 @ 40001-0, 1585147826000 1 connected dc0702625743c48c75ea935c87813c4060547cef 0-5460 127.0.0.1:30006 @ 40006 slave 3 da35c40c43b457a113b539259f17e7ed616d13d 0 1585147826930 2 connected F5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003 @ 40003 master - 0 1585147826000 3 connected. 10923-16383 1 d09d26fd755298709efe60278457eaa09cefc26 127.0.0.1:30008 @ 40008 myself, slave df0190853a53d8e078205d0e2fa56046f20362a7 0 1585147823000 7 connected 3 da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002 @ 40002 master - 0 1585147827933 2 connected 5461-10922Copy the code

You can see that 30008 has become the slave node of 30007.

3. Delete the node

To remove a node from a cluster, run the cluster forget nodeId command. When this command is different from the meet command, the node Id needs to be used to delete the node. You can run the Cluster Nodes command to view the Id information of all nodes, where the combination of the first 40 letters and array in each line is the NODE Id, as shown in the following figure:

127.0.0.1:30001 > cluster forget df0190853a53d8e078205d0e2fa56046f20362a7 OKCopy the code

Run the cluster nodes command to view information about all nodes in the cluster:

127.0.0.1:30001 > cluster nodes dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006 @ 40006 slave 3da35c40c43b457a113b539259f17e7ed616d13d 0 1585143789940 6 connected f5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1: master 30003 @ 40003-0 1585143791000 3 connected 3 da35c40c43b457a113b539259f17e7ed616d13d. 10923-16383 127.0.0.1:30002 @ 40002 master - 0 1585143789000 2 connected abec9f98f9c01208ba77346959bc35e8e274b6a3. 5461-10922 127.0.0.1:30005 @ 40005 slave e6fefe8ad19ea7569e99f5eb8a803e3785 0 1585143789000 5 887397 connected 887397 e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001 @ 40001 myself, master 1585143786000 connected 1 0 0-5460 1 a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 @ 40004 slave f5958382af41d4e1f5b0217c1413fe19f390b55f 0 1585143791945 4 connectedCopy the code

It can be seen that the node with the previous port 30007 has been successfully removed.

summary

Create-cluster start and Cluster create are two ways to build a Redis cluster: create-cluster start and cluster create. Although the former method is relatively fast, it can only create a fixed number of primary and secondary nodes, and all nodes are on the same server, so it can only be used in the test environment. We also looked at the Redis cluster’s ability to dynamically add master and slave nodes and remove arbitrary nodes.

I hope this article has been helpful to you

Follow the qr code below and subscribe for more exciting content.