Using Docker to build Redis Cluster, the most important link is the problem of container communication, which we have solved in the previous article “Docker network mode and network communication between containers”. This article mainly practices using multiple containers to complete the construction of Redis Cluster environment, and pave the way for learning Docker Compose. As the saying goes, there is no harm without comparison. Only by comparison can we feel the benefits of Docker Compose 😄.

For more information about Redis Cluster, please read the Most accessible Redis Architecture Pattern in detail.

IO /topics/clus… In order to make Docker compatible with Redis Cluster, you need to use Docker host network mode.

The host network mode needs to be specified by the parameter –net host or –network host when creating the container. The host network mode allows the container to share the network stack of the host. The container does not virtualize its network card or configure its OWN IP address, but uses the IP address and port of the host.

　　

The environment

　　

To make the environment more realistic, this article uses the multi-machine environment:

  • 192.168.10.10
  • 192.168.10.11

　　

The infrastructure environment used by each machine is as follows:

  • CentOS 7.8.2003
  • Docker version 19.03.12

　　

Set up

　　

The overall construction steps are mainly divided into the following steps:

  • Download the Redis image (this step can be omitted, because when creating the container, if the local image does not exist, it will be pulled remotely);
  • Writing Redis configuration files;
  • Create a Redis container;
  • Create a Redis Cluster.

　　

Write the Redis configuration file

　　

Create directories and files

　　

Perform the following operations on 192.168.10.10 and 192.168.10.11 respectively.

#Create a directory
mkdir -p /usr/local/docker-redis/redis-cluster
#Switch to the specified directory
cd /usr/local/docker-redis/redis-cluster/
#Compile the redis-cluster.tmpl file
vi redis-cluster.tmpl
Copy the code

　　

Writing configuration files

　　

192.168.10.10 The contents of the redis-cluster. TMPL file are as follows:

port ${PORT}
requirepass 1234
masterauth 1234
protected-mode no
daemonize no
appendonly yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000
cluster-announce-ip 192.168.10.10
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
Copy the code

192.168.10.11 The contents of the redis-cluster. TMPL file are as follows:

port ${PORT}
requirepass 1234
masterauth 1234
protected-mode no
daemonize no
appendonly yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000
cluster-announce-ip 192.168.10.11
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
Copy the code
  • port: Node port;
  • requirepass: Adds access authentication.
  • masterauth: If access authentication is enabled on the primary node, the secondary node requires authentication to access the primary node.
  • protected-mode: Indicates the protection mode. The default value is yes. This parameter is required after the protection mode is enabledbind ipOr set an access password; If the protected mode is disabled, the external network can be directly accessed.
  • daemonize: Whether to start in daemon mode (background startup), default no;
  • appendonly: Whether to enable AOF persistence mode. Default: no.
  • cluster-enabled: Specifies whether to enable the cluster mode. The default value is no.
  • cluster-config-file: cluster node information file;
  • cluster-node-timeout: Connection timeout duration of the cluster node.
  • cluster-announce-ip: Cluster node IP address, enter the HOST IP address.
  • cluster-announce-port: cluster node mapping port.
  • cluster-announce-bus-port: Bus port of a cluster node.

Each Redis cluster node needs to have two TCP connections open. A normal Redis TCP port for serving clients, such as 6379. There is also a port based on 6379 plus 10000, such as 16379.

The second port is for the cluster bus, which is a node-to-node communication channel using binary protocols. Nodes use the cluster bus for fault detection, configuration updates, failover authorization, and so on. The client should never attempt to communicate with the cluster bus port, just the normal Redis command port, but make sure both ports in the firewall are turned on or the Redis cluster nodes will not be able to communicate.

　　

Run the following command in the redis-cluster directory of 192.168.10.10:

for port in `seq 6371 6373`; do \ mkdir -p ${port}/conf \ && PORT=${port} envsubst < redis-cluster.tmpl > ${port}/conf/redis.conf \ && mkdir -p ${port}/data; \ doneCopy the code

Run the following command in the redis-cluster directory of 192.168.10.11:

for port in `seq 6374 6376`; do \ mkdir -p ${port}/conf \ && PORT=${port} envsubst < redis-cluster.tmpl > ${port}/conf/redis.conf \ && mkdir -p ${port}/data; \ doneCopy the code

Create a directory and file from 6371 to 6376.

　　

Yum install -y tree yum install -y tree yum install -y tree

Run the command on 192.168.10.11. The result is as follows:

　　

The following details the configuration file of each node.

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 192.168.10.10 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = [root @ localhost redis - cluster] # cat /usr/local/docker-redis/redis-cluster/637{1.. 3}/conf/redis.conf port 6371 requirepass 1234 masterauth 1234 protected-mode no daemonize no appendonly yes Cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 15000 cluster-announce-ip 192.168.10.10 cluster-announce-port 6371 cluster-announce-bus-port 16371 port 6372 requirepass 1234 masterauth 1234 protected-mode no daemonize no appendonly yes cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 15000 Cluster-announce-ip 192.168.10.10 cluster-announce-port 6372 cluster-announce-bus-port 16372 port 6373 requirepass 1234 masterauth 1234 protected-mode no daemonize no appendonly yes cluster-enabled yes cluster-config-file nodes.conf Cluster-node-timeout 15000 cluster-announce-ip 192.168.10.10 cluster-announce-port 6373 cluster-announce-bus-port 16373 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 192.168.10.10 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 192.168.10.11  ============================== [root@localhost redis-cluster]# cat /usr/local/docker-redis/redis-cluster/637{4.. 6}/conf/redis.conf port 6374 requirepass 1234 masterauth 1234 protected-mode no daemonize no appendonly yes Cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 15000 cluster-announce-ip 192.168.10.11 cluster-announce-port 6374 cluster-announce-bus-port 16374 port 6375 requirepass 1234 masterauth 1234 protected-mode no daemonize no appendonly yes cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 15000 Cluster-announce-ip 192.168.10.11 cluster-announce-port 6375 cluster-announce-bus-port 16375 port 6376 requirepass 1234 masterauth 1234 protected-mode no daemonize no appendonly yes cluster-enabled yes cluster-config-file nodes.conf Cluster-node-timeout 15000 cluster-announce-ip 192.168.10.11 cluster-announce-port 6376 cluster-announce-bus-port 16376 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 192.168.10.11 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =Copy the code

　　

Create a Redis container

　　

Create a container

　　

Map ports between 6371 and 6376 of the host computer to six Redis containers, and map the directory of the host computer to the directory in the container (directory mount). Remember to specify network mode, use host network mode.

Run the following command on 192.168.10.10:

for port in $(seq 6371 6373); do \
  docker run -di --restart always --name redis-${port} --net host \
  -v /usr/local/docker-redis/redis-cluster/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
  -v /usr/local/docker-redis/redis-cluster/${port}/data:/data \
  redis redis-server /usr/local/etc/redis/redis.conf; \
done
Copy the code

Run the following command on 192.168.10.11:

for port in $(seq 6374 6376); do \
  docker run -di --restart always --name redis-${port} --net host \
  -v /usr/local/docker-redis/redis-cluster/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
  -v /usr/local/docker-redis/redis-cluster/${port}/data:/data \
  redis redis-server /usr/local/etc/redis/redis.conf; \
done
Copy the code

　　

Run the docker ps -n 3 command on host 192.168.10.10 to check whether the container is successfully created.

Run the docker ps -n 3 command on host 192.168.10.11 to check whether the container is successfully created.

　　

Create a Redis Cluster

　　

Enter any container node and go to /usr/local/bin/ :

#Into the container
docker exec -it redis-6371 bash
#Switch to the specified directory
cd /usr/local/bin/
Copy the code

　　

We can then create the Redis Cluster by using the following command.

Redis -cli -a 1234 --cluster create 192.168.10.10:6371 192.168.10.10:6372 192.168.10.10:6373 192.168.10.11:6374 192.168.10.11 192.168.10.11:6375:6376 - cluster - replicas of 1Copy the code

　　

Enter yes as prompted, and the result is as follows:

The cluster is successfully created as follows:

　　

The following is the details returned when the cluster is created, which is everything in the previous two figures.

root@localhost:/usr/local/bin# redis-cli -a 1234 --cluster create 192.168.10.10:6371 192.168.10.10:6372 192.168.10.10:6373 192.168.10.11:6374 192.168.10.11:6375 192.168.10.11:6376 --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 192.168.10.11:6376 to 192.168.10.10:6371 Adding Replica 192.168.10.10:6373 to 192.168.10.11:6374 Adding Replica 192.168.10.11:6375 to 192.168.10.10:6372 M: 299 cf79ddafc83dced27f628f1f82dac483fbc4e 192.168.10.10:6371 slots: [0-5460] (5461 slots) master M: Ac805b90b6e20e26dc4268454bb2855beea6cc19 192.168.10.10:6372 slots: [10923-16383] (5461 slots) master S: Db35494fcc5db0c88d27da7885c817e6cdcc9373 192.168.10.10:6373 replicates d37eeab79b9cd0272e934d4548136a 7013270480 M: 7013270480 d37eeab79b9cd0272e934d4548136a 192.168.10.11:6374 slots: [5461-10922] (5462 slots) master S: 8435 e1b0d51f2690c5f94f9a5682a4ac34e94326 192.168.10.11:6375 replicates ac805b90b6e20e26dc4268454bb2855beea6cc19 S: 7 b13c16fa6fe8e13cdc0b4846b87edffed55c62e 192.168.10.11:6376 replicates 299 cf79ddafc83dced27f628f1f82dac483fbc4e Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.10.10:6371)M: 299 cf79ddafc83dced27f628f1f82dac483fbc4e 192.168.10.10:6371 slots: [0-5460] (5461 slots) master 1 additional up (s) S: 8435 e1b0d51f2690c5f94f9a5682a4ac34e94326 192.168.10.11:6375 slots: (0 slots) slave replicates ac805b90b6e20e26dc4268454bb2855beea6cc19 S: Db35494fcc5db0c88d27da7885c817e6cdcc9373 192.168.10.10:6373 slots: (0 slots) slave replicates 7013270480d37eeab79b9cd0272e934d4548136a S: 7 b13c16fa6fe8e13cdc0b4846b87edffed55c62e 192.168.10.11:6376 slots: (0 slots) slave replicates 299cf79ddafc83dced27f628f1f82dac483fbc4e M: 7013270480 d37eeab79b9cd0272e934d4548136a 192.168.10.11:6374 slots: [5461-10922] (5462 slots) 1 additional master replica(s) M: Ac805b90b6e20e26dc4268454bb2855beea6cc19 192.168.10.10:6372 slots: [10923-16383] (5461 slots) 1 additional master replica(s) [OK] All nodes agree about slots configuration.>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Copy the code

　　

So far, a highly available Redis Cluster has been built, as shown in the figure below. This Cluster contains six Redis nodes, three primary and three secondary. The three master nodes allocate slots to handle client command requests, while the slave nodes can be used to replace the master node if it fails.

　　

Viewing Cluster Status

　　

Let’s go into the container and check the status of the cluster using some common cluster commands.

#Into the container
docker exec -it redis-6371 bash
#Switch to the specified directory
cd /usr/local/bin/
Copy the code

　　

Checking cluster Status

　　

Redis -cli -a 1234 --cluster check 192.168.10.11:6375Copy the code

　　

View cluster information and node information

　　

#Connects to a node in the clusterRedis -cli -c a 1234 -h 192.168.10.11 -p 6376#Viewing Cluster Information
cluster info
#View cluster node information
cluster nodes
Copy the code

　　

SET/GET

　　

Execute write and read on node 6371 with the following command:

#Enter the container and connect to a node in the clusterDocker exec it redis-6371 /usr/local/bin/redis-cli -c a 1234 -h 192.168.10.10 -p 6371#Write data
set name mrhelloworld
set aaa 111
set bbb 222
#Read the data
get name
get aaa
get bbb
Copy the code

Don’t worry, let me explain how it works in the image above:

  • First enter the container and connect to a node in the cluster.
  • Then performThe first oneThe set commandset name mrhelloworld.nameThe key is evaluated by the hash function[5798]. Slot allocation in the current cluster environment is as follows:A scale of 0-5460 to 6371 nodes.[5461-10922] 6374 nodes.[10923-16383] 6372 nodes, so the storage for the key is allocated6374Nodes;
  • Look atThe secondThe set commandset aaaYou might have some questions here, why don’t you see itaaaWhat’s the value of the hash function for the key? Because I just redirected to6374The node inserts data. If there is still data to insert, and the value obtained by the hash function of the key is still in the range of the node, then the data can be directly inserted.
  • Then there isThe thirdThe set commandset bbb.bbbThe key is evaluated by the hash function[5287], so the storage for the key is allocated6371Nodes;
  • And then the read operation,The fourthThe commandget name.nameThe key is evaluated by the hash function[5798]Is redirected to6374Node reading;
  • The fifthThe commandget aaa.aaaThe value of the hash function of the key is also in6374Node, read directly;
  • The sixthThe commandget bbb.bbbThe key is evaluated by the hash function[5287]Is redirected to6371Node read.

　　

From the above we know that the storage of the name key is allocated to node 6374. What if we connect to node 6374 directly and get the value? That’s right, there’s no need to redirect the node, because the data is there, so it just reads and returns.

　　

Client connection

　　

Finally, a wave of client connection operations will be performed on any node to see if the Redis Cluster can be accessed externally.

In fact, the overall construction process is not particularly troublesome, because:

  • To create a Redis cluster, you need To use Ruby, otherwise you have to associate nodes to build the cluster and allocate slots by yourself.
  • If you use Ruby to build a Redis cluster, you need to install a Ruby environment;
  • Redis is available directly from version 5redis-cliCommand to create a cluster, save a lot of hassle;
  • We also simplified the build process by using shell for loop statements that would otherwise be annoying to execute one by one.

Given all this, is there an easier way? Of course I do, or I wouldn’t be here to sell you anything.

Docker Compose solves this problem. Docker Compose: Docker Compose: Docker Compose: Docker Compose: Docker Compose: Docker Compose: Docker Compose: Docker Compose

This article is licensed under a Creative Commons attribution – Noncommercial – No Deductive 4.0 International license.

You can check out more Docker articles in the category below.

　　

🤗 your likes and retweets are the biggest support for me.

📢 Scan code pay attention to Mr. Hallward “document + video” each article is equipped with a special video explanation, learning more easily oh ~