Abstract

Two VMS (IP address: 10.60.44.76, IP address: 10.60.44.105) with six nodes, one VM has three nodes, and three master and three Salve environments are simulated

  • Redis version: RedIS-3.2.4
  • Linux version: centos6.5 (IP: 10.60.44.76), centos7.0 (IP: 10.60.44.105)

Note:

  • A Redis instance is a node
  • For Linux, it is best to operate under root
  • The minimum redis-cluster configuration is three active and three standby nodes
  • Disable the firewall (The specified ports need to be opened for communication between nodes. For convenience, do not use the firewall in the production environment)

Centos 6.5

Service iptables stop command: chkconfig iptables off service iptables status run the two commands at the same time and check whether the firewall is disabledCopy the code

Centos 7.0

Firewall systemctl stop firewalld. Service # stop firewall systemctl disable firewalld. Service # Stop firewall start firewall-cmd --state # check whether the default firewall status is notrunning or runningCopy the code

1. Download the installation package

The installation package is attached redis-3.2.4.tar.gz. It can also be downloaded using the WGET plug-in

Wget HTTP: / / http://download.redis.io/releases/redis-3.2.4.tar.gzCopy the code

2. Single point installation and deployment

A single point refers to an instance of Redis. You need to download the source code of Redis, compile it, install it, and start the instance by command

Decompress, compile, install

[root@localhost Desktop]# tar ZXVF redis-3.2.4.tar.gz [root@localhost Desktop]# CD redis-3.2.4 [root@localhost Redis-3.2.4]# make installCopy the code

Create related directories

#Save redis command files
[root@localhost src]# mkdir -p /usr/local/redis/bin
#Store the redis configuration file
[root@localhost src]# mkdir -p /usr/local/redis/etc
Copy the code

Move commands, configuration files

#Move the Redis configuration file (configuration file in the unzipped directory)
[root@localhost redis-3.2.4]# mv redis.conf /usr/local/redis/etc
#Move redis command file (command file in SRC directory in unzipped directory)
[root@localhost src]# mv mkreleasehdr.sh redis-benchmark redis-check-aof redis-check-rdb redis-cli redis-server redis-sentinel redis-trib.rb /usr/local/redis/bin
Copy the code

Start the service

#Start Redis by specifying the configuration file
[root@localhost src]# /usr/local/redis/bin/redis-server /usr/local/redis/etc/redis.conf
Copy the code

Check the status

#Or the netstat tunpl | grep 6379
ps -ef | grep redis
Copy the code

3. Cluster construction and deployment

Create a cluster node folder

The folder named 7031-7033 is placed on the machine 10.60.44.76 and the folder 7034-7036 is placed on the machine 10.60.44.105

mkdir -p /usr/local/redis-cluster
cd /usr/local/redis-cluster
#Create individual node foldersMkdir 7034 7035 7036 # mkdir 7031 7032 7033 # mkdir 7034 7035 7036 # mkdir 7031 7032 7033 # mkdir 7034 7035 7036 # mkdir 10.60.44.105Copy the code

Modify or copy the configuration file

This section uses the 10.60.44.76 machine as an example. The operations on the other machine are the same

cp /usr/local/redis/etc/redis.conf /usr/local/redis-cluster/7031
cd /usr/local/redis-cluster/7031
vi redis.conf 
Copy the code

The following is an example of modifying a configuration file

Port 7031 # bind local IP # physical IP: For example, 10.60.44.76 dir /usr/local/redis-cluster/7031 # Specify data storage path cluster-enabled yes # enable cluster mode cluster-config-file Conf # set cluster node configuration file daemonize yes # start cluster-node-timeout 5000 # set cluster node timeout time appendonly yes # set persistent modeCopy the code

Conf file of 7031, copy it to the remaining two directories (7032 and 7033), and replace 7031 in redis.conf with node numbers (7032 and 7033) globally. Similarly, 10.60.44.105 machine

The structure of the /usr/local/ directory whose IP address is 10.60.44.76 is as follows:

[root @ localhost local] # CD/usr/local / && tree - L 3. ├ ─ ─ bin... │ ├── Red flag │ ├─ Red flag │ ├─ Red flag │ ├─ Red flag │ ├─ Red flag │ ├─ Red flag │ ├─ Red flag │ red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ Red Flag │ ├ ─ ─ redis - cli │ │ ├ ─ ─ redis - sentinel │ │ ├ ─ ─ redis - server │ │ └ ─ ─ redis - trib. Rb │ └ ─ ─ etc │ └ ─ ─ redis. Conf ├ ─ ─ Redis - cluster │ ├ ─ ─ 7031 │ │ └ ─ ─ redis. Conf │ ├ ─ ─ 7032 │ │ └ ─ ─ redis. Conf │ └ ─ ─ 7033 │ └ ─ ─ redis. Conf... └ ─ ─ the SRCCopy the code

Install Ruby and start the plug-in

To start a cluster, you need to use the Ruby plugin redis-trib.rb, so you need to install the Ruby environment and its plug-in.

#Step 1: You are not advised to install the system using yum. Use the form of online download source code installation, version can be a little bit newer
yum install ruby
#Step 2: Usually you can't go wrong
yum install rubygems
#Step 3: Error prone, the following common solutions
gem install redis
Copy the code

This step requires a lot of environment and is prone to error. Here are some common error solutions:

No such file to load — zlib

Go to the Ruby source folder and install the Zlib package provided by Ruby itself

[root@localhost ruby-2.3.3]# CD ext/zlib [root@localhost zlib]# ruby./extconf.rb [root@localhost zlib]# make [root@localhost zlib]# make installCopy the code

–with-openssl-include=/usr/local/ SSL /include/ –with-openssl-lib=/usr/local/ SSL /lib –with-openssl-lib=/usr/local/ SSL /lib

The processing method is the same as error 1:

[root@localhost ruby-2.3.3] #cd ext/openssl
[root@localhost zlib]# ruby ./extconf.rb
[root@localhost zlib]# make
[root@localhost zlib]# make install
Copy the code

Unable to require OpenSSL, install openssl and rebuild Ruby (preferred) or use non-https sources

Check $openSSL version to see if openSSL is installed

[root@localhost /]# openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013
Copy the code

If no, a new OpenSSL installation will be performed. If yes, change the gem source address:

#Look at the gem source
[root@localhost /]# gem source
#Delete the HTTPS source
[root@localhost /]# gem source -r https://rubygems.org/ to remove
#Add the HTTP source
[root@localhost /]# gem source -a http://rubygems.org/ to read
Copy the code

Start a cluster node

Run the following command on each machine to start each node in the cluster

 # 10.60.44.76for((i=1; i<=3; i++)); do /usr/local/redis/bin/redis-server /usr/local/redis-cluster/703$i/redis.conf; done# 10.60.44.105for((i=4; i<=6; i++)); do /usr/local/redis/bin/redis-server /usr/local/redis-cluster/703$i/redis.conf; doneCopy the code

View the status of each node

Check the node status

ps -ef | grep redis-server
Copy the code

The results are as follows:

## # 10.60.44.76root 18992 1 0 04:37 ? 00:00:09 / usr/local/redis/bin/redis - server 10.60.44.76:7031 / cluster root 18994 1 0 04:37? 00:00:09 / usr/local/redis/bin/redis - server 10.60.44.76:7032 / cluster root 18998 1 0 04:37? 00:00:10 / usr/local/redis/bin/redis - server 10.60.44.76:7033 [their] root 41471 16554 0 05:29 PTS / 2 00:00:00 grep redis-server## # 10.60.44.105root 50565 1 0 04:37 ? 00:00:07 / usr/local/redis/bin/redis - server 10.60.44.105:7034 / cluster root 50567 1 0 04:37? 00:00:08 / usr/local/redis/bin/redis - server 10.60.44.105:7035 / cluster root 50571 1 0 04:37? 00:00:08 / usr/local/redis/bin/redis - server 10.60.44.105:7036 [their] root 51273 34592 0 05:29 PTS / 1 00:00:00 grep --color=auto redis-serverCopy the code

Create the cluster

After each node is normally started, it can run on either of the two machines:

/ usr/local/redis/bin/redis - trib. Rb create -- replicas 10.60.44.76:1 7031 10.60.44.76:7032 10.60.44.76:7033 10.60.44.105 10.60.44.105:7034 10.60.44.105:7035:7036Copy the code

Check the cluster status

Log in to the client and test the read and write (remember to add the -c option, otherwise an error will be reported)

# 10.60.44.76/ usr/local/redis/bin/redis - 10.60.44.76 cli - c - h - p 7031 cluster info # show cluster informationCopy the code

If cluster_STATE: OK is displayed, the cluster is successfully set up

You can use Cluster Nodes to view the current status of each node in a cluster. You can view the allocation, connection, and status of primary and secondary nodes. As follows:

10.60.44.76:7031 > cluster nodes 1212605 fcb1dd351d07d167c26b368e992c85723 10.60.44.76:7032 slave 1baa876789f68ed2e273ab7cb1d4e954fa7fc60c 0 1498187489957 11 connected 396e92e2e8207be6a4a93e29fd16670656477131 10.60.44.105:7036 master - 0 1498187486928 9 connected ecfb611a8201ce2bf40a2bf0ce398704083baf 37, 5461-10922 10.60.44.76:7031 slave e92e2e8207be6a4a93e29fd16670656477131 0 1498187488947 396 9 connected 82326 f032fc040d7ed59f87ee9ddd500ad19d704 10.60.44.76: slave dfc7bdbef98f5d5f07d6b1fcd6474dc 0 1498187483893 135173619 7033 6 connected 1 baa876789f68ed2e273ab7cb1d4e954fa7fc60c 10.60.44.105: master 7034-0. 1498187487937 11 connected 0-5460 135173619 dfc7bdbef98f5d5f07d6b1fcd6474dc 10.60.44.105:7035 myself, master - 0 0 3 connected. 10923-16383Copy the code

Shut down the cluster

To shut down a cluster, you need to shut down nodes one by one. You can use scripts to do this automatically:

# 10.60.44.76for((i=1; i<=3; i++)); Do/usr/local/redis/bin/redis - 10.60.44.76 cli - c - h - p 703 $I shutdown; done# 10.60.44.105for((i=4; i<=6; i++)); Do/usr/local/redis/bin/redis - 10.60.44.105 cli - c - h - p 703 $I shutdown; doneCopy the code

Deleting a Cluster File

Delete the automatically generated configuration file and database before creating a cluster. Run the following command:

# 10.60.44.76for((i=1; i<=3; i++)); do rm -f /usr/local/redis-cluster/703$i/dump.rdb /usr/local/redis-cluster/703$i/redis-703$i.conf /usr/local/redis-cluster/703$i/appendonly.aof; done# 10.60.44.105for((i=4; i<=6; i++)); do rm -f /usr/local/redis-cluster/703$i/dump.rdb /usr/local/redis-cluster/703$i/redis-703$i.conf /usr/local/redis-cluster/703$i/appendonly.aof; doneCopy the code

4. Cluster operations

Cluster nodes exist as follows:

#7034 (master) - 7032 (slave)
#7035 (master) - 7033 (slave)
#7036 (master) - 7031 (slave)82326 f032fc040d7ed59f87ee9ddd500ad19d704 10.60.44.76: slave dfc7bdbef98f5d5f07d6b1fcd6474dc 0 1498199288536 135173619 7033 6 connected 396 e92e2e8207be6a4a93e29fd16670656477131 10.60.44.105:7036 master - 0 1498199282454 9 connected. 5461-10922 135173619 dfc7bdbef98f5d5f07d6b1fcd6474dc 10.60.44.105: master 7035-0 1498199289546 3 connected. 10923-16383 1 baa876789f68ed2e273ab7cb1d4e954fa7fc60c 10.60.44.105: master 7034-0. 1498199285499 11 connected 0-5460 37 ecfb611a8201ce2bf40a2bf0ce398704083baf 10.60.44.76:7031 myself, slave e92e2e8207be6a4a93e29fd16670656477131 0 0 1 396 Connected 1212605 fcb1dd351d07d167c26b368e992c85723 10.60.44.76:7032 slave 1 baa876789f68ed2e273ab7cb1d4e954fa7fc60c 0 1498199287523 11 connectedCopy the code

Add a primary node

To add a new master node on top of the existing cluster, start the new node, add the new node to the cluster and allocate slots, as described below:

1. Create files (folders and configuration files) required by the node.

[root@localhost /]# mkdir /usr/local/redis-cluster/7037
Copy the code

2. Copy and modify the configuration file

Replace all the 7036 characters in the configuration file with 7037

[root@localhost /]# cp /usr/local/redis-cluster/7036/redis.conf  /usr/local/redis-cluster/7037/
Copy the code

3. Start the newly created node

[root@localhost /]# /usr/local/redis/bin/redis-server /usr/local/redis-cluster/7037/redis.conf
Copy the code

4. Add the new node to the existing cluster as a primary node

#Parameter 1: IP address and port of the new node to be added. Parameter 2: IP address and port number of any online node in the existing cluster
[root@localhost /]# redis-trib.rb add-node 10.60.44.105:7037 10.60.44.105:7034
Copy the code

If successful, the following information is displayed:

>>> Adding node 10.60.44.105:7037 to cluster 10.60.44.105:7034
>>> Performing Cluster Check (using node 10.60.44.105:7034). omit>>> Send CLUSTER MEET to node 10.60.44.105:7037 to make it join the cluster.
[OK] New node added correctly.
Copy the code

5. View information about the cluster node after the new node is added

3 f2c74e5888907c494b8b728210175144657f218 10.60.44.105:7037 master - 0, 1498199380639 connected 396 e92e2e8207be6a4a93e29fd16670656477131 10.60.44.105:7036 master - 0 1498199384698 9 connected. 5461-10922 135173619 dfc7bdbef98f5d5f07d6b1fcd6474dc 10.60.44.105: master 7035-0 1498199381655 3 connected. 10923-16383 1 baa876789f68ed2e273ab7cb1d4e954fa7fc60c 10.60.44.105: master 7034-0. 1498199383685 11 connected 0-5460Copy the code

6. Assign a hash slot to the new node

[root @ localhost /] # / usr/local/redis/bin/redis - trib. Rb reshard 10.60.44.105:7037#How many slots are allocated to the new node
How many slots do you want to move (from 1 to 16384)? 1000
#Which is the accept node? (Fill in the ID of the new node)What is the receiving node ID? 3f2c74e5888907c494b8b728210175144657f218 Please enter all the source node IDs. Type 'all' to use all the nodes as source  nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.#Randomly selected from all primary nodes
Source node #1: all
Copy the code

7. View the cluster node information after the slot is allocated

3 f2c74e5888907c494b8b728210175144657f218 10.60.44.105: master 7037-0 12 connected a scale of 0-332 to 1498201081749, 5461-5794 10923-11255, 396 e92e2e8207be6a4a93e29fd16670656477131 10.60.44.105:7036 master - 0 1498201080733 9 connected. 5795-10922 135173619 dfc7bdbef98f5d5f07d6b1fcd6474dc 10.60.44.105: master 7035-0 1498201082755 3 connected. 11256-16383 1 baa876789f68ed2e273ab7cb1d4e954fa7fc60c 10.60.44.105: master 7034-0 1498201078714 11 connected. 333-5460Copy the code

Add a slave node

Add a new slave node (7038) to a primary node (7037) in the existing cluster. Enable the new slave node and add the new slave node to the cluster as follows:

1. Create files (folders and configuration files) required by the node.

[root@localhost /]# mkdir /usr/local/redis-cluster/7038
Copy the code

2. Copy and modify the configuration file

Replace the string 7036 in the configuration file with 7038

[root@localhost /]# cp /usr/local/redis-cluster/7036/redis.conf  /usr/local/redis-cluster/7038/
Copy the code

3. Start the newly created node

[root@localhost /]# /usr/local/redis/bin/redis-server /usr/local/redis-cluster/7038/redis.conf
Copy the code

4. Add the new node to the cluster

#Parameter 1: IP address and port of the new node to be added. Parameter 2: IP address and port number of any online node in the existing cluster
[root@localhost /]# redis-trib.rb add-node 10.60.44.105:7038 10.60.44.105:7034
Copy the code

If successful, the following information is displayed:

>>> Adding node 10.60.44.105:7038 to cluster 10.60.44.105:7034
>>> Performing Cluster Check (using node 10.60.44.105:7034). omit>>> Send CLUSTER MEET to node 10.60.44.105:7038 to make it join the cluster.
[OK] New node added correctly.
Copy the code

5. View information about the cluster node after the new node is added

9 cc99dc2e2721d6712c55e293c21c7f3b9c2f95c 10.60.44.105: master 7038-0, 1498201399078 connected 3 f2c74e5888907c494b8b728210175144657f218 10.60.44.105: master 7037-0 12 connected a scale of 0-332 to 1498201398374, 5461-5794 10923-11255...Copy the code

6. Set the node whose port is 7038 to the secondary node whose port is 7037

[root @ localhost /] # redis - cli 10.60.44.105 - c - h - p 7038 cluster 3 f2c74e5888907c494b8b728210175144657f218 replicateCopy the code

7. View information about the secondary cluster node

3 f2c74e5888907c494b8b728210175144657f218 0 9 cc99dc2e2721d6712c55e293c21c7f3b9c2f95c 10.60.44.105:7038 slave 1498201731111 12 connected 3 f2c74e5888907c494b8b728210175144657f218 10.60.44.105:7037 1498201398374 12 master - 0 connected 0-332 5461-5794 10923-11255 .......Copy the code

Delete a primary node

Delete the primary node whose port number is 7037 from the cluster and enter the IP port number of any node in the cluster and the ID of the node to be deleted. (You need to allocate slots to other master nodes first.)

[root @ localhost /] # redis/usr/local/bin/redis - trib. Rb del -node 10.60.44.105:7034 3f2c74e5888907c494b8b728210175144657f218Copy the code

Delete a slave node

Similarly, the winner node

5. Cluster testing

Simulating OOM (Out Of Montgomery)

redis-cli debug oom
Copy the code

Simulation of downtime

redis-cli debug segfault
Copy the code

Simulation of hang

redis-cli -p 6379 DEBUG sleep 30
Copy the code

References:

www.cnblogs.com/hjwublog/p/… Blog.csdn.net/cfl20121314… www.cnblogs.com/wuxl360/p/5… www.redis.cn/topics/clus… Wiki.jikexueyuan.com/project/all…