Principle:

Redis cluster uses a consistent hash slot to allocate a certain hash slot to each primary node in the cluster, and hashes the written data to a primary node for storage. The cluster uses formula (CRC16 key) & 16384 to calculate the key key data slot. 16,384 slots are evenly distributed on each node. Each primary node in a cluster maintains a portion of slots where data is stored. Each primary node has at least one secondary node for high availability.Copy the code

Node communication mode: Enable one port. Port number +10000 is used for communication between nodes in clusters.

By default, each node selects 5 random nodes to send ping messages 10 times per second to transmit its own information and known cluster information. After receiving the ping message, pong message is returned to reply. Finally, through this random message exchange, each node will finally obtain all information.

When a master node fails, all nodes will notice that the master node has failed, and the slave node, acting as the master node, will take over the work of the master node and tell all other nodes that he is the master. The other surviving nodes will then update the information tables they maintain and the slave node will take over. If they all fail, the cluster will report an error. When operated from a node, the consistency hash calculation will be stored in one of the master nodes, and the slave node will synchronize the master data.

Redis Cluster is decentralized, each node in the cluster is equal, each node stores its own data and the status of the whole cluster. Each node is connected to all other nodes, and these connections remain active.

When building a cluster, there will be a master node for each shard, corresponding to a slave node. Slaveof function is implemented. At the same time, when the primary node is down, automatic failover of Sentinel is implemented

Redis Distributed Cluster (Deployment) :

Port number: 7000-7005

This distributed shard cluster can be installed on a LInux system. You only need to install multiple instances as the cluster configuration. Yum -y install Ruby Rubygems

Yum installs version 2.0.0, but gem requires version 2.2.2 or higher, so select compile, download and install ruby environment:

Wget https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.1.tar.gz tar xf ruby - 2.6.1. Tar. Gz && CD ruby - 2.6.1 /. / configure --prefix=/usr/local/ruby make && make install && echo $? echo 'export PATH=$PATH:/usr/local/ruby/bin' >> /etc/profile source /etc/profileCopy the code

Modify gem tool

# see gem tool source address gem sources - l # add a ali cloud gem tool source gem sources - a http://mirrors.aliyun.com/rubygems/ # delete gem tools abroad source gem by default Sources --remove https://rubygems.org/ # Download gem install redis -v 4.1.0Copy the code

Cluster node preparation: mkdir /data/700{0.. 5}

Example of configuring port 7000:

vim /data/7000/redis.conf
port 7000
daemonize yes
pidfile /data/7000/redis.pid
loglevel notice
logfile "/data/7000/redis.log"
dbfilename dump.rdb
dir /data/7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
Copy the code

Copy other port instances:

Conf file cp /data/7000/redis.conf /data/7001/ cp /data/7000/redis.conf /data/7002/ cp /data/7000/redis.conf /data/7003/ Cp /data/7000/redis.conf /data/7004/ cp /data/7000/redis.conf /data/7005/ /data/7001/redis.conf sed -i 's#7000#7002#g' /data/7002/redis.conf sed -i 's#7000#7003#g' /data/7003/redis.conf sed -i 's#7000#7004#g' /data/7004/redis.conf sed -i 's#7000#7005#g' /data/7005/redis.confCopy the code

Start all instances:

redis-server /data/7000/redis.conf
redis-server /data/7001/redis.conf
redis-server /data/7002/redis.conf
redis-server /data/7003/redis.conf
redis-server /data/7004/redis.conf
redis-server /data/7005/redis.conf
Copy the code

Create command soft link: (this command expired, now use redis-cli command) (optional execute command)

Ln -s/usr/local/redis - 5.0.2 / SRC/redis - trib. Rb/usr/sbin /Copy the code

View the process:

ps -ef |grep redis-server

Add all instance nodes to cluster management:

Redis -cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005Copy the code

The distributed master-slave rule is that the first three instance nodes are the master, and the next three are the slave nodes. If Replicas is 2, it adds three more instance nodes

View the status of the primary node:

redis-cli -p 7000 cluster nodes|grep master
Copy the code

View the status of the secondary node:

redis-cli -p 7000 cluster nodes|grep slave
Copy the code

Redis- Distributed Clustering (Administration)

Preparations for adding cluster nodes:

mkdir /data/700{6.. 7} Copy other port instances: Cp /data/7000/redis.conf /data/7006/ cp /data/7000/redis.conf /data/7007/ /data/7006/redis.conf sed -i 's#7000#7007#g' /data/7007/redis.confCopy the code

Start a new node instance:

redis-server /data/7006/redis.conf
redis-server /data/7007/redis.conf
Copy the code

View the process:

ps -ef |grep redis-server

Add the primary node :(the 7000 instance is the management node)

Redis -cli --cluster add-node 127.0.0.1:7006 127.0.0.1:7000Copy the code

View the status of the primary node:

redis-cli -p 7000 cluster nodes|grep master
Copy the code

Transfer slot (resharding) :

#' Operation cluster management node is reassigned, Redis-cli --cluster reshard 127.0.0.1:7000 How Many slots do you want to move (from 1 to) On the INTERFACE, specify the fragment size and select the node ID for receiving fragments 16384)? Calculate the total size of the shard divided by the master node manually. # 2129 d28f0a86fc89571e49a59a0739812cff7953 choose to receive data fragmentation of node ID, (this is the 7006 instances of the new node ID) Source node # 1: Proceed with the proposed reshard plan (yes/no) Yes # Confirm the modification of the above actionCopy the code

Re-view the master node status :(you can see the re-sharding of the cluster data)

redis-cli -p 7000 cluster nodes|grep master
Copy the code

Adding slave nodes:

Add 7007 to primary 7006 Redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7000 --cluster-slave --cluster-master-id specifies the management node where the primary 7006 instance resides 2129d28f0a86fc89571e49a59a0739812cff7953Copy the code

View the status of the secondary node:

redis-cli -p 7000 cluster nodes|grep slave
Copy the code

Preparations for deleting cluster nodes:

Move the shard of the node to be deleted:

Redis -cli --cluster reshard 127.0.0.1:7000 reshard 127.0.0.1:7000 reshard 127.0.0.1:7000 How many slots do you want to move (from 1 to 16384)? What is the receiving node ID? E64f9074a3733fff7baa9a4848190e56831d5447 # choose to receive data fragmentation of node ID, (this is the 7006 instances of the new node ID) Source node # 1: D28f0a86fc89571e49a59a0739812cff7953 # 2129 to choose from which Source the master node to shard to the new master node (this is to delete the master node ID number) of the Source node # 2: Do you want to proceed with the proposed reshard plan (yes)? Yes # Confirm the modification of the above actionCopy the code

Re-view the master node status :(you can see the re-sharding of the cluster data)

redis-cli -p 7000 cluster nodes|grep master
Copy the code

Continue moving the data slice:

Redis -cli --cluster reshard 127.0.0.1:7000 reshard 127.0.0.1:7000 reshard 127.0.0.1:7000 How many slots do you want to move (from 1 to 16384)? 1366 # What is the receiving node ID? F6c1aaea3a8c56e0c7dee8ad7ae17e26dd04244c # choose to receive data fragmentation of node ID, (this is the 7006 instances of the new node ID) Source node # 1: D28f0a86fc89571e49a59a0739812cff7953 # 2129 to choose from which Source the master node to shard to the new master node (this is to delete the master node ID number) of the Source node # 2: Do you want to proceed with the proposed reshard plan (yes)? Yes # Confirm the modification of the above actionCopy the code

Re-view the master node status :(you can see the re-sharding of the cluster data)

redis-cli -p 7000 cluster nodes|grep master
Copy the code

Last move of data slice:

Redis -cli --cluster reshard 127.0.0.1:7000 reshard 127.0.0.1:7000 reshard 127.0.0.1:7000 How many slots do you want to move (from 1 to 16384)? What is the receiving node ID? 5a0df4ea0af5f35c1248e45e88d44c3f2e10169f Source node #1: 2129d28f0a86fc89571e49a59a0739812cff7953 Source node #2: doneCopy the code

Re-view the master node status :(you can see the re-sharding of the cluster data)

redis-cli -p 7000 cluster nodes|grep master
Copy the code

Delete the master node of the data slice:

# 'delete empty data redis - 7006 instances of cli - cluster del -node 127.0.0.1:7006 # 2129 d28f0a86fc89571e49a59a0739812cff7953 delete no 7007 examples of the main library Redis - cli - cluster del -node 127.0.0.1:7007 821 bcf78c5e4c0b08aa7a5d514214b657ebec4abCopy the code

Other configuration management:

Redis -cli -p 6382 -a Redhat info memory redis-cli -p 6382 -a redhat config set maxmemory 102400000Copy the code