This is the sixth day of my participation in the First Challenge 2022. For details: First Challenge 2022.
preface
The principle of Sentry mode is analyzed above. It is a high availability solution provided by Redis, which makes up for the defect that the cluster mode cannot automatically fail over. However, in a high concurrency system, there will still be a bottleneck of single redis server, which will bring great pressure to Redis. High performance solution Cluster. Redis Cluster provides redis data sharding and scale-out capabilities, reducing the pressure on a single master.
A, think about it, if you need to do redis sharding storage, where can you start?
1. First of all, think of the client. The client performs sharding calculation according to the operation key and routes the request to the destination REDis server
2. The second option is to develop a proxy middleware, similar to MyCAT, where the client connects to the middleware and the middleware completes the logic of sharding and routing.
3, another is based on redis server development, the official is doing so.
What are the problems that sharding needs to solve?
1. The data source needs to be dynamically selected and the client request needs to be routed to which Redis server
2. How to deal with the new, deleted and data migration of Redis nodes?
Common fragment routing algorithms include the consistent hash algorithm
This algorithm has a ring data structure, maps the server nodes to the ring structure, then uses a hash algorithm to compute the subscript of the server node, and searches clockwise for the first server node, at which point the shard is selected. The advantage of this scheme is that the data distribution is stable in the event of server changes. Because there are fewer server nodes, some are often virtualized.
In the Redis Cluster, Redis uses neither modular nor consistent hash. It’s a virtual slot. Create 16384 slots, each Redis node is responsible for an interval segment.
Master Stores slot relationships based on the bitmap. 0 indicates that slots do not belong to this node, and 1 indicates that slots belong to the current node.
Why is RedisCluster designed with 16384 slots?
1. If the slot number is 65536 and the number of heartbeat message headers reaches 8 KB, the number of heartbeat packets sent is too large.
In the header, the most space is myslots[Cluster_slots/8]. When the slot number is 65536, the size of this block is: 65536÷8÷1024= 8KB Because every second, the Redis node needs to send a certain number of ping messages as heartbeat packets. If the slot number is 65536, the ping message header is too large, which wastes bandwidth.
2. The number of redis cluster master nodes is almost impossible to exceed 1000.
The more nodes in the cluster, the more data the heartbeat packets carry. If the number of nodes exceeds 1,000, network congestion will also occur. Therefore, the authors of Redis do not recommend that the number of redis cluster nodes exceed 1000. Therefore, for a Redis cluster with less than 1000 nodes, 16,384 slots are sufficient. There is no need to expand to 65536.
3. The smaller the number of slots and nodes, the higher the compression rate
The configuration information of the Redis master node is saved in the form of a bitmap. During transmission, the bitmap is compressed. However, if the bitmap filling rate of slots/N is high (N indicates the number of nodes), the bitmap compression rate is very low. If the number of nodes is small and the number of hash slots is large, the bitmap compression rate is low.
Client slot routing rules
1. In redission, slots are calculated for each key operation on the client.
if (key.hasArray()) { end = CRC16.crc16(key.array(), key.position(), key.limit() - key.position()) % 16384; return end; } // Assign slot end = crc16. CRC16 (key) % 16384;Copy the code2. Client initialization and scheduled tasks pull slot information from the master and cache it locally.
This article documents the fundamentals of Redis Cluster, but there are many details that need to be further refined.