1. Redis3.0
advantages
A. There is no central node
B. Data is stored on multiple Redis instances by slot
C. Smoothly expand or shrink nodes
D. Automatic failover (Status information is exchanged between nodes through the Gossip protocol and the voting mechanism is implemented to promote the role from Slave to Master)
E. Reduces o&M costs and improves system scalability and high availability
disadvantages
A. Heavy reliance on external Redis-trib
B. Lack of monitoring management
C. Need to rely on Smart Client(connection maintenance, cached routing table, MultiOp and Pipeline support)
D. The Failover node is not as timely as ZooKeeper on the central node
E. the overhead of the Gossip message
F. Cold and hot data cannot be statistically distinguished
G. Slave Indicates cold standby, which cannot relieve the read pressure
2.Proxy +Redis Cluster
advantages
The Smart Client:
A. Compared with using proxy, it reduces the consumption of one-layer network transmission and has higher efficiency;
B. Independent of third-party middleware, the implementation method and code can be controlled by themselves and can be adjusted at any time.
Proxy:
A. Provide a SET of HTTP Restful interfaces to isolate underlying storage. Fully transparent to the client and called across languages.
B. It is easy to upgrade and maintain the Redis Cluster. You only need to smooth the upgrade Proxy.
C. Hierarchical storage. The underlying storage is hot and cold heterogeneous storage.
D. Permission control. Proxy can control the whitelist through the secret key and filter out some illegal requests. In addition, it can also control and filter the large Value requested by users.
E. Security, can block some dangerous commands, such as Keys, Save, Flush All, etc.
F. Capacity control: Limits the capacity based on users’ capacity applications.
G. Resource logical isolation: Prefixes the keys of different users to isolate resources.
H. Monitor buried points for different interfaces.
disadvantages
The Smart Client:
A. Immature clients affect application stability and make development difficult.
B. MultiOp and Pipeline support is limited.
C. Connection maintenance: The Smart client maintains the Socket connected to each node in the cluster.
Proxy:
A. The proxy layer has one more forwarding, resulting in performance loss.
B. Operation and maintenance (O&M) requirements are high for capacity expansion or reduction, and smooth capacity expansion or reduction is difficult to achieve
3. Technical selection
The official Redis document reads as follows:
The redis-cli cluster support is very basic so it alwaysuses the fact that Redis Cluster nodes are able to redirect a client to theright node. A serious client is able to do better than that, and cache the map betweenhash slots and nodes addresses, to directly use the right connection to theright node. The map is refreshed only when something changed in the clusterconfiguration, for example after a failover or after the system administratorchanged the cluster layout by adding or removing nodes.
The redis cluster client relies on redis node redirection to find the redis instance in the cluster. A more sophisticated client is required to implement consistent hash, failover, and cluster management functions. Therefore, it is not a wise choice to use the official Redis Cluster client. This article provides three schemes for your reference. If there is any unreasonable place, you are welcome to discuss with me.
Scenario 1 Development using Nginx (OpenResty approach)
Here’s why:
A. Single-master multi-work mode. Like Redis, each Work is single-process and single-thread mode, and is based on Epoll event-driven mode.
B. Nginx uses an asynchronous non-blocking way to handle requests, an efficient asynchronous framework.
C. It has its own memory pool management mode because it consumes less memory. Clustering a large number of small memory applications together can be faster than Malloc. Reduce memory fragmentation and prevent memory leaks. Reduce memory management complexity.
D. To improve Nginx access speed, Nginx uses its own connection pool.
E. The most important thing is to support custom module development.
F. Within the industry, Redis has two great reputations for Nginx. The performance is very good.
Scheme 2 coDIS (Agent-based Redis cluster scheme adopted by Wandoujia)
Refer to coDIS official documentation github.com/CodisLabs/….
Codis is a full set of caching solutions including high availability, data sharding, monitoring, dynamic scaling, etc.
Go to Apps-> agent -> RedisCluster, after a certain scale, basically adopt this way.
Scheme 3: Independently develop redis intelligent client
It mainly implements redis slots management, failover, and consistent hash functions.
4. Redis 3.0 cluster
Redis 3.0 cluster adopts P2P model and is completely decentralized. Redis divides all keys into 16,384 slots, and each Redis instance is responsible for some of the slots. All information (nodes, ports, and slots) in a cluster is updated periodically through data exchange between nodes.
A Redis client makes a request at any Redis instance, and if the desired data is not in the instance, the client is directed to the desired instance by a redirection command.
The workflow of the Redis 3.0 cluster is shown below.
The machines in the Redis cluster exchange data periodically, and the workflow is as follows:
(1) The Redis client accesses some data on the Redis2 instance;
(2) within the Redis2 found this instance, the data is in Redis3 to Redis client sends a redirect orders;
(3) After receiving the redirection command, the Redis client accesses the Redis3 instance to obtain the required data.
Redis 3.0 has two problems with clustering:
-
A Redis instance has “data store” and “route redirection”, completely decentralized design. The advantage of this is that it is very simple to deploy Redis instead of having as many components and dependencies as Codis. However, the problem is that it is difficult to make a painless upgrade to the business. If there is a serious Bug in the Redis cluster, the entire Redis cluster can only be rolled back.
-
Major changes have been made to the protocol and the corresponding Redis client needs to be upgraded. Who can make sure there are no bugs after updating the Redis client? It is also a hassle to upgrade the Redis client in the code for online businesses that are already running on a large scale.
5. Redis4.0
(1) All Redis nodes are interconnected (PING PONG mechanism), and binary protocol is used internally to optimize transmission speed and bandwidth;
(2) The node Fail takes effect only when more than half of the nodes in the cluster are detected to Fail;
(3) The client is directly connected to the Redis node without the intermediate proxy layer. The client does not need to connect to all nodes in the cluster, but to any available node in the cluster.
(4) Redis-cluster maps all physical nodes to [0-16383] slots. The cluster maintains node<->slot<->value.
