Overview of the three clustering approaches to Redis

1. Master/slave replication

The principle of

  1. The secondary server connects to the primary server and sends the SYNC command.
  2. When the master server receives the SYNC name, it starts executing the BGSAVE command to generate an RDB file and uses a buffer to record all write commands executed thereafter.
  3. After the primary BGSAVE completes, it sends snapshot files to all secondary servers and continues to record write commands executed during sending.
  4. After receiving the snapshot file from the server, discard all old data and load the received snapshot.
  5. After the primary server snapshot is sent, the write command in the buffer is sent to the secondary server.
  6. The slave server finishes loading the snapshot, starts receiving command requests, and executes write commands from the master server buffer. (Initialization from the server is complete)
  7. The master sends the same write command to the slave server for each write command executed, and the slave server receives and executes the received write command (after the slave server is initialized).

advantages

  1. Supports master/slave replication. The master machine automatically synchronizes data to the slave machine, enabling read/write separation
  2. In order to share the Master’s read load, the Slave server can provide the client with the read-only service, while the Master must still perform the write service
  3. Slave can also accept connection and synchronization requests of other Slaves so as to effectively subload the synchronization pressure of Master.
  4. Master Server serves Slaves in a non-blocking manner. Therefore, during master-slave synchronization, clients can still submit query or modify requests.
  5. The Slave Server also performs data synchronization in a non-blocking manner. During synchronization, if a client submits a query request, Redis returns the data before synchronization

disadvantages

  1. Redis does not have automatic fault tolerance and recovery functions. The breakdown of the slave host will lead to the failure of some front-end read and write requests, which can be recovered only after the machine restarts or manually changing the front-end IP address.
  2. When the host is down, some data cannot be synchronized to the secondary host in a timely manner. After the IP address is switched, data inconsistency may occur, which reduces system availability.
  3. Redis is difficult to support online capacity expansion, which can be complicated when the cluster capacity reaches its maximum.

2. Sentinel mode

The principle of

When the primary server is out of service, a secondary server can be upgraded to the primary so that service can continue, but this process requires manual operation. To this end, Redis 2.8 provides the Sentinel tool for automated system monitoring and recovery.

The sentry’s role is to monitor the health of the Redis system. Its functions include the following two.

(1) Monitor the normal operation of the primary and secondary servers.

(2) When the primary server fails, the secondary server will be automatically converted to the primary server.

Way to work

  • Each Sentinel process sends a PING command to the Master, Slave, and other Sentinel processes in the cluster at a frequency of once per second.
  • The Sentinel process marks an instance as SDOWN if it has taken longer than the value specified in the down-after-milliseconds option for the last effective PING response.
  • If a Master server is marked as a subjective offline (SDOWN), all Sentinel processes that are monitoring the Master server confirm once per second that the Master server is indeed in a subjective offline state
  • The Master server is marked as ODOWN when a sufficient number of Sentinel processes (greater than or equal to the value specified in the configuration file) confirm that the Master server is in a subjective SDOWN state within the specified time period.
  • In general, each Sentinel process sends INFO commands to all Master and Slave servers in the cluster every 10 seconds.
  • When the Master server is marked as ODOWN by the Sentinel process, the Sentinel process sends INFO commands to all slaves of the offline Master server once every 10 seconds instead of once every second.
  • The objective offline status of the Master Master server is removed if a sufficient number of Sentinel processes do not agree to take the Master offline. If the Master server reping the Sentinel process and returns a valid response, the subjective offline status of the Master server will be removed.

advantages

  • The Sentinel mode is based on the master-slave mode, which has all the advantages of the master-slave mode.
  • The master and slave can be switched automatically, making the system more robust and more available.

disadvantages

Redis is difficult to support online capacity expansion, which can be complicated when the cluster capacity reaches its maximum.

3. Redis-cluster

The principle of

Redis sentinel mode can basically achieve high availability, read and write separation, but in this mode, each Redis server stores the same data, it is a waste of memory, so in Redis3.0 added cluster mode, to achieve redis distributed storage, This means that each Redis node stores different content.

Redis-cluster adopts a centrless structure and has the following characteristics

  1. All redis nodes are ping-pong with each other and use binary protocols internally to optimize transmission speed and bandwidth.
  2. Nodes fail only when more than half of the nodes in the cluster have detected failure.
  3. The client is directly connected to the Redis node without the need for an intermediate proxy layer. The client does not need to connect to all nodes in the cluster, but to any available node in the cluster.

Way to work

Each redis node has two items. One is a slot. Its value ranges from 0 to 16383. Another is cluster, which can be understood as a cluster management plug-in. When our access key arrives, Redis will get a result according to crC16 algorithm, and then take the remainder of the result to 16384, so that each key will correspond to a hash slot with the number between 0 and 16383. Through this value, we can find the node corresponding to the corresponding slot. Then jump directly to the corresponding node for access operation.

To ensure high availability, a redis-cluster cluster adopts the master-slave mode. One master node corresponds to one or more slave nodes. When the master node goes down, the slave node is enabled. When other primary nodes ping A primary node A, if more than half of the primary nodes time out, primary node A is considered to be down. If both primary node A and its secondary node A1 go down, the cluster can no longer be serviced

 

This article will take you through the essence – Redis three clustering methods + penetration and avalanche prevention and resolution

Prevention and resolution of avalanche and penetration in REDis

Understanding cache penetration

Cache penetration refers to the query of data that does not exist. If the cache does not hit, the database must be queried. If no data is found, the data will not be written into the cache.

The solution

  • All possible query parameters are stored in hash format and verified at the control layer. If they do not match, they are discarded. The most common is to use a Bloom filter, hash all possible data into a large enough bitmap, a non-existent data will be intercepted by the bitmap, thus avoiding the query pressure on the underlying storage system.
  • If a query returns empty data (whether data is nonexistent or a system failure), we can still cache the empty result, but it will expire for a short time, no more than five minutes.

Get to know cache avalanche

If the cache set is invalidated for a period of time, a large number of cache penetrations occur and all queries fall on the database, causing a cache avalanche.

There is no perfect solution, but you can analyze user behavior and try to distribute failure points evenly. Most system designers consider locking or queuing cached single-threaded (process) writes to avoid large numbers of concurrent requests falling on the underlying storage system in the event of a failure.

The solution

  • After cache invalidation, the number of threads that read the database write cache is controlled by locking or queuing. For example, only one thread is allowed to query data and write to the cache for a key, while the other threads wait.
  • You can use the reload mechanism to update the cache in advance, and then manually trigger the cache to be loaded just before large concurrent accesses occur
  • Set different expiration times for different keys, so that the cache expiration time is as uniform as possible
  • Do level 2 caching, or double caching. A1 is the original cache and A2 is the copy cache. If A1 fails, access A2. Set the expiration time of A1 cache to short-term and A2 to long-term.

The above is what I have sorted out about Redis clustering and caching, welcome your criticism.

Programmers to find a way out or as far as possible in advance of career planning and preparation, do not say what: “step, calculate step” words. In this era of competition when you open your eyes, you can relax, but others will move on and not wait for you.