SpringBoot integration with Redis standalone

  1. Introduce redis dependencies
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
Copy the code
  1. Configure redis
Spring: redis: database: 1 host: 192.168.1.191 Port: 6379 Password: 12345Copy the code

Master/Slave Replication Architecture (Master/Slave)

  1. With persistence, Redis guarantees that data will not be lost (or lost in small amounts) even if the server is restarted, but because the data is stored on a single server, if that server fails, such as a bad hard drive, data will be lost.
  2. So to avoid a single point of failure, we need to replicate data to multiple different servers so that even if one server fails, the other servers can continue to provide service.
  3. This requires that when the data on one server is updated, the updated data is automatically synchronized to other servers.

How to do that? Redis has one master, many slave architecture

  1. We can deploy multiple Redis and specify the master-slave relationship among these redis in the configuration file. The master is responsible for writing data and asynchronously copying the written data to the slave machine. This mode is called master-slave replication, namely master/slave. Writing data to slave causes errors.
  2. To implement master/slave replication of Redis, you only need to modify the master Redis configuration file redis.conf
  3. The master configuration does not need to be changed
  4. The salve must be configured as follows:
 replicaof <masterip> <masterport>
 masterauth <master-password>
 replica-serve-stale-data yes
 replica-read-only yes
Copy the code

Primary/secondary replication: full replication

  1. Full Redis replication generally occurs during the Slave initialization phase. In this case, the Slave needs to copy all data on the Master. Specific steps are as follows:

  1. After completing the above steps, all the slave server data is initialized and the Savle server can now receive read requests from users.
  2. Master/Slave The master/slave synchronization process is asynchronous. It means that after the master executes the command requested by the client, it immediately returns the result to the client and asynchronously synchronizes the command to the slave.
  3. This feature ensures that master performance is not affected after master/slave is enabled.
  4. On the other hand, if the master/slave is disconnected due to network problems during this inconsistent window, the master will not be able to know how many slave databases a command was finally synchronized to. However, Redis provides a configuration item that restricts the master to be writable only if the data is synchronized to at least how many slaves:
    • Min-replicas-write 3: indicates that the master is writable only when three or more slaves are connected to the master.
    • Min-replicas-max-lag 10: specifies the longest time that the slave is allowed to be disconnected. If no response is received within 10 seconds, the master considers that the slave is disconnected.

Primary/secondary replication: incremental replication

  1. Since Redis 2.8, breakpoint continuation of master/slave replication is supported. If the network connection is down during the master/slave replication, the replication can continue where the last replication was made, rather than starting from scratch.
  2. Master nodes create a backlog in memory. Both master and slave nodes save a replica offset and a master ID. Offset is stored in the backlog. If the connection between the master and slave is disconnected, the slave tells the master to continue copying from the replica offset. If no offset is found, a full synchronization is performed.

Primary/secondary replication: No hard disk replication

  1. Redis master-slave replication is implemented based on RDB persistence, that is, the master saves the RDB snapshot in the background, and the slave receives and loads the RDB file, but there are some problems with this method:
  2. When the master disables the RDB file, Redis still generates a snapshot of the RDB file if the master performs a replication initialization operation. When the master recovers the RDB file on its next startup, data problems may occur.
  3. In 2.8.18 or later, Redis introduced the no hard disk replication option, which can send data directly without using RDB file to synchronize. This function can be enabled by the following configuration:
    • repl-diskless-sync yes
  4. The master creates the RDB in memory and sends it to the slave. The RDB is not saved on its own disk.

Master/slave replication summary

  1. The master/slave replication mode consists of one master and multiple slaves. You can configure the master/slave relationship in the redis.conf configuration file.
  2. When the slave redis (slave) breaks down, the read request processing performance deteriorates.
  3. When the primary Redis (master) is down, write requests cannot be executed.
  4. If the master is faulty, run the slaveof no one command to elevate one slave to master, and run the slaveof command to point to the new master to form a new master/slave relationship.
  5. Failover in master-slave replication mode requires manual operation, which is not intelligent. To automate the process, Sentinel Sentinel is required to implement automatic failover.

Highly available Sentinel Sentinel

  1. Sentinel Sentinel is a high availability solution provided by Redis. Sentinel Sentinel can be used to monitor the operation of multiple Instances of Redis service.

Basic principles of sentry:

  1. Sentinel is used to monitor the Master and Slave servers of Redis. It constantly checks whether the Master and Slave servers are working properly.
  2. If Sentinel fails, it cannot be monitored, so you need multiple sentinels to form a Sentinel network. Each Sentinel monitoring the same Master will communicate with each other to form a distributed Sentinel network, exchanging information about the monitored Redis server.
  3. When a Sentinel believes that a monitored Redis server is faulty, it checks with other Sentinels in the network to determine whether the server is indeed faulty. If the faulty Redis server is the primary server, The Sentinel network will automatically failover the failed primary Redis server, bringing the whole master-slave mode back to normal by promoting one of the slave servers under the failed primary Redis server to the new master server and transferring the others to the new master server.
  4. When the failed old primary server is brought online again, Sentinel makes it become a slave Redis server and hangs under the new primary Redis server.
  5. So sentinel is automatic failover, without manual intervention, is a highly available cluster solution;

How is Sentinel Sentinel implemented

  1. Configuration sentinel. Conf:
port 26379
pidfile "/usr/local/redis/sentinel/redis-sentinel.pid"
dir "/usr/local/redis/sentinel"
daemonize yes
protected-mode no
logfile "/usr/local/redis/sentinel/redis-sentinel.log"
Copy the code
  1. The core configuration
# configure sentrySentinel Monitor MyMaster 127.0.0.1 6379 2# your password
sentinel auth-pass <master-name> <password>
# Master is the interval between failures identified by Sentinel
sentinel down-after-milliseconds mymaster 30000
# number of parallel slaves to synchronize with new Master
sentinel parallel-syncs mymaster 1
If the sentry fails to perform failover, the sentry is also a process. If the sentry fails to perform failover, it will be handled by other sentries
sentinel failover-timeout mymaster 180000
Copy the code
  1. Start the sentry
    • redis-sentinel sentinel.conf

Analysis of the internal principle of sentinel mode

  1. ? In a Redis system with one master and many slaves, multiple sentinels can be used for monitoring tasks to ensure that the system is stable enough.
  2. Sentinels not only monitor the master and slave, but also monitor each other. This mode is called sentry cluster. The sentry cluster needs to solve the problem of fault discovery and negotiation mechanism with the master.

Sentinel Sentinel model summary

  1. In the master/slave replication cluster mode, multiple slave servers can be used to share read requests. If the slave server fails, the read request capability deteriorates. However, if the master server fails, the write request cannot be processed.
  2. The Sentinel mode automatically performs failover when the master master goes offline, promoting one slave master and allowing other slaves to be attached to the new master master.

Springboot integrates sentinel mode

spring:
  redis:
    database: 0
    password: 12345
    sentinel:
      master: mymaster # # master name
      ## Sentry node IP and port is good, Sentry will host master/slave architectureNodes: 192.168.225.129:26379192168 225.132:26379192168 225.133:26379Copy the code

Redis Cluster Cluster

  1. Sentinel mode can achieve the purpose of high availability of REDis, but at this time, each REDis stores all the data in the cluster, resulting in the total data storage capacity of the cluster is limited to the node with the smallest available storage memory, forming the barrel effect.
  2. Sentinels and clusters are two independent functions. Sentinels are sufficient when data is not sharded. Redis Cluster is a good way to scale the data horizontally.
  3. Each node in the Redis cluster has two roles: master node and slave node. The master node stores data and the slave node backs up data of the master node.
  4. A Redis Cluster consists of multiple Redis nodes, and there is no intersection of Redis data of different node groups, that is, a fragment of data corresponding to each node group.
  5. Some nodes in a node group are active and standby, corresponding to master and slave nodes. Data of the two nodes is consistent in quasi-real time, which is guaranteed by the asynchronous master/slave replication mechanism.
  6. A node group has only one master node and can have 0 (none) to multiple slave nodes. In this node group, only the master node provides some services to users.

Redis Cluster Data partition, hash slot slot

  1. The Redis Cluster proposes the concept of hash slots. The Redis Cluster has 16384 hash slots. Each node in the cluster uses CRC16 to determine which slot to place a key in
  2. Slot number = crc16(key) % 16384, which is a hash function.
  3. For example, the current cluster has 3 nodes, then:
    • Node A contains hash slots 0 to 5500;
    • Node B contains hash slots 5501 to 11000;
    • Node C contains hash slots 11001 to 16383;

This structure makes it easy to add or remove nodes (horizontal scaling)

  1. If I want to add A new node D, I need some slots from nodes A, B, and C to D;
  2. If I want to remove node A, I need to move the slots in NODE A to nodes B and C, and then remove node A without any slots from the cluster.
  3. Can you dynamically expand online?