Hello, today I would like to share with you the interview questions of Redis. Please take out your notebook and write them down
RDB and AOF mechanisms
RDB:Redis DataBase
Write a snapshot of the data set in memory to the disk at a specified interval. Fork a sub-process to write the data set to a temporary file. After the data is written to a temporary file, the original file is replaced and stored in binary compression.
advantages
- The entire Redis database will contain only one file, dump. RDB, to facilitate persistence
- Good Dr, convenient backup
- To maximize performance,fork the sub-process to complete the write operation and let the main process continue to process the command, so it is IO maximization, use a separate sub-process for persistence, the main process does not do any IO operations, ensuring the high performance of Redis
disadvantages
- Data security is low. RDB is to persist data at an interval. If REDis fails during the persistence, data will be lost
- Because the RDB is forked to assist in data persistence, large data sets can cause the entire server to go out of service for hundreds of milliseconds, or even a second
AOP:Append Only File
Each write, delete, and query operation processed by the server is recorded in the form of logs. Instead, it is recorded in text mode. You can open the file to see detailed operation records
advantages
- In data security, Redis provides three synchronization strategies, namely synchronous every second, and each modified synchronous and asynchronous, synchronous and asynchronous completion of every second, and, in fact, its efficiency is very high, the difference is once the system outage occurs, then the modified data will be lost within one second, and each change synchronization, we can see it as a synchronous persistence, namely hair at a time All data changes are recorded to disk immediately
- If you write files in Append mode, the existing content will not be damaged even if the server breaks down during the process. You can use the Redis-check-aof tool to solve data consistency problems
- The rewrite mode of the AOF mechanism, which periodically rewrites AOF files for compression purposes
disadvantages
- AOF files are larger than RDB files, and the recovery speed is slow
- When the data set is large, it is less efficient than RDB startup
- The operation efficiency is not as high as RDB
AOF files are updated more frequently than RDB. AOF is more secure than RDB, and RDB performs better than AOF if both files are configured with AOF
Redis delete policy for expired keys
Redis is a key-value database. We can set the expiration time of cached keys in Redis. Redis’s expiration policy means that when a key cached in Redis expires, what does Redis do
Lazy expiration: Only when a key is accessed, the system determines whether the key has expired, and the expired key is cleared. This strategy maximizes CPU savings, but is very memory unfriendly. In extreme cases, a large number of expired keys may not be accessed again, and thus will not be cleared, occupying a large amount of memory
If the timer expires, delete it when the time is up
Periodic expiration: A compromise between scanning a set number of expires keys in the expires dictionary of a set number of databases at regular intervals and clearing expired keys. By adjusting the interval of periodic scan and the time limit of each scan, you can achieve the optimal balance of CPU and memory resources in different situations
(The Expires dictionary holds expiration data for all keys with an expiration date set, where key is a pointer to a key in the key space,value is the expiration date represented by the millisecond precision UNIX timestamp for that key, and the key space is all keys held in the Redis cluster.)
Redis uses both lazy expiration and periodic expiration strategies
Redis threading model, why is a single thread fast
Redis developed a network event handler based on Reactor model, which is called file Event Handler. This file event handler, it is single-threaded, so Redis is called single-threaded model, it uses IO multiplexing mechanism to listen to multiple sockets at the same time, according to the Socket event type to select the corresponding event handler to process the event. It can realize high-performance network communication model, and can interconnect with other internal single-threaded modules, ensuring the simplicity of Redis internal threading model
The structure of the file event handler consists of four parts: multiple sockets,IO multiplexer, file event dispatcher and event handler (command request handler, command reply handler, connection reply handler).
Multiple sockets may concurrently produce different operations, each corresponding to a different file event, but the IO multiplexer listens for multiple sockets, queues the Socket into a queue, and fetches one Socket at a time to an event dispatcher, which sends the Socket to the corresponding event handler
The IO multiplexer then sends the next Socket in the queue to the event dispatcher. The file event dispatcher selects the corresponding event handler based on the current events generated by each Socket
The core of pure memory operations is the non-blocking I/O multiplexing mechanism. Single-threading instead avoids the performance problems associated with multiple threads’ frequent context switching
Cache avalanche, cache penetration, cache breakdown
Cache avalanche
Cache avalanche is when the cache fails in a large area at the same time, so that all requests fall on the database, causing the database to collapse under a large amount of requests in a short period of time (cache restart).
The solution
The cache data expiration time is set randomly to prevent a large amount of data from expiring at the same time
Add a corresponding cache flag to each cached data, record whether the cache is invalid, and update the data cache if the cache flag fails
Cache preheating (reboot)
Mutex (check database to lock redis)
The cache to penetrate
Cache penetration is when data is not in the cache or in the database, causing all requests to fall on the database, causing the database to collapse under a large number of requests in a short period of time
The solution
Add verification at the interface layer, such as user authentication verification,id basic verification, ID <=0 direct interception
If the data cannot be cached or retrieved from the database, you can write the key-value pair to key-null. The cache validity period can be shorter, for example, 30 seconds. (If the value is too long, the cache cannot be used in normal cases.) This prevents the attacker from using the same ID for violent attacks
A Bloom filter is used to hash all possible data into a large enough bitmap. A non-existent data will be intercepted by the bitmap, thus avoiding the query pressure on the underlying storage system
Cache breakdown
Cache breakdown refers to that there is no data in the cache but there is data in the database (generally, the cache event expires). At this time, due to a large number of concurrent users, the data in the cache is not read at the same time, and they go to the database to fetch data at the same time, causing a sudden increase in database pressure, resulting in excessive pressure. Unlike cache avalanche, which refers to simultaneous searches for the same data, cache avalanche is when different data is out of date and a lot of data is not available and therefore the database is searched
The solution
Set hotspot data to never expire
Add a mutex
Briefly describe the redis transaction implementation
The watch key can be used before the transaction begins
The transaction start
The execution of MULTI commands marks the start of a transaction. The MULTI command does this by turning on the REDIS_MULTI flag in the flags attribute of the client state
The command team
When a client switches to the transactional state, the server performs different operations based on the commands sent by the client. If the client sends the command to MULTI, EXEC, DISCARD, WATCH, MULTI one of four command, the server immediately execute this command, otherwise will command in a transaction queue, and then returned to the client QUEUED to recover
If the client sends one of the EXEC,DISCARD,WATCH, or MULTI commands, the server executes the command immediately
If the client sends a command other than the four commands, the server does not execute the command immediately
First check that the command is formatted correctly. If not, the server will turn off the REDIS_MULTI flag in the flags attribute of the client state and return an error message to the client
If correct, place the command in a transaction queue and return QUEUED to the client
A transaction queue is a FIFO that holds the enqueued commands
Transaction execution
The client sends the EXEC command and the server executes the EXEC command logic.
If the flags attribute of the client state does not include the REDIS_MULTI flag, or contains the REDIS_DIRTY_CAS or REDIS_DIRTY_EXEC flag, the transaction is cancelled
Otherwise, the client is in transaction state (flags REDIS_MULTI), the server traverses the transaction queue of the client, executes all commands in the transaction queue, and returns all the results to the client
Redis does not support a transaction rollback mechanism, but it does check each transaction for incorrect commands
Redis transactions do not support checking for programmer logic errors, such as performing operations on HashMap types on String database keys
The WATCH command is an optimistic lock that provides check-and-set(CAS) behavior for Redis transactions. It can monitor one or more keys, and once one of them is modified (or deleted), subsequent transactions will not be executed until the EXEC command
The MULTI command is used to start a transaction, which always returns OK.MULTI After execution, the client can continue to send as many commands to the server as it wants. These commands are not executed immediately, but are placed in a queue, and when the EXEC command is invoked, all the commands in the queue will be executed
EXEC: Executes all commands in the transaction block, returns the return values of all commands in the transaction block, in the order in which the commands are executed, and returns null if the operation is interrupted
By calling DISCARD, the client can empty the transaction queue, abandon the transaction, and exit from the transaction state
Using the UNWATCH command, you can disable watch from monitoring all keys
Redis cluster solution
master-slave
The guard mode
Sentinel is a very important component in redis cluster, which has the following functions:
- Cluster monitoring: Monitors whether the Redis master and slave processes are working properly
- Message notification: If a Redis instance fails, the sentry is responsible for sending a message as an alarm notification to the administrator.
- Failover: If the master node fails, it is automatically transferred to the slave node
- Configuration center: Notifies the client client of the new master address if failover occurs
Sentinel is used to achieve high availability of redis cluster, itself is also distributed, as a sentinel cluster to run, work with each other
- During failover, determining whether a master node is down requires the agreement of most of the sentinels, involving distributed elections
- Even if some sentinels fail, the sentinels cluster still works
- Sentinels typically need three instances to maintain their robustness
- Sentinel + Redis master-slave deployment architecture does not guarantee zero data loss, only high availability of redis cluster
- For sentry + Redis master-slave complex deployment architectures, try to do adequate testing and practice in both test and production environments.
Redis Cluster is a server Sharding technology officially provided in version 3.0. It adopts the concept of Slot(Slot), which is divided into 16384 slots. It sends requests to any node
Solutions that
Data is divided into hash segments. Each node stores data in a certain range of hash slots (hash values). By default, 16,384 slots are allocated
Each data fragment is stored on multiple nodes that are master and slave to each other
Data is written to the primary node and then synchronized to the secondary node (blocking synchronization can be configured).
Data of multiple nodes in the same fragment does not maintain strong consistency
When reading data,redis returns a turn instruction pointing to the correct node if the key that the client operates on is not assigned to that node
During capacity expansion, some data on the existing node must be migrated to the new node
In the Redis cluster architecture, each Redis should release two port numbers, such as 6379, and add 10000 port numbers, such as 16379
Port 16379 is used for communication between nodes, namely for cluster bus communication, for fault detection, configuration updates, and failover authorization. Cluster Bus uses another binary protocol, Gossip protocol, for efficient data exchange between nodes, which consumes less network bandwidth and processing time
advantages
- Without a central architecture, it supports dynamic capacity expansion and is transparent to services
- With Sentinel monitoring and automatic Failover capabilities
- The client does not need to connect to all nodes in the cluster, but only one available node in the cluster
- High performance, client directly connected to Redis service, face to take the loss of proxy
disadvantages
- Operation and maintenance are complex, and database migration requires manual intervention
- Only database 0 can be used
- Batch operations are not supported
- Distributed logic and storage module coupling, etc
Redis Sharding is a multi-redis instance Cluster method used by the industry before Redis Cluster came out. The main idea is to use hash algorithm to hash the Redis data B9key, through the number, the specific key will be mapped to the specific Redis node. The Javaredis client drives Jedis, which supports the ßRedisSharding function ShardedJedis and ShardedJedisPooI combined with the cache pool
advantages
The advantage is that it is very simple. The Redis instances on the server side are independent and unrelated to each other. Each Redis instance runs like a single server, which is very easy
Linear expansion, system flexibility is very strong
disadvantages
As Sharding processing is placed on the client side, operation and maintenance will be challenged when the scale is further expanded.
Client Sharding does not support dynamic addition and deletion of nodes. When the service Redis instance group topology changes, each client needs to be updated and adjusted. Thought connection cannot be shared, and when the application scale increases, resource waste restricts optimization
The core principles of Redis master-slave replication
