Trail source Java notes – Redis server

preface

The body of the

Redis

Cache invalidation policy

There are three main algorithms:

• FIFO:First In First OutFirst in, first out. Determine how long the data has been stored, and the data farthest away from the present is eliminated first.
• LRU:Least Recently Used, least recently used. Determine the most recently used data, and the most recent data is eliminated first. (Distance in time)
• LFU:Least Frequently UsedAnd used least often. The data that has been used the least over a period of time will be eliminated first. (How many times)

Redis offers six data elimination strategies:

• Volatile – LRU: Selects the least recently used data from a set with expiration time
• Volatile – TTL: Selects data to be obsolete from a set with an expiration time
• Volatile -random: Randomly selects data from a set whose expiration time has been set
• Allkeys-lru: Cull the least recently used data from the dataset
• Allkeys-random: Randomly select data from the dataset for elimination
• No-enviction: disables data expulsion by default

There are three expiration strategies:

• Time to delete: in the SettingskeyThe expiration time of thekeyCreate a timer and set the timer tokeyWhen the expiration date comes, yeskeyTo delete
• Lazy to delete:keyDo not delete when expired, each time from the databasekeyTo check whether the expiration date, if expired, delete, returnnull.
• Periodically delete: The deletion expires once in a whilekeyoperation

Redis typically uses timed deletion, periodic deletion, and Mercached uses lazy deletion only.

Redis persistence

Redis transactions

A Redis transaction is a collection of commands. Redis transactions can execute multiple commands at once with the following guarantees:

• Batch operations are being sentEXECThe command is placed in the queue cache.
• receivedEXECAfter the command is executed, the transaction is executed. Any command in the transaction fails to execute, and the remaining commands are still executed (without atomicity).
• During transaction execution, command requests submitted by other clients are not inserted into the transaction execution command sequence.

A transaction goes through the following three phases from inception to execution:

1. Start the transaction: The transaction starts firstRedisServer sendingMULTIThe command
2. Command enqueue: Then send the commands that need to be processed in this transaction
3. Perform transactions: Send lastEXECCommand indicates the end of the transaction command

Commands are not executed immediately; they are executed one by one only when the EXEC command is executed.

Transaction-related commands:

• MULTI: enables a transaction
• EXEC: Commits a transaction
• DISCARD: DISCARD a transaction
• WATCH: monitoring
• QUEUED: Queue for adding commands to execution

Differences between Redis transactions and Mysql transactions

Redis Distributed lock There are three behaviors of a Redis distributed lock:

• lockUse:setnxTo rob the lock, set the lock identifier to 1 to indicate that the lock is occupied.
• unlockUse:setnxTo release the lock and set the lock identifier to 0 to indicate that the lock has been released.
• Lock expired:expireAdd an expiration date to the lock in case it forgets to release,expireTime expiration returns 0.

Setnx and EXPIRE are atomic operations, and in practice lua scripts are used to ensure atomicity.

Redis publishes subscriptions

Data types supported by Redis databases

• String: a String, integer, or floating point number
• Hash: An unordered Hash table containing key-value pairs
• List: a linked List in which each node is a string
• Set: unordered collector
• Zset: Ordered set

String Redis uses dynamic strings:

• There are no memory overflow problems caused by string changes
• The time complexity of obtaining string length is 1
• Space pre-allocated, inert space releasedfreeBy default, enough space is left to prevent multiple memory reallocations

Application Scenarios:

• StringCache structure user information, count

Hash array + linked list based on some rehash optimization;

1. ReidstheHashThe chain address method is used to handle collisions, and it does not use red-black tree optimization.
2. Hash table node adopts single – linked list structure.
3. rehashOptimization (Take a divide-and-conquer approach and divide the huge migration effort into each oneCURDTo avoid busy service.)

Rehash indicates that when the hash table load factor reaches the upper limit, the hash table automatically doubles the capacity (number of buckets) and redistributes the original objects to new buckets.

Application Scenarios:

• Saving structure information can be partially retrieved without serializing all fields

List

Application Scenarios:

• Such astwitterFollow list, fan list, etcRedisthelistStructure to implement
• ListIs implemented as a bidirectional linked list, that is, can support reverse lookup and traversal

The internal implementation of a Set is a HashMap with a value of null, which actually computes the hash to quickly sort out the weights, which is why a Set provides the ability to determine whether a member is in the Set.

Application Scenarios:

• Deduplicated scene, intersection (sinter), union (sunion), difference set (sdiff)
• Such as common concern, common preferences, two degrees of friends and other functions

Zset uses HashMap and SkipList internally to keep data stored and ordered:

• HashMapIt puts the members inscoreThe map,scoreIs the basis for sorting.
• Skip list is stored in all the members, it maintains multiple Pointers to other nodes in each node, so as to achieve the purpose of fast access to nodes

Application Scenarios:

• Implement delay queue

Jump table

• Skip list is an extension based on ordered linked list.
• Jumping represents the sacrifice of storage space for performance by building indexes (possibly multilevel indexes) in linked lists.
• Indexes take up memory. The original linked list may store very large objects, while the index node only needs to store key values and a few Pointers, and does not need to store objects. Therefore, when the node itself is large or the number of elements is large, its advantages will be amplified, while its disadvantages can be ignored.
  
  

Redis cluster

Cache and database consistency issues

CAP Principle The CAP principle means that a storage system that provides data services cannot simultaneously meet the following requirements:

• C Data consistency: All applications can access the same data.
• A Data availability: Any application can read and write access at any time.
• P zone tolerance: The system can scale linearly across network partitions. (In layman’s terms, the scale of data is scalable)
  
  

In large sites, it’s often the case that you sacrifice C and choose AP. In order to minimize the impact of data inconsistency, various measures are taken to ensure data consistency.

• Strong data consistency: Data from copies is always consistent in physical storage.
• Data user consistency: Data copies stored in the physical storage may be inconsistent, but a consistent and correct data is returned to the user through error correction and verification.
• Data consistency: Data stored physically may be inconsistent, and end user access may be inconsistent, but data will be consistent over time.

Redis is the ultimate data consistency because of the asynchronous replication nature of Redis.

Solutions to cache consistency:

• Delayed dual-delete policy
• Update the cache via message queues
• throughbinlogTo synchronizemysqlThe database toredisIn the

Delayed dual-delete Policy A write operation performs the following operations:

1. Make caching obsolete
2. Rewrite database
3. Hibernate for 1 second to flush the cache again

Next, we need to figure out why we should use the strategy of eliminating the cache first and writing the database later.

Write to the database and then update the cache: 1. Write to the database and then update the cache.

• Thread A updates the database;
• Thread B updates the database;
• Thread B updates the cache;
• Thread A updates the cache; (A network fluctuation)

A should update the cache earlier than B, but B updates the cache earlier than A because of network reasons. This leads to dirty data. In addition, this situation can only be resolved when the cache is invalid. In this case, services will be greatly affected.

2. Business direction (why phase-out cache instead of updating cache)

• The purpose of caching is to improve the performance of read operations. If you frequently update the cache without a read operation, the performance will be wasted. Therefore, the cache generation should be triggered by the read operation, and the cache elimination strategy should be adopted during the write operation.
• Sometimes we may do some other conversion while caching, but if it is changed immediately, it will also waste performance.

It is not a perfect solution, but it is the most reasonable method. It has the following special cases:

1. Write request A performs write operations and deletes the cache.
2. Read request B finds that the cache does not exist.
3. Read request B to the database query to get the old value;
4. Read request B writes the old value to the cache;
5. Write request A writes the new value to the database. (Taking no action here will cause the database to be inconsistent with the cached data)

This leads to inconsistencies.

The delayed double deletion policy is used to solve the problem that data inconsistency may occur when the cache is eliminated before the database is written. In this case, write request A should sleep for one second and then the cache is eliminated again:

• If the above method is adopted, data inconsistency will occur for nearly 1 second (less than 1 second – read request operation time) during the first write operation. After 1 second, cache flushing will be performed again. After the next read operation, database and cache data consistency will be guaranteed.
• The 1 second mentioned here is used to ensure that the read request ends (usually several hundred ms), and the write request can delete the cache dirty data caused by the read request.

In addition, there is an extreme case: if the cache fails to flush the second time, the data and cache will always be inconsistent, so:

• The cache is set to expire
• Set up a retry mechanism or use message queuing to ensure that the cache is eliminated.

Updating cache through message queue The message queue middleware ensures the consistency between database data and cache data.

• Asynchronous update cache is implemented to reduce the coupling of the system
• But it destroys the timing of data changes
• The cost is relatively high
  
  

Any changes made to the Mysql database at any time are recorded in the binlog; When data is added, deleted, or modified, the created database object is recorded in the binlog. Database replication is also based on the binlog to synchronize data:

• inmysqlWhen the pressure is not high, the delay is low;
• Completely decoupled from the business;
• The timing problem is solved.
• The cost is relatively large

Caching mechanisms