Design of cache architecture

Common cache architecture design

The file cache

The file cache here is typically HTTP based file cache, which is typically placed in NignX. Nginx uses proxy_cache to cache the user’s requests to a local directory if they are not updated frequently. The next same request can be directly fetched from the cache without having to request the server.

JVM Cache THE JVM cache is a local cache, designed to work in the application server (Tomcat). Ehcache and Guava Cache are commonly used. In Internet applications, Guava Cache is usually chosen because of high concurrency. Local (JVM) cache scenarios: 1. Very high performance requirements. 4. There is a need to access the entire collection. 5. Data is not allowed to be consistent all the time (no strong consistency is required).

Redis cache Distributed cache

The cache problem

The cache to penetrate

Cache penetration is when data with non-existent keys (non-existent keys) is queried through the cache to the database with high concurrency. The database is overburdened and breaks down

Solution:

1, the query result is empty also cache, cache time (TTL) set a bit shorter, or the corresponding data of the key after the insert cache. The problem: Caching too many null values takes up more space

2, use bloom filter. Before caching, add a layer of Bloom filter. When querying, go to the bloom filter first to check whether the key exists. If it does not exist, return directly.

A brief introduction to Bloom filter

A Bloom filter is actually a long binary vector and a series of random hash mapping functions.

Bloem filters work: When an element is added to a collection, the element is mapped to K points in an array by K Hash functions, setting them to 1. When we search, we know (roughly) if it is in the set by looking at whether all of the points are 1’s: if any of the points have 0’s, the element must not be there; If both are 1, the element being checked is most likely in. That’s the basic idea of a Bloom filter.

Advantages binary storage, saving space. Fast location by subscript, save time.

Cache avalanche

When the cache server restarts or a large number of caches fail in a certain period of time, the failure will also put a lot of pressure on the back-end system (such as DB).

Suddenly a large number of keys fail or Redis restarts, a large number of access to the database, the database crashes

Solution: 1. Set different expiry dates for different keys. 2. Set level 2 cache (data may not be consistent) 3.

Cache breakdown

For some keys that are set to expire, it is a very “hot” data if the keys are likely to be accessed at a very high concurrency at some point in time. At this point, there is a problem to consider: cache “breakdown”. The difference between this and cache avalanche is that the cache is for one key, whereas the cache is for many keys.

When the cache expires at a certain point in time, a large number of concurrent requests for the Key come in, and these requests find that the cache is out of date and usually load data from the back-end DB back to the cache, the large number of concurrent requests can overwhelm the back-end DB instantly.

Solution: Threads that use distributed locks to control access use Redis’s SETnx mutex to determine first, so that other threads are in a waiting state, ensuring that no large concurrent operations are performed on the database.

Data inconsistency

The cache and DB data are inconsistent. Solution: Different policies can be used based on different service scenarios. 1. When data is required to be locked consistently, write lock is added to update data; When querying data, add read lock

2, 1, first delete the cache request data consistent and update the database again Under the condition of the database is not updated, the cache is deleted, so under the condition of high concurrency, there is another thread to get data from the cache, but not cache, so have to take in the database and then in the cache, because by this time the database is not updated, so the cache is old data.

2. The delay of primary/secondary synchronization after updating the database and deleting the cache will also cause data inconsistency

Solution: Delay dual-delete

1, update the database first and delete the cache key, and fill the cache when reading 2, 2 seconds later delete the cache key

Disadvantages: delay double delete, delay time is not easy to determine

Another possibility is to update the database and then delete the cache or delete the cache and then update the database. The second step may fail

For the above questions, you can passMessage queue + asynchronous retryorSubscribes to the binlogTo solve

Additional scenarios and solutions for reference:www.zhihu.com/question/31…

Data concurrency competition

Problem Description: When multiple clients concurrently write a key, the data that should have been received first may arrive later, resulting in incorrect data version. Or if multiple clients get a key at the same time, change the value and write it back, as long as the order is wrong, the data is wrong. The value of a key is 1, which was originally changed to 2,3,4, and finally 4, but changed to 4,3,2, and finally 2.

Solution 1: Distributed lock + timestamp

2. Use message queue + timestamp

When the amount of concurrency is too large, it can be processed by message middleware to serialize parallel reads and writes. Serialize Redis’ set operations by placing them in a queue, which must be executed one by one.

So how do you ensure that the messages arriving in the message queue are in order? There’s no guarantee, so put a time stamp in the message and compare the time stamp when consuming.

Hot key

When there are a large number of requests (hundreds of thousands) to access a certain Redis key, the Redis server breaks down due to the concentrated traffic reaching the network upper limit. A cache breakdown is caused, and subsequent accesses to this key will directly access the database and cause the database to crash, or access the database to backfill Redis and then access Redis and continue to crash.

How do I find hot keys

1. Estimate hotkeys, such as goods that kill seconds, hot news, etc. 2. Make statistics on the client side, which is simple to implement, just add a line of code. 5. Use Streaming computing techniques based on the big data field for real-time data access statistics, such as Storm, Spark Streaming, Flink, these techniques are all possible. After hotspot data is discovered, it can be written to ZooKeeper

How to handle hot keys:

1. Change the distributed cache to the local cache. After the hot key is found, the cache data is extracted and directly loaded into the local cache. Ehcache and Guava Cache can be used, so that the system can directly access its Cache when accessing hot key data. (Data is not required to be consistent all the time) 2. Backup hot key data on each Redis master node, so that the access pressure can be loaded onto each Redis by random reading. 3. Each system instance can request cache cluster read operation no more than 400 times per second by using the fusing protection of limiting the flow of hotspot data access. Once the fusing is over, the cache cluster can be fused, and a blank message will be returned directly, and then the user will refresh the page again by himself later. (The home page is not good, the system is not friendly) through the system layer directly add current limit fusing protection measures, can be very good to protect the cache cluster behind.

Big Key

A large key refers to a large Value stored. Common scenarios are as follows:

• Discussion under hot topics
• Big V’s fan list
• Serialized image
• Garbage data that is not processed in a timely manner

.

Impact of big Key:

• Large keys occupy a large amount of memory and cannot be balanced in the cluster
• The performance of Redis deteriorates and the master/slave replication is abnormal
• The active deletion or expiration deletion will take too long and cause service congestion

How to find big keys:

Redis -cli –bigkeys You can find the maximum key for five data types (String, hash, list, set, and zset) of an instance. 2. Obtain the RDB file of Redis, use rDBTools to analyze the RDB and generate a CSV file. Import the file to MySQL or other databases for analysis and statistics, and collect bigkeys according to size_IN_bytes

Big key processing:

The principle of big key optimization is to reduce the length of string and reduce the number of members of list, hash, set, zset, etc.

1. Big keys of string type should not be stored in Redis, but can be stored in MongoDB or cached in CDN.

If you must store it in Redis, it is best to store it separately, not with other keys. Adopt one master, one subordinate or multiple subordinate.

2. A single simple key stores a large number of values. You can try to split the object into several key-values and use MGET to obtain the values.

2. Hash, set, zset, list contains too many elements. (common)

Take the hash type as an example. If there are too many fields, you can generate a new key based on the hash type, such as the original keyhash_key: {filed1:value, filed2:value, filed3:value ... }, can be moulded to form the following key:valuehash_key:1: {filed1:value} 
hash_key:2: {filed2:value} 
hash_key:3: {filed3:value} ... After the module is taken, the original single key is divided into multiple keys, and the filed number of each key is the original1/N
Copy the code

3. Do not use del when deleting large keys, because del is a blocking command and will affect performance. 4, use lazy delete (unlink)

Deletes the specified key(s). If the key does not exist, the key is skipped. However, unlike DEL, which blocks, this command reclaims memory in another thread, so it is non-blocking. This is where the command gets its name: just remove the keys from the key space, and the actual deletion will be done asynchronously later.

redis> SET key1 "Hello" 
"OK" 
redis> SET key2 "World" 
"OK" 
redis> UNLINK key1 key2 key3 
(integer) 2
Copy the code