preface
How to effectively understand and tell the difference between Reids penetration, breakdown and avalanche has always puzzled me. In particular, penetration and breakdown, over a period of time is confused.
To help me, and my friends who have the same problem, distinguish between these three scenarios. The author summarizes some key words, I hope you can be like me through the way of association to distinguish and understand the difference between these three scenes!
The cache to penetrate
Key words: Cross Redis and database
When we don’t have the data we want in Redis or the database, we need to consider cache penetration. The following logic we use will be more: first go to Redis to find a resource, Redis can not be found in the DB to check, DB back to write a data to Redis.
This logic is not a problem under normal circumstances, but if the user repeatedly requests resource X maliciously, the resource does not exist in Redis or DB. Then each request will be directly sent to the DB, even causing the physical DB to break down.
Solution:
1. Cache empty results
If the system finds that the resource does not exist in Redis or DB, it caches the null result for a period of time. It should be noted that the failure time should not be set too long, otherwise the effectiveness of the data will cause great problems.
2. Verify user validity
Verify the validity of user requests and block malicious repeated requests.
3, Bloom filter
Don’t panic when you see this noun. The simple purpose of a Bloom filter is to help you determine whether a value exists. For example, suppose we now have an array of 9 bits. Each bit can hold only 1 or 0, 1 indicating that the bit is occupied, and 0 indicating that the bit is unused.
For key1, we Hash it separately with three Hash functions.
And then you take these three hashes and you take the modulo of 9.
Finally, the three modules are dropped into the bit array.
Key2 and key3 are processed again in the same way.
Finally, we find that only position 3 is still empty in the array of bits. If a new key4 comes in with Hash values of 1, 2, and 3, we can conclude that key4 does not exist.
The principle of a Bloom filter is relatively simple. It’s important to note that the Bloom filter may have some miscalculation potential, but it can still help you block most of the data that definitely doesn’t exist.
Cache breakdown
Key words: targeted strike
Imagine if all the requests were stuck on one key, wouldn’t that be a targeted attack?
How do you understand that? Take an extreme example: for example, a certain star broke a shocking and cruel news, and a large number of people visited Weibo to check the gossip news. However, the data in the Redis cluster of Weibo just expired at the moment, so the numerous requests were directly sent to the physical DB of weibo system, and the DB was suddenly suspended.
Solution:
1. Hot data never expires
For example, we can set the cache time of a key to 25 hours, and then have a JOB in the background to batch refresh the hot data every 24 hours. We can solve this problem.
2. Use mutex
It is easy to affect throughput, and most projects have no problem setting hot key to never expire.
Cache avalanche
Key words: Redis crash, no data
The Redis crash does not mean that the Redis cluster is down. It means that at some point the hot keys in the Redis cluster all fail. If the hot key in the cluster fails simultaneously at some point, imagine that a large number of requests will be directed to the DB, and the DB may be overwhelmed in an instant.
Solution:
1. Redis expiration time plus random number
Redis expiration time plus random number is a tricky solution. The instantaneous pressure of DB is reduced to some extent, but this scheme also increases the maintenance cost to some extent.
2. Redis never expires
The implementation scheme was briefly mentioned above.
conclusion
Finally, let’s get back to the subject! How can I easily tell the difference between Redis cache penetration, breakdown and avalanche in an associative way?
Cache penetration – passes (bypasses) Redis and DB to screw you
Cache breakdown – targeted attack to screw you
Cache avalanche – Hot keys fail simultaneously at some point