preface

In today’s Internet, high concurrency, large data volume and large traffic have become the word, so our system is also under great pressure, the first solution is Redis.

Then the improper use of Redis will produce avalanche, penetration, breakdown and other problems, which is also a test of a programmer’s technical ability of the moment.

Of course, when interviewing, this is also a frequent interview question, almost big factories will ask. Let’s take a look at these technologies.

Cache avalanche

 

For example:

During Singles’ Day, all users will enter the home page as soon as they open Taobao, and the home page is under great pressure. In order to improve the concurrency, the data of the home page of the website will be cached in Redis, and the expiry time of all redis keys is 3 hours.

On double 11, a large number of users cut hands to carnival, at this time 3 hours passed, redis home page key cache all invalid, at this time redis can not query the data, can only go to the database, causing the database can not respond to hang.

Users can not go to the home page can not chop hands, Ma Father is not happy, the programmer sent to Africa.

 

In a word:

In the case of high concurrency, a large number of cache keys fail at the same time and a large number of requests fall directly on the database, causing the database to break down.

Solution:

  • Set the key expiration time randomly to prevent a large number of keys from failing collectively.
SetRedis (Key, value, time + math.random () * 10000);Copy the code
  • In cluster deployment, hotspot data can be evenly distributed in different Redis libraries to avoid all key failures
  • No expiration time is set
  • Run timed tasks to flush new caches before they expire

The cache to penetrate

 

For example:

The elder brother made a website fire, moved the cake of others, then began to attack the website of the elder brother crazily, because the elder brother network safety respect learns art not essence to be drilled by the person loophole.

Redis does not have such data. At this time, it penetrates redis and directly hits the database.

The elder brother was sleeping in the middle of the night and did not notice, he crazy attack elder brother one night, the results of the database to hang, and then the elder brother’s website also hung.

In a word:

There is no relevant data in redis cache and database (for example, the user directly carries the parameter ID <=0 to continuously initiate requests). There is no such data in Redis, so it cannot be intercepted and directly penetrated to the database, causing the database to break down due to excessive pressure.

Solution:

  • Cache non-existent data in Redis, set key and value to null(whether data is not null or system bug), and set a short expiration period to avoid affecting normal users.
  • Mask the IP address
  • Verify the parameters and intercept invalid parameters
  • The Bloom filter hashes all possible data into a bitmap(bitmap) large enough to intercept data that must not exist, thus avoiding query pressure on the underlying storage system.

Cache breakdown

 

For example:

On Double Eleven, Ma’s father had a whim to auction the old cloth shoes he had worn for 20 years with his signature attached. The programmer saved the information of the shoes in Redis and set the expiration date for 3 hours. He thought three hours would be enough for them, but he underestimated papa Horse’s charms.

The item attracted the attention of 10 million people, who kept bidding for the shoes, and the price kept going up and up, and The father horse was delighted.

After 2 hours and 59 minutes of bidding, the key data in redis of this pair of shoes was out of date. As a result, a large number of requests for the key were sent to the database, which directly caused the database to hang up and the service could not respond.

The auction is over, the shoes are not sold, and Father Ma is unhappy again and sends the programmer to Africa.

 

In a word:

A hot key is constantly carrying high concurrency. When the hot key fails, the continuous high concurrency will break the cache and directly access the database, causing the database to break down.

Solution:

  • Set hotspot data to “never expire”
  • Plus the mutex, the above phenomenon is the multiple threads to query the database of the data at the same time, so we can request on the first query data using a mutex locking it other threads can’t get this far the lock is wait, order of merit a thread query to the data, and then put the data in redis cached. The next thread comes in and finds that there is already a cache, so it goes directly to the cache

The final summary

Avalanches are massive key cache failures; Redis does not have this cache key; Breakdown is the sudden failure of a hot key in Redis, and the ultimate victim is the database.

thinking

Be proactive: Build redis, MySQL, etc into highly available clusters to prevent single points.

Better late than never: limit the flow in the service + downgrade, prevent MySQL from being crashed.

Regroup: Redis persistent RDB+AOF, downtime and restart, automatically load data from disk, fast recovery of cached data.