preface

In today’s Internet, high concurrency, large data volume, large traffic has become the language, so our system is also under great pressure, the first solution is Redis.

When redis is misused, avalanche, penetration, breakdown, etc., it is also a time to test a programmer’s technical ability.

Of course, when the interview, this is also a high-frequency interview questions, almost big factory will ask. Here’s a look at some of these technologies.

Cache avalanche

For example,

During the Singles’ Day, when all users open Taobao, they will enter the home page, which is under great pressure. In order to improve the concurrency, the data of the home page of the website is cached into Redis, and the expiration time of all Redis keys is 3 hours.

On The Day of Double 11, a large number of users cut their hands and had a binge. At this time, 3 hours passed, and all the key ccache of the home page in Redis was invalid. At this time, the data could not be queried in Redis, so they had to query in the database, resulting in the failure of the database response and the failure of the database.

When users couldn’t get to the home page and cut their hands off, Ma’s father was not happy and sent the programmer to Africa.

One-sentence summary

In high concurrency, a large number of cache keys fail at the same time, and a large number of requests fall directly on the database, resulting in database downtime.

The solution

  • Set the key failure time randomly to avoid the collective failure of a large number of keys.
SetRedis (Key, value, time + math.random () *10000);Copy the code

  • In cluster deployment, hotspot data can be evenly distributed among different Redis libraries and all key failures can be avoided

  • The expiration time is not set

  • Run scheduled tasks to flush new caches before the cache expires

The cache to penetrate

For example,

Elder brother made a website fire, moved other people’s cake, then began to attack elder brother’s website crazily, because elder brother network security aspect of learning art not fine was drilled by the loopholes.

Someone used a script to send a crazy request to the elder brother, query the id = -1 data, Redis does not have such data, at this time, through Redis, directly into the database.

In the middle of the night, while my brother was sleeping, he didn’t notice. He attacked my brother all night, and the database was down, and then my website was down.

One-sentence summary

There is no relevant data in the Redis cache and database (for example, the user directly initiates requests with the parameter id<=0). There is no such data in Redis, so it cannot be intercepted and directly penetrates into the database, resulting in excessive pressure and database breakdown.

The solution

  • Cache non-existent data into Redis, set key and value to null(whether the data is not null or the system bug), and set a short expiration period to avoid the impact of a long expiration period on normal users.

  • Mask the IP address

  • Verify parameters and intercept illegitimate parameters

  • The Bloom filter hashes all possible data into a bitmap large enough that a certain non-existent data will be intercepted by the bitmap, thus avoiding query pressure on the underlying storage system.

Cache breakdown

For example,

On Nov 11, Ma’s father had a whim to auction off his old cloth shoes, which he had worn for 20 years, with his signature attached. The programmer saved the information of the shoes in Redis and set the expiration date for 3 hours. He thought three hours would be enough for them, but he underestimated papa Ma’s charm.

The product attracted the attention of 10 million people, who kept bidding for the shoes, the price of the higher and higher, Ma’s father is very happy.

After bidding for 2 hours and 59 minutes, we were about to get 100 million yuan. Suddenly, the key data of this pair of shoes in Redis expired. As a result, a large number of requests for this key were sent to the database, which directly caused the database to hang up and the service could not respond.

At the end of the auction, the shoes were not sold, and Father Ma was unhappy again and sent the programmer to Africa.

One-sentence summary

A hot key carries high concurrency constantly. When the hot key fails at the moment, the continuous high concurrency will break the cache and directly access the database, resulting in database downtime.

The solution

  • Set hotspot data to “never expire”

  • Adding a mutex: The phenomenon above is that multiple threads are querying the data in the database at the same time, so we can use a mutex on the first query request to lock the data

    Other threads get to this point and wait until the first thread queries the data and then puts it in the Redis cache. Later threads come in and see that there is already a cache and go straight to the cache

The final summary

Avalanche is a large area of key cache invalidation; The cache key does not exist in Redis. A breakdown is the sudden failure of a hot key in Redis, and the ultimate victim is the database.

thinking

Prepare for a rainy day: Build redis, MySQL and other highly available clusters to prevent single points.

Fix: Limit the flow of the MySQL server and downgrade it to prevent MySQL from crashing.

Rebooting: Redis persists RDB+AOF, automatically loads data from disk after downtime, and quickly recovers cached data.

Follow the wechat official account: IT Brother

Java actual combat project video tutorial: can get 200G, 27 sets of actual combat project video tutorial

Java Learning Route, you can get the latest and most complete a learning route

Reply: Java ebook, you can get 13 top programmers must read books

Reply: Java full set of tutorials, you can get: Java foundation, Java Web, JavaEE all tutorials, including Spring Boot, etc

Reply: resume template, you can get 100 exquisite resume

This article was typeset using MDNICE