Concern public number: XY’s technical circle

Using caching can relieve heavy traffic pressure and significantly improve program performance. We often encounter some “knotty problems” when using caching systems, especially in the case of large concurrency. This article summarizes some common problems with caching and their solutions, which can be used as a reference for future problems and should be considered when designing a cache system.

For the convenience of expression, this paper takes database query cache as an example, using cache can reduce the pressure on the database.

The cache to penetrate

When we use the cache, we usually try to remove the value in the cache first. If there is no value in the database, we return null or throw an exception according to service requirements.

If a user keeps accessing data that does not exist in the database, such as data with id -1, each request will be checked in the cache first and then in the database again, resulting in serious performance problems. This is called cache penetration.

The solution

The following solutions are available:

  • Verify request parameters, such as user authentication, id base verification,id <= 0Direct interception.
  • If no value is found in the database, the corresponding key is also saved in the cache. Value isnull. The next query will be returned directly from the cache. However, the cache time of the key here should be relatively short, such as 30s. In case this data is inserted into the database later and the user cannot obtain it.
  • Use bloom filter to determine if a key has already been checked, and if so, do not query the database.

Cache breakdown

Cache breakdown is when the number of visits to a key is very high, such as a seckill activity, with 1w/s of concurrency. This key expires at some point, and all of a sudden these requests will hit the database, and the database will crash.

The solution

There are several solutions to cache breakdown that can be used together:

  • For hot data, carefully consider the expiration time and ensure that keys do not expire during hot data. Some keys can even be set to never expire.
  • With a mutex (such as Java’s multithreaded locking mechanism), the first thread accesses the key, waits for the query database to return, inserts the value into the cache, and releases the lock so that subsequent requests can fetch the data directly from the cache.

Cache avalanche

Cache avalanche is when, at some point, multiple keys fail. This will result in a large number of requests that do not fetch values from the cache and all go to the database. There is also a case where the cache server goes down, which is also called a cache avalanche.

The solution

There are two solutions to cache avalanche in both cases:

  • Set the expiration time of each key to a random value, rather than all keys being the same.
  • Ensure high availability of caches by using a highly available distributed cache cluster, such as Redis-cluster.

Double write inconsistent

When using a database cache, the read and write flow tends to look like this:

  • When it reads, it reads from the cache first, or if it’s not in the cache, it reads directly from the database, and then retrieves the data and puts it into the cache
  • When updating, delete the cache first and then update the database

The so-called double write inconsistency is that during or after a write operation (update), the value in the database may be different from the value in the cache.

Why delete the cache before updating the database? If you update the database and then fail to delete the cache, the value in the cache will be inconsistent with the value in the database.

However, this does not completely avoid the problem of double – write inconsistency. Suppose that in a large concurrency scenario, one thread removes the cache and then fetches the updated database. At this point, another thread fetches the cache, finds no value, reads the database, and sets the old value of the database into the cache. By the time the first thread has finished updating the database, the new values are in the database and the old values are in the cache, so there is a data inconsistency problem.

A simple solution is to set the expiration time low so that data inconsistencies exist only before the cache expires, which is acceptable in some business scenarios.

Another solution is to use queue assistance. Update the database first, then delete the cache. If the deletion fails, it is put into the queue. Another task then fetches the message from the queue and retries to delete the corresponding key.

Another solution is to serialize read and write operations using a queue for each data. For example, create a queue for data with ID N. Write operations to this data, delete the cache, put into a queue; Then another thread comes along, finds no cache, and puts the read into the queue.

However, this will increase the complexity of the program, serialization will also reduce the throughput of the program, may not be worth the loss. The prevailing solution is to remove the cache first and update the database later. It can meet most of your needs.

Write articles carefully and share with your heart.

Personal website: Yasinshaw.com

Public number: XY’s technical circle