preface
For c-side related services, it is difficult for the mainstream relational database to achieve high concurrency with low latency under the high concurrency query request scenario, and it may even be suspended. Therefore, introducing cache middleware is a common solution, but how to ensure the consistency of cache and database has become a thorny problem. This time, we take Mysql and Redis as examples.
The body of the
The simplest way to maintain the consistency between the cache and the database is to double write or delete the cache to maintain the consistency. To decouple from the business, do this by subscribing to binlog or periodically refreshing.
Consistent solutions for business coupling
The coupling update cache in the business
One common approach is to update the cache, updating the corresponding cached data to the cache as the data changes. This behavior can be decoupled within the same business as well as by messaging middleware. When the query request comes, it directly queries the cache returned data, as shown below:
- When concurrent data changes occur, for example, when A and B change the same row of data, A completes the change first, and then B completes the change. Then B updates the cache, and A updates the cache. The data in the database is B’s data, but in the cache is A’s data.
- Coupling with business is undesirable, increasing business complexity as well as relying on middleware.
The business coupling removes the cache
In fact, both pre-update and post-update caching can cause cache inconsistencies in some extreme concurrent scenarios (think pre-update), so one way to go isDelete the cache. When the data is changed, the cache of the corresponding key is deleted, and the update of the cache is guaranteed by c-end services, as shown in the following figure:
Disadvantages:
- An obvious downside to removing the cache is that if the data query remains high in the second step of removing the cache, then a large number of requests will be made to the DB through the cache, which is still a great risk;
- It is still not decoupled from the business.
A consistent solution for business decoupling
For Mysql, if it wants to decouple from services, the first thing that comes to mind must be binlog, because binlog records all the real data in the database after all. It disguizes itself as a slave node, subscribing to binlog can complete the decoupling of services, as shown in the following figure:
Introduce binlog cache update mechanism of scheduled task and data verification mechanism
So how can we solve the problem of binlog listening data out of order, and have some fault tolerance to ensure the ultimate consistency of the cache?
As shown in the figure below, we have added two changes based on the previous binlog listening scheme.
- Record the binlog generation time when monitoring the consumption of binlog, and compare the binlog time each time to solve the problem of inconsistency between cache and DB data in the case of multithreading consumption;
- Each update generates a binlog. Therefore, all updates (non-service fields, for example, set update_time=now()) in the configuration table can be synchronized to the cache and triggered by scheduled tasks. In this way, service fault tolerance is improved.
conclusion
There is no perfect solution, only the most suitable solution for their own business. If the current infrastructure is weak and the construction period is short, then the cache deletion solution coupled with business will basically meet most scenarios. If the time limit is met, better consistency schemes can be explored, and even distributed transactions can be introduced to ensure that cached data is updated.
Recently, some service requirements have a lot to do with the cache. We will continue to update hot keys and share some solutions of large keys.