The topic I want to talk about is: the final chapter of distributed multi-level cache architecture, how to solve the business scenarios of large traffic and high concurrency, depending on whether you can become the top of the pyramid in this field? How well you think about this will determine how fast you grow.
Many people in an industry for 5 or 10 years, still not reached the middle or even stay at the bottom of the industry, because they never care about such topics. As a practitioner of forging ahead, I feel it is necessary to share with you.
The opening
Server caching is an important part of the overall cache architecture, and you have seen the importance of server caching in system performance since the beginning of the evolution of web architecture.
But the database is the system of “amateurs | slow coach”, sometimes beat database tuning is to small to big, without changing the architecture and code logic premise, cache parameters adjustment is often a shortcut.
In the process of system development, the cache framework can be directly used in the platform side. When the cache framework cannot meet the performance requirements of the system, the application level cache needs to be independently developed in the application layer.
Redis is commonly used in caches. What exactly is platform level and application level cache?
We’ll find out later. But the platform level is what language you choose to use for caching, and the application level is what the application does for caching.
01 Database Cache
Why are databases slow? Slow is not the answer for you, who now love fast. It’s like I always feel like girls are slow to respond
Because databases are IO intensive applications, they are mainly responsible for data management and storage. Multiple queries can be inherently slow, which is why queries like to speed up with indexes when data is on the table. Of course, the database itself has a “cache” to solve this problem.
Shouldn’t more data query be slow? Xiaobai said za-za-hui you don’t understand
. Well, that’s not all you said, but the score. For example, there are billions of lines of data
The reason:
- Because the result of simple SQL is not very much. You’re not asking much. The disk’s on top
- The total number of concurrent requests exceeds the disk throughput limit
Even if you don’t like zahs, I’m going to write
Database caching is a special caching mechanism of its own. Most databases run quickly without configuration, but are not optimized for specific needs. Cache optimization is something you can consider when tuning your database.
Take MySQL as an example. MySQL uses the query buffering mechanism to store SELECT statements and query results in a buffer as key-value pairs. In the future, the same SELECT statement will be read directly from the buffer to save query time and improve the efficiency of SQL queries.
1.1.MySQL query cache
Query cache applies to the entire MySQL instance. It is mainly used to cache the ResultSet in MySQL, which is the ResultSet executed by an SQL statement, so it only applies to select statements.
When Query Cache is enabled, when MySQL receives a request for a select statement, if the statement meets the conditions of Query Cache, MySQL will directly HASH the received select statement as a string based on the preset HASH algorithm. Then Query Cache directly to check whether the Cache has been cached.
If the result set is already in the cache, the SELECT request returns the data directly, eliminating all subsequent steps (such as SQL statement parsing, optimizer optimization, and requesting data from the storage engine), which greatly improves performance.
Of course, if the data changes very frequently, using Query Cache may not be worth the cost.
Why is that? Isn’t that supposed to speed things up? It’s not worth it
Because MySQL remaintains the cache whenever data changes are involved.
- If the SQL request volume is large, you can retrieve it through cache off disk during maintenance. That must put a lot of pressure on the database.
- Rebuild cached data, which requires mysql background threads to work. It also increases the load on the database.
So it has been cancelled in MySQL8. Therefore, it can be used in scenarios where the data does not change much, such as: blog
Query Cache usage requires several parameters, the most important of which are query_cache_size and query_cache_type. The former is used to set the memory size of the Cache ResultSet, and the latter to set the scenarios in which Query Cache is used.
1.2. Verify the rationality of Query Cache
To check whether the Query Cache is properly set, run the following command on the MySQL console:
- SHOW VARIABLES LIKE ‘%query_cache%’;
- SHOW STATUS LIKE ‘Qcache%’; Check the following parameters to see if query_CACHE_size is set properly:
- Qcache_inserts: Indicates how many times the Cache misses and is then inserted into the Cache
- Qcache_hits: indicates the number of hits. It reflects the effect of cache usage.
If the value of Qcache_hits is very large, it indicates that the query buffer is being used very frequently. If the value is too small to be efficient, consider not using the query buffer.
- Qcache_lowmem_prunes: Indicates how many queries are cleared from Query_Cache due to insufficient memory.
If the value of Qcache_lowmem_prunes is very large, it indicates that the cache is often underbuffered due to increased cache capacity.
- Qcache_free_blocks: indicates cache fragmentation
A very large value of Qcache_free_blocks indicates that there is a lot of fragmentation in the cache and you may need to find an opportunity to defragment.
The hit ratio of Query Cache can be calculated using the Qcache_hits and Qcache_inserts parameters:
The combination of Qcache_lowmem_prunes and Qcache_free_memory gives you a better idea of whether the system’s Query Cache memory is really large enough and whether queries are frequently swapped out due to insufficient memory.
1.3.InnoDB cache performance
When selecting InnoDB, the innodb_buffer_pool_size parameter is probably the most critical parameter affecting performance. It is used to set the size of the memory area for caching InnoDB indexes and data blocks, adaptive HASH, write buffers, etc. More like Oracle database db_cache_size.
In simple terms, when operating on an InnoDB table, all data returned or any index blocks used in a query are queried in this memory area.
Like key_buffer_size in MyISAM, Innodb_buffer_pool_size sets the memory area that the InnoDB engine needs the most, which is directly related to the performance of the InnoDB storage engine, so if you have enough memory, This parameter can be set to large enough to put as many InnoDB indexes and data into the cache as possible, until all.
When you’re talking about caching, you’re talking about cache hit ratio. So how does InnoDB calculate?
After calculating the cache hit ratio, the innodb_buffer_POOL_size parameter size is optimized based on the hit ratio
Except for the query cache. The performance of database queries is also related to the number of MySQL connections
Table_cache Sets the number of table caches.
show global status like ‘open%_tables’; # query parameters
Because each client connection accesses at least one table, this parameter is related to max_connections. When a connection accesses a table, MySQL checks the number of currently cached tables.
If the table is already open in the cache, it is directly accessed to speed up the query. If the table is not cached, the current table is added to the cache for query.
The table_cache parameter is used to limit the maximum number of cached tables before performing caching operations:
If the number of tables currently cached does not reach table_cache, a new table is added. If the value is reached, MySQL releases the cache based on the last query time and query rate of the cache table.
02 Platform level cache
What is platform-level caching?
Platform-level caching refers to the development language you use, depending on which platform you choose, since caching itself is for upper-level calls. It is mainly aimed at application frameworks with caching features, or special libraries that can be used for caching functions.
Such as:
- Smarty template library in PHP
- In Java, there are more caching frameworks, such as Ehcache, Cacheonix, Voldemort, JBoss Cache, OSCache, and so on.
Ehcache is the most popular pure Java open source cache framework. It is simple in configuration, clear in structure and powerful in function. It is widely used from Hibernate cache. EhCache has the following features:
What is distributed caching? Like I don’t get it yet, Zazz
First, look at the immutable “distribution,” which is independently deployed to multiple service nodes or independent processes that communicate and coordinate with each other only through messaging.
That is, distributed caching, which either has multiple instances on a single machine or is deployed independently to different servers to spread the cache around
Finally, the client connects to the corresponding node for caching operation.
Voldemort is a Java-based distributed key-value caching system. Like JBoss’s caching, Voldemort supports cache synchronization between multiple servers to improve system reliability and read performance.
Voldemort has the following features:
Simply put, in terms of platform-level caching, you only need to configure properties on the framework side without calling specific methods or functions.
The introduction of caching technology in the system is often from the platform level cache, the platform level cache is often used as a level 1 cache.
Since platform-level caching is implemented using framework configuration, how does this achieve distributed caching? The nodes no longer communicate with each other
If you just look at the framework cache call, it is really not possible to achieve distributed cache, because it does not have a distributed deployment like Redis, through the network to connect the nodes. However, the local platform cache can manipulate the platform cache data distributed across nodes through remote procedure calls.
In the Ehcache:
03 Application-level Cache
Application level caching is considered when platform level caching is not sufficient for system performance. Application level caching requires developers to implement caching mechanisms through code.
There is a bit of a difficult party, p plus support feeling. If you can’t do it yourself, ask someone else
This is the battleground for NoSQL, whether Redis or MongoDB, and Memcached as support for application-level caching. A typical approach is to uniformly generate some type of page every minute or over a period of time and store it in the cache, or the cache can be updated as hot data changes.
Why is platform caching not enough to meet system performance requirements? Doesn’t it also reduce the network overhead of application caching
Here are a few things to look at:
3.1 Redis-oriented cache application
Redis is an open source, bSD-licensed advanced key-value pair caching and storage system. For example, Sina Weibo has almost the largest Redis cluster in the world.
Why is Sina Weibo the largest Redis cluster in the world?
Weibo is a social platform, in which the user follows and is followed, weibo hot search list, click volume, high availability, cache penetration and other business scenarios and technical problems. Redis has corresponding hash, ZSet, bitmap, cluster and other technical solutions to solve.
It is easy to use in scenarios where data relationships are complex and variable. Such as:
User follow and unfollow: It’s easy to maintain a list of users using a hash. You can just go to the key and change the following user in the value.
If you are like memcache, you can only serialize the user care list store first, and then deserialize the changes. And then cached, like big V has millions, tens of millions of users, once followed/cancelled. There is a delay in the operation of the current task.
Reddis main features
- Master-slave synchronization
Redis supports master/slave synchronization. Data can be synchronized from the master server to any number of slave servers, and the slave server can act as the master server associated with other slave servers. This allows Redis to perform a single layer tree copy.
- Publish/subscribe
Since the publish/subscribe mechanism is implemented, the slave server can subscribe to a channel and receive the complete message publishing record from the master server when synchronizing the tree anywhere. Synchronization helps with scalability and data redundancy for read operations.
- The cluster
Redis 3.0 adds cluster functionality to solve the problem that Redis single point cannot scale horizontally. The Redis cluster is implemented in node-less mode without proxy proxy. The client directly connects to each node of the Redis cluster, calculates the slot corresponding to the key according to the same hash algorithm, and runs commands on the Redis corresponding to the slot.
From the Redis perspective, response time is the most critical condition, and the overhead of adding another layer is unacceptable. Therefore, Redis implements direct client access to nodes. For decentralization, nodes exchange each other’s status through the Gossip protocol and probe the information of newly added nodes. Redis clustering supports dynamic node addition, dynamic slot migration, and automatic failover.
An architectural schematic of the Redis cluster is shown.
So what is the Gossip protocol? It’s so cool. All these deals are popping up
The Gossip protocol is a multicast protocol. The basic idea is that a node wants to share some information with other nodes in the network. It then periodically selects random nodes and passes information to them. The nodes that receive the information then do the same thing, passing the information on to some other randomly selected node. Up to all the nodes.
That is, the Redis cluster adds, removes, elects the master node, is based on this way.
For example, when joining a new node (meet), a node will be randomly selected in the cluster to invite a new node. At this time, only the inviting node and the invited node know about this event, and the other nodes have to wait for the ping message to spread layer by layer. In addition to Fail, the network is notified immediately. Other nodes, such as new nodes, node re-online, node election as master node, and slot change, need to be notified. Therefore, the Gossip protocol is also the final consistency protocol.
This kind of multicast way, is not suddenly a kind of good things do not go out, bad things spread thousands of feet
However, the Gossip protocol also has imperfections, such as the Byzantine problem. That is, if there is a maliciously propagating node, the distributed system of the Gossip protocol will have a problem.
Note: Redis cluster node communication message type
All Redis nodes are connected to each other through the PING PONG mechanism, using binary protocols internally to optimize transmission speed and bandwidth.
Why does ping increase transmission speed and bandwidth? I don’t know much about it, Zack. So this is related to the OSI network hierarchy
Under the OSI network-level model, ping belongs to the network layer, so it reduces the overhead of network-level transmission, while binary is the smallest unit of 0,1 bits.
The bandwidth is fixed, so if you send packets that are small, it’s fast, and there’s no complicated unpacking of big packets. Equivalent to others on business 1 catty more MacPro. You’re on a business trip with five pounds of Ares.
What are the bottlenecks of Redis? Za-za-hui to arrange
Redis itself is an in-memory database, read and write I/O is its strength, the bottleneck is single thread I/O and memory capacity. We have multithreading now,
For example: Redis6 has multithreaded mode for network transmission, keyDB is directly multithreaded.
What? Redis6 multi-threaded mode is not understood yet, so I will talk about it separately
How can I discover cluster node faults?
Node failures take effect only when more than half of the nodes in the cluster have been detected as failing. The client is directly connected to the Redis node. The client does not need to connect to all nodes in the cluster, but to any available node in the cluster.
The Redis Cluster maps all physical nodes to slots. The Cluster maintains the mapping between nodes, slots, and values. When a node is faulty, all the master nodes in the cluster participate in the election process. If the communication between more than half of the master nodes and the current master node times out, the current master node is considered to be down.
Why not do this without the Slave node?
In cluster mode, requests are automatically separated from read/write requests. But now you choose the primary node. Only the master node can participate in identity.
After all, in cluster mode, there are multiple master nodes, and each slave node corresponds to only one master node. Can your slave nodes vote for the master node in the whole cluster mode?
Just like the little sister has an object, that is, the famous flower has a master, you can still in the case of a master, to choose a? Beware of social beatings
If more than half of the master nodes in the cluster fail, the entire Redis cluster becomes unavailable regardless of whether there is a slave cluster.
When the cluster is unavailable, all operations on the cluster are unavailable and you will receive an error message:
[(error) CLUSTERDOWN The cluster is down].
There are a number of client programming languages that support Redis for most applications, as shown in the figure below.
3.2. Multi-level cache instances
An application architecture using Redis clustering and a variety of other caching technologies is shown in figure 1
Load balancing
First of all, user requests are distributed to Nginx by load balancing services. The commonly used load balancing algorithms here are polling or consistent hash. Polling can make server requests more balanced, while consistent hash can improve the cache hit ratio of Nginx applications.
What is a consistent hash algorithm?
The hash algorithm results in a value that is itself unique, allowing each user’s request to land on the same server. By default, the user logs in from that server and generates session files to that server, but needs to log in again the next time a request is redistributed to another server.
A consistent hash algorithm can cure this problem by directing requests to the same server. Of course, I’m not going to show you the consistency hash principle here. Behind schedule
Nginx local cache
The request is sent to the Nginx application server, and the local Cache is read first. The local Cache can be implemented by Lua Shared Dict, disk – or memory-oriented Nginx Proxy Cache, or the local Redis implementation, etc. If the local Cache hits, the local Cache is returned directly.
What makes this local cache feel so special? It’s like your little sister, so close to you, it’s a pity you can’t eat it. Pooh, pooh, that’s beside the point
- Lua Shard Dict is a memory space created by Lua to store cached data on Nginx. This is equivalent to using nginx process resources
- Nginx Cache means that nginx retrieves data from upstream services and caches it locally.
- Local Redis means that nginx and Redis are deployed on the same service and Nginx operates Redis directly
What! Nginx can also operate directly Redis ah, listen to me carefully
These methods vary. Lua Shard Dict uses Lua scripts to control the size of cached data and allows you to modify the cached data logically.
Nginx Proxy Cache development is relatively simple, is to obtain upstream data to the local Cache processing. Local Redis needs to be set up through lua scripting logic, although the operation is cumbersome, but solves the problem of local memory limitations. So nginx operation Redis needs to use Lua
What are the advantages of nginx local caching?
Nginx application servers use local caching to improve overall throughput and reduce back-end stress, especially when dealing with repeated reads of hotspot data.
How to resolve a local cache miss?
If the local cache of the Nginx application server is not hit, then the corresponding distributed cache will be read further. Redis distributed cache cluster can consider using the master-slave architecture to improve performance and throughput. If the distributed cache is hit, the corresponding data will be returned directly and written back to the local cache of the Nginx application server.
If the Redis distributed cache also does not hit, the source is returned to the Tomcat cluster. Polling and consistent hashing can also be used as load balancing algorithms when the source is returned to the Tomcat cluster.
What if I’m a PHP tech stack? You don’t even use Tomcat in Java
Nginx is often used in the reverse proxy layer. Whereas Tomcat is more of an application server, if PHP is used, the phP-FPM or Swoole service accepts the request. That is, whatever the language, you should find something that accepts the request for distribution in that language.
Of course, if the Redis distributed cache is not hit, the Nginx application server can also try to read the primary Redis cluster again to prevent the traffic impact that may occur if the secondary Redis cluster has a problem.
This design is beyond my comprehension
If your site has a lot of traffic, if it is not read in the Redis distributed cache, it will be directly through the database, the traffic may flush the database. The primary read here also takes into account the master-slave delay in the Redis cluster, in order to prevent cache breakdown.
In Tomcat | PHP – FPM cluster applications, the first reads the local platform level cache, if the platform level cache hit the return data directly, and synchronize writes the main Redis cluster, by the master-slave synchronization to from Redis cluster.
Multiple Tomcat instances may write to the primary Redis cluster at the same time, which may cause data confusion. Therefore, you need to pay attention to the update mechanism and atomization of the cache.
How do you ensure that the atomization operation is performed?
When multiple instances want to write Redis cache at the same time, in order to keep atomization, lua script should be used at least to encapsulate the Key involved in this business, and then obtain the data in Redis cluster through distributed lock or re-merging the same request into a queue, so that the request that obtains the lock or from queue POP can read the data in Redis cluster.
If none of the caches are hit, the system can only query the database or other related services to get the relevant data and return it. Of course, we already know that the database also has a cache. Is the arrangement clear?
When, the little elder sister also can understand my good and bitter heart… Silently alone shed tears
3.3. Caching algorithm
Caches generally use memory as storage medium, and the cost of using indexes is relatively high. So there are a few terms you need to know about caching when using it.
Cache elimination algorithm
The implementation of the substitution strategy is the cache elimination algorithm.
Frequency of use:
- Least-recently-used (LRU) Replaces the Least Recently requested object.
Works well in CPU cache flushing and virtual memory systems. However, it is less effective in direct applications and proxy caches because the time locality of Web access often varies greatly. Browsers generally use LRU as a caching algorithm. New objects are placed at the top of the cache, and when the cache reaches its capacity, the bottom objects are removed by placing the newly accessed objects at the top of the cache pool.
- Least-frequency-used (LFU) replaces the Least Frequently accessed cache. This strategy is intended to retain the most Frequently Used and popular objects and replace the data that is rarely Used.
However, some documents may be used frequently but never again. Traditional LFU policies do not provide any mechanism for removing such files, and thus result in “cache contamination”, where a previously popular cache object stays in the cache for a long time, thus preventing replacement of it by new and potentially popular objects.
- Pitkow/Recker replaces the least recently used object
Unless all objects are accessed today. If so, the largest object is replaced. This strategy attempts to fit a specific pattern of daily Web page visits. This strategy is also recommended to run at the end of each day to free up space occupied by “old”, least-recently used objects.
- Adaptive Replacement Cache (ARC) ARC is between LRU and LFU. In order to improve the effect, it is composed of two LRU.
The first contains items that have been used only once recently, while the second LRU contains items that have been used twice recently, thus yielding new objects and commonly used objects. ARC is self-regulating and low load.
- Most Recently Used (MRU) MRU is opposite to LRU. It removes the Most Recently Used object.
When something is unpredictable and finding the least recently used object in the storage system is a very time-intensive operation, MRU is considered, which is common in database memory caches.
Access to the count
- Least Recently Used2 (LRU2)
A variant of LRU puts objects that have been accessed twice into the cache pool, and when the cache pool is full, removes the objects that have been used the least twice.
Because objects need to be traced twice, the access load increases as the cache pool increases.
- Queues Two Queues are another variant of LRU.
The accessed data is placed in the LRU cache, and if the object is accessed again, it is moved to the second, larger LRU cache, using multi-level caching. The cache object is removed to keep the first cache pool 1/3 the size of the second.
When the access load of the cache is fixed, replacing LRU with LRU2 is better than increasing the capacity of the cache.
Cache capacity algorithm
-
SIZE replaces the object with the largest footprint, a strategy that improves hit ratio by eliminating one large object instead of multiple small objects. However, it is possible that some small objects that are put into the cache are never accessed again. The SIZE policy does not provide a mechanism to weed out such objects and can also lead to “cache contamination.”
-
Lru-threshold does not cache objects larger than a certain size. Other objects are the same as LRU.
-
Log (Size) +LRU replaces the object with the largest Size. If the Size is the same, replace the object with LRU.
Cache time
-
Hyper-g LFU is an improved version of hyper-G LFU that takes into account both the last access time and object size.
-
Lowest-latency -First replaces documents with the least time to download. Obviously the goal is to minimize the average delay.
Cache assessment
- Hybrid Hybrid has a goal to reduce average latency.
A retention utility is calculated for each document in the cache, and the object with the lowest retention utility is replaced. The utility function of document F on server S is defined as follows:
Bs is the bandwidth of server S; FRF stands for the frequency of f; Sizef specifies the size of document F, in bytes. K1 and K2 are constants, and Cs and BS are estimated based on the last time a document was retrieved from server S.
- LRV is also based on calculating the retention utility of documents in the cache and then replacing the document with the Lowest retention utility.
Random and queue algorithms
- First in First out
FIFO keeps track of all cache objects through a queue. The most recently used cache objects are placed behind, and older cache objects are placed in front. When the cache capacity is full, the first cache objects are kicked out and new cache objects are added.
- Random Cache is the Random substitution of cached data. It is better than FIFO and, in some cases, even better than LRU, but generally LRU is better than Random Cache.
There are also many caching algorithms, such as Second Chance, Clock, Simple time-based expiration, Extended time-based expiration, Sliding time-based expiration… There is no superior or inferior cache algorithm, and different cache algorithm will be used in different application scenarios. When implementing a caching algorithm, factors such as ** frequency of use, acquisition cost, cache capacity and time are often considered. **
04. Use the cache service of public cloud
- Dynamic capacity expansion:
Users can through the control panel to upgrade Redis storage space, service does not need to interrupt or stop the expansion process, the whole process is transparent to users and no capacity of perception, and the use of cluster to solve Redis smooth expansion is a very tedious task, now you need to use your hands press a mouse can fix, greatly reduce the burden of the operations.
- Multiple data backup:
The data is stored on two machines, one of which is down, and the data is backed up on the other.
- Automatic DISASTER recovery:
When a host is down, the system automatically detects the host and switches over to the standby host, achieving high availability of services.
- Low cost:
In many cases, in order to improve the performance of Redis, it is necessary to purchase a special server for Redis storage service. However, this will lead to the waste of some resources. Purchasing Redis cloud storage service can solve this problem well.
With Redis cloud storage service, background developers can be liberated from cumbersome operation and maintenance. In the application background service, if we independently build a high availability and high performance Redis cluster service, we need to invest considerable operation and maintenance costs and energy.
If you use cloud services, you don’t need to invest all that money and energy, and you can keep the backend application developers focused on the business.
I’m Za-za-hui, and I love to analyze advanced knowledge. See you next time. If you find this article helpful, welcome to share + attention. At the same time, I have also sorted out e-books about the improvement of back-end system and knowledge cards about technical problems, and shared them with you. I will continue to update them later. Your attention will be the biggest motivation for me to continue writing. Friends in need can search on wechat [Lotus Child Ne Zha] reply: knowledge