First, we need to take a look at some of the categories and noun descriptions in the cache
According to the name, it can be divided into:
Fixed name: usually a table name, field name, function name, or a combination of the preceding items…… And so on according to each person’s preferences to name, call also directly convenient
Fixed prefixes (single or multiple variable suffixes or combination of variable suffixes and fixed suffixes) : table name + record Id, temporary table name (table name changes according to certain rules, for example, the date suffix or Id of associated table), function name + code…… The call requires the specified parameters to be passed in dynamically, and you cannot operate on the cache without knowing the parameters
According to the cache dependent content, it can be divided into:
Dependency on a specified table: When records in a specified table are added, modified, deleted, or updated, the cache content needs to be updated
Depending on some or some records in a specified table: ditto
Rely on multiple table data: same as above
Depending on the specified field value: When the silent field value changes, synchronous modification (mainly used for updating the fields with high frequency, such as page click count, etc., if the cache is needed, it needs to be stored independently to avoid the synchronization clearing function during update)
Dependent on other caches: When you specify a cache value change, you need to synchronize changes to other cache values that depend on it (for example, depending on some calculation result or state value; Store some temporary records, etc.)
.
Operations affecting cached values can be classified as:
Add, modify and delete data table records; Update changes to other cached values; The change of some calculation results, etc
When the dependent content changes, the relevant cache needs to be updated synchronously, which can be divided into real-time synchronization and delayed synchronization
Cache update strategies include: real-time update (mainly for record level cache, directly synchronize the update of specified records; , of course, also can update the entire table but it has a larger effect on program execution performance), timeout detection (such as cache depend on other cache, set up a last update time and access time, by comparing the two time to determine whether the cache expires), absolute time expired (set for cache expiration time), dynamic time expired (cache expiration time after being accessed postpone), etc.
By cache data set size:
Single value, single record, small data set, medium data set, large data set, super large data set
For cache management, the smaller the data set, the faster the access and conversion speed. Therefore, when the data set is too large, it must be divided into small pieces as far as possible to improve the performance of cache
By cache update frequency:
Fixed value (that is, some configuration information whose value does not change after it is stored in the cache), updated occasionally, updated frequently, updated frequently
By cache level:
None Tier cache, tier 2 cache
Other:
Data caching, page caching……
Generally speaking, most people use cache directly key-value, this kind of operation is simple and convenient, without too many algorithms to deal with. Such operations for the larger data record collection (of course not large or very large data cache) directly, frequently to convert the data access will also consume a lot of resources, so sometimes need on the basis of this to add a second level cache, will be no cache read out the data of load the IIS cache, program code directly call, Reloading only when the relevant values change reduces the performance cost of big data conversion, and of course the complexity of the program is greatly increased.
For caches that use level 2 caches or rely on other caches, frequent or frequent updates have the greatest impact, and poorly written programs directly cause a geometric drop in performance (because each update requires all caches to be updated synchronously).
Current system use of cache status
At present, my framework uses the two-level cache. First, Redis cache is used to store various table records and values. Then, for some data with a large amount of data and little modification but relatively frequent use, in order to reduce the deserialization operation after constantly reading from Redis cache, It will be read from the Redis cache and stored in the IIS cache. When these data are updated, the data in the IIS cache will be updated synchronously in real time. These codes are encapsulated in the logical layer, and unified template generation is convenient and fast.
The modules with a large amount of service data are divided and N temporary tables are generated according to different attributes every day. Scheduled tasks are performed in the early morning of the next day to analyze and process these temporary tables. After invalid data is removed, the temporary tables are updated to the history table (the history table is generated monthly). After service data is divided, the number of records in each table is small, and they are stored in the corresponding cache for the call processing of the front and back end, service, Socket and other interfaces. At present, all functions of different servers share one cache service. Various system services store frequently used data or records in designated caches, reducing cross-temporary or full-table data queries. Some functions only use some of the latest data to be used that day. Old data is no longer used and does not need to participate in the query. A separate cache is also used for storage, and the cache storage is managed according to fixed prefix + regular suffix.
The front-end page is cached directly in Redis.
.
There are problems with cache processing
In addition to the foregoing cache, while some backend update our data (such as commodity information), it is necessary to remove all the front page caching, fully regenerates (because many pages will show commodity information), without a comprehensive management caching framework, the update will be some don’t have to delete the cache synchronization also clear. In some cases, when a cache is dependent on many other caches, clearing that cache will also clear some of the excess cache, rather than pinpoint it. For caches with dynamically generated variable suffixes, it can be difficult to update these cache contents synchronously when the suffix parameter cannot be passed at some point.
Cache processing solution
For some of the above problems, after comprehensive consideration, want to write an independent cache processing plug-in to deal with these problems. The cache is directly bound to data table, field, record Id, associated cache, page cache and other associated content through configuration. When these cache updates, the corresponding cache module is cleared synchronously, so that other caches restart the cache loader to load the corresponding data into the cache. It is targeted at clearing, without clearing redundant caches across borders. Do not know everybody has what good suggestion?
Here are some solutions:
First, the cache plug-in must be a separate program
2. Calls must be handled through a unified interface
Cache associations must be configured to bind
4. Cache naming must conform to certain specifications
Specific implementation methods:
Get cache: Get Cache => Check null => Load => Save => Return Cache The Load function is implemented by the operator, using configuration +IoC. The IoC configuration file and interface file can be generated directly from the T4 template. The data is then stored in the cache, and the desired cache is returned. Of course, if the cache is not empty, return the cache directly.
Storing data: Save Cache => Save => Check Relevance => Delete Relevance Cache The caches are cleared synchronously so that they can be reloaded the next time they are fetched.)
Delete Cache: Delete Cache => Delete => Check Relevance => Delete Relevance Cache
Set the cache parameters: Set (modify some global configurations of the cache plug-in)
Only Get/Save/Delete is directly called to the external, and the interface that needs to be realized by the external program is temporarily designated as Load, which realizes the code of data loading
Cache dependencies must be unidirectional when configured to avoid an infinite loop (you can write a program to check the configuration)
In order to deal with dynamic suffix cache, we can judge the association of cache intelligently by parameter control. For example, when a cache named tablename_id is loaded, the id will be truncated and passed to the operator.
For frequently updated data, such as page click count, etc., if cache is needed, access and update should be independently, so as to avoid synchronous clearing during update
Set is used to enable or disable Load, Delete Relevance Cache, and so on