In-depth analysis of SpringMVC core principle: from the handwritten simple version of the MVC framework (SmartMvc) : github.com/silently952…

IDEA multithreaded file download plugin: github.com/silently952…

Public number: Beta learning JAVA

Abstract

In the last article on caching (part 1), we mainly focused on how to do caching optimization around Http. There are also many places to do caching in the application layer of the back-end server to improve the efficiency of the service. In this article, we’ll continue talking about application-level caching.

Cache hit ratio

Cache hit ratio refers to the ratio of the number of times data is retrieved from the cache to the total number of reads. The higher the hit ratio, the better the cache effect. This is an important metric to monitor to see if our cache is properly set up.

Cache reclamation policy

Based on the time

  • Duration: When setting the cache, set how long the cache can live. No matter how many times it is accessed during the lifetime, the cache will expire
  • Idle period: Refers to how long cached data expires without being accessed

Based on space

Set the cache storage space. For example, set the cache space to 1 GB. When the cache space reaches 1 GB, some data will be removed according to certain policies

Based on number of caches

Set the maximum number of entries in the cache. When the maximum number of entries is reached, old data is removed according to a certain policy

Based on Java object references

  • Weak references: When the garbage collector begins to reclaim memory, if a weak reference is found, it is immediately reclaimed.
  • Soft references: When the garbage collector finds that memory is running low, it reclaims soft-referenced objects to free up space and prevent memory overflow. Soft references are good for heap caching

Cache reclamation algorithm

  • FIFO algorithm
  • LRU least recently used algorithm
  • LFU least commonly used algorithm

The type of Java cache

Pile of cache

Heap cache is the process of caching data in the JVM’s heap memory. The advantage of using heap cache is that there are no serialization and deserialization operations. It is the fastest cache. If the amount of cached data is very large, to avoid the OOM usually uses soft reference to store cached objects; The disadvantages of heap caching are that the cache space is limited and the garbage collector pauses for longer periods.

The Gauva Cache implements heap caching

Cache<String, String> cache = CacheBuilder.newBuilder()
                .build();
Copy the code

Build cache objects with CacheBuilder

The main configuration and methods of Gauva Cache

  • put: Sets key-value to the cache
  • V get(K key, Callable<? extends V> loader)Get a cached value. If not, call loader to get one and put it in the cache
  • expireAfterWrite: Sets the lifetime of the cache. The cache expires after a specified period of time
  • expireAfterAccess: Sets the idle period of the cache. If it is not accessed in a given period of time, it will be reclaimed
  • maximumSize: Sets the maximum number of entries cached
  • weakKeys/weakValues: Sets the weak reference cache
  • softValues: Sets the soft reference cache
  • invalidate/invalidateAll: Actively invalidates the cache data of the specified key
  • recordStats: Enables record statistics to view the hit ratio
  • removalListener: This listener is called when the cache is deleted. It can be used to check why the cache was deleted

Caffeine implements heap caching

Caffeine is a rewritten version of the Guava cache using Java8, a high-performance Java native cache component, as well as an implementation of Spring’s recommended heap cache. Integration with Spring can be found in the documentation docs. .

Since this is a rewrite of the Guava cache, many of the configuration parameters are consistent with the Guava cache:

  • initialCapacity: Indicates the initial cache space size
  • maximumSize: Indicates the maximum number of cache entries
  • maximumWeight: Indicates the maximum weight of the cache
  • expireAfterAccess: Expires at a fixed time after the last write or access
  • expireAfterWrite: Expires at a fixed time after the last write
  • expireAfter: User-defined expiration policy
  • refreshAfterWrite: Refreshes the cache at a fixed interval after the cache is created or updated
  • weakKeys: Enables the weak reference of key
  • weakValues: Opens a weak reference to value
  • softValues: Opens the soft reference of value
  • recordStats: Enables the statistics function

Caffeine’s official documentation: github.com/ben-manes/c…

  1. Add dependencies to pom.xml
< the dependency > < groupId > com. Making. Ben - manes. Caffeine < / groupId > < artifactId > caffeine < / artifactId > < version > 2.8.4 < / version >  </dependency>Copy the code
  1. Caffeine Cache provides three Cache populating strategies: manual, synchronous, and asynchronous loading.
  • Manual loading: specify a synchronous function for each get key and call this function to generate a value if the key does not exist
public Object manual(String key) { Cache<String, Object> cache = Caffeine.newBuilder() .expireAfterAccess(1, Timeunit.seconds) // Set the idle period. MaximumSize (10).build(); return cache.get(key, t -> setValue(key).apply(key)); } public Function<String, Object> setValue(String key){ return t -> "https://silently9527.cn"; }Copy the code
  • Synchronous loading: When constructing a Cache, the build method passes in a CacheLoader implementation class. Implement the load method to load the value through the key.
public Object sync(String key){ LoadingCache<String, Object> cache = Caffeine.newBuilder() .maximumSize(100) .expireAfterWrite(1, Timeunit.minutes) // Set the storage duration. Build (k -> setValue(key).apply(key)); return cache.get(key); } public Function<String, Object> setValue(String key){ return t -> "https://silently9527.cn"; }Copy the code
  • Asynchronous loading: AsyncLoadingCache inherits from the LoadingCache class. Asynchronous loading uses Executor to call a method and return a CompletableFuture
public CompletableFuture async(String key) { AsyncLoadingCache<String, Object> cache = Caffeine.newBuilder() .maximumSize(100) .expireAfterWrite(1, TimeUnit.MINUTES) .buildAsync(k -> setAsyncValue().get()); return cache.get(key); } public CompletableFuture < Object > setAsyncValue () {return CompletableFuture. SupplyAsync (() - > "number: public beta learn JAVA"); }Copy the code
  1. Listen for events in which the cache is cleared
public void removeListener() {
    Cache<String, Object> cache = Caffeine.newBuilder()
            .removalListener((String key, Object value, RemovalCause cause) -> {
                System.out.println("remove lisitener");
                System.out.println("remove Key:" + key);
                System.out.println("remove Value:" + value);
            })
            .build();
    cache.put("name", "silently9527");
    cache.invalidate("name");
}
Copy the code
  1. statistical
public void recordStats() { Cache<String, Object> cache = Caffeine.newBuilder() .maximumSize(10000) .recordStats() .build(); Cache. Put (" public number ", "beta learning JAVA"); Cache.get (" public id ", (t) -> ""); cache.get("name", (t) -> "silently9527"); CacheStats stats = cache.stats(); System.out.println(stats); }Copy the code

CacheStats is obtained via cache.stats (). CacheStats provides the following statistical methods:

  • hitRate(): Returns the cache hit ratio
  • evictionCount(): Number of cache reclaims
  • averageLoadPenalty(): The average time to load a new value

EhCache implements heap caching

EhCache is an old Java open source cache framework, which appeared as early as 2003 and has been very mature and stable until now. It is widely used in The field of Java applications and can be well integrated with mainstream Java frameworks such as Srping. Compared to Guava Cache, EnCache supports more features, including off-heap Cache, disk Cache, and of course is a bit heavier to use. Maven dependencies using Ehcache are as follows:

<dependency> <groupId>org.ehcache</groupId> <artifactId> Ehcache </artifactId> <version>3.6.3</version> </dependency>Copy the code
CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder().build(true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.heap(10); CacheConfiguration<String, String> cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource) .withExpiry(ExpiryPolicyBuilder.timeToIdleExpiration(Duration.ofMinutes(10))) .build(); Cache<String, String> cache = cacheManager.createCache("userInfo", cacheConfig);Copy the code
  • ResourcePoolsBuilder.heap(10)Set the maximum number of entries in the cache, which is shorthand, equivalent toResourcePoolsBuilder.newResourcePoolsBuilder().heap(10, EntryUnit.ENTRIES);
  • ResourcePoolsBuilder.newResourcePoolsBuilder().heap(10, MemoryUnit.MB)Set the maximum cache space to 10MB
  • withExpiry(ExpiryPolicyBuilder.timeToIdleExpiration(Duration.ofMinutes(10)))Set the cache idle time
  • withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofMinutes(10)))Set the cache lifetime
  • remove/removeAllActive invalidation caches, similar to Guava caches, do not clear the collection immediately after a call to a method, but only determine whether the Cache expires on get or PUT
  • withSizeOfMaxObjectSize(10,MemoryUnit.KB)Limit the size of a single cache object. Objects exceeding these two limits are not cached

Out of the cache

Out-of-heap cache means that cached data is stored in out-of-heap memory and the size of the cache is only limited by the size of the native memory. It is not managed by GC. Using out-of-heap cache can reduce GC pause time, but objects in out-of-heap memory need to be serialized and deserialized. In Java, you can set the upper limit of off-heap memory with the -xx :MaxDirectMemorySize parameter

CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder().build(true); // Out-of-heap memory cannot be limited by the items stored, only by the size of memory, More than limit, recycling cache ResourcePoolsBuilder resource = ResourcePoolsBuilder. NewResourcePoolsBuilder (.) offheap (10, MemoryUnit. MB); CacheConfiguration<String, String> cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource) .withDispatcherConcurrency(4) .withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofMinutes(10))) .withSizeOfMaxObjectSize(10, MemoryUnit.KB) .build(); Cache<String, String> cache = cacheManager.createCache("userInfo2", cacheConfig); cache.put("website", "https://silently9527.cn"); System.out.println(cache.get("website"));Copy the code

Disk cache

The cached data is stored on disk, the cached data is not affected when the JVM restarts, and both the heap and off-heap caches are lost. And the disk cache has more storage space; However, data cached on disk also needs to support serialization, which is slower than memory. It is recommended to use faster disks for greater throughput, such as flash memory instead of mechanical disks.

CacheManagerConfiguration<PersistentCacheManager> persistentManagerConfig = CacheManagerBuilder .persistence(new File("/Users/huaan9527/Desktop", "ehcache-cache")); PersistentCacheManager persistentCacheManager = CacheManagerBuilder.newCacheManagerBuilder() .with(persistentManagerConfig).build(true); //disk If the third parameter is set to true, data is persisted to the disk ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder().disk(100, MemoryUnit.MB, true); CacheConfiguration<String, String> config = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource).build(); Cache<String, String> cache = persistentCacheManager.createCache("userInfo", CacheConfigurationBuilder.newCacheConfigurationBuilder(config)); Cache. Put (" public number ", "beta learning JAVA"); System.out.println(cache.get(" public id ")); persistentCacheManager.close();Copy the code

The JVM stops, must remember call persistentCacheManager. Close (), can ensure that data in memory dump to disk.

This is a typical heap + offheap + disk structure. The upper layer is faster than the lower layer, and the lower layer has more storage space than the upper layer. In EhCache, the space size is setheap > offheap > diskOtherwise, an error will be reported. Ehcache stores the hottest data in a higher-level cache. The code for this structure is as follows:

CacheManagerConfiguration<PersistentCacheManager> persistentManagerConfig = CacheManagerBuilder .persistence(new File("/Users/huaan9527/Desktop", "ehcache-cache")); PersistentCacheManager persistentCacheManager = CacheManagerBuilder.newCacheManagerBuilder() .with(persistentManagerConfig).build(true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder() .heap(10, MemoryUnit.MB) .offheap(100, Memoryunit.mb) // Set the third parameter to true to support persistence.disk (500, memoryUnit.mb, true); CacheConfiguration<String, String> config = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource).build(); Cache<String, String> cache = persistentCacheManager.createCache("userInfo", CacheConfigurationBuilder.newCacheConfigurationBuilder(config)); // Write cache cache.put("name", "silently9527"); System.out.println(cache.get("name")); / / program before closing again, need to manually release resources persistentCacheManager. Close ();Copy the code

Distributed centralized cache

The in-heap and out-of-heap caches mentioned earlier have two problems if multiple JVM instances exist: 1. Capacity is limited; 2. Data cached by multiple JVM instances may be inconsistent; 3. If all cached data is invalid at the same time, all requests will be sent to the database, which increases the pressure on the database. At this time, we need to introduce distributed cache to solve the problem. The most used distributed cache is Redis

The application cache architecture can be adjusted to this structure when distributed caching is introduced.

Practice of caching usage patterns

There are two types of caching: cache-aside and cache-as-SOR (SoR refers to the actual system storing the data, that is, the data source)

Cache-Aside

Business code is written around the cache. It usually gets data from the cache. If the cache doesn’t hit, it looks up the data from the database and puts it into the cache. When the data is updated, the corresponding data in the cache needs to be updated. This pattern is the one we use most often.

  • Read the scenario
value = cache.get(key); If (value == null) {value = loadFromDatabase(key); // Query cache from database. Put (key, value); // put it in the cache}Copy the code
  • Write scenarios
wirteToDatabase(key, value); // Write to database cache. Put (key, value); Cache. remove(key), and check again before readingCopy the code

In order to separate the reading and updating of the business code from the Cache, Spring uses AOP to encapsulate the cache-aside mode, and provides multiple annotations to realize the reading and writing scenarios. Official Reference Documents:

  • @Cacheable: is usually placed in the query method, implementation isCache-AsideRead scenario, first check the cache, if not in the query database, finally put the query results into the cache.
  • @CachePutThis is usually used to save update methodsCache-AsideWrite scenario where data is put into the cache after updating the database.
  • @CacheEvict: Deletes the cache for the specified key from the cache

For basic data that allows a slight update delay, consider using the Canal subscription binlog to perform incremental updates to the cache.

There is also a problem with cache-aside. If the Cache fails at some point, there will be a lot of requests to the back-end database at the same time, and the pressure on the database will suddenly increase

Cache-As-SoR

The cache-as-SOR mode treats the Cache As the data source, delegating all operations to the Cache and delegating reads and writes to the real SoR. Only Cache operations are seen in business code, which is divided into three modes

Read Through

The application always requests data from the cache, and if there is no data in the cache, it is responsible for retrieving the data from the database using the supplied data loader, after which the cache updates itself and returns the data to the calling application. This mode is supported by Gauva Cache, Caffeine, and EhCache.

  1. Caffeineๅฎž็ŽฐRead Through

Since Gauva Cache is similar to Caffeine’s implementation, only the implementation of Caffeine is shown here. The code below is from the Caffeine official documentation

LoadingCache<Key, Graph> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

// Lookup and compute an entry if absent, or null if not computable
Graph graph = cache.get(key);
// Lookup and compute entries that are absent
Map<Key, Graph> graphs = cache.getAll(keys);
Copy the code

Specify a Cache reader when building a Cache

  • [1] directly called in the applicationcache.get(key)
  • [2] First query the cache and return the data directly if the cache exists
  • [3] If it does not exist, it will delegate toCacheLoaderGo to the data source and query the data, then put it in the cache and return it to the application

You are advised to encapsulate your own null object into the cache to prevent cache breakdown

In order to prevent the backend database from being overloaded due to a hotspot data failure, I can use the lock limit in CacheLoader to allow only one request to query the database, and all other requests to wait for the first request to complete the query from the cache. In our last swastika on Caching (Part 1), we talked about Nginx having similar configuration parameters

value = loadFromCache(key); if(value ! = null) { return value; } synchronized (lock) { value = loadFromCache(key); if(value ! = null) { return value; } return loadFromDatabase(key); }Copy the code
  1. EhCache implements Read Through
CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder().build(true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder().heap(10, MemoryUnit.MB); CacheConfiguration<String, String> cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource) .withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofMinutes(10))) .withLoaderWriter(new CacheLoaderWriter<String, String>(){ @Override public String load(String key) throws Exception { //load from database return "silently9527"; } @Override public void write(String key, String value) throws Exception { } @Override public void delete(String key) throws Exception { } }) .build(); Cache<String, String> cache = cacheManager.createCache("userInfo", cacheConfig); System.out.println(cache.get("name"));Copy the code

EhCache uses a cache writer to load data from the database. Load can also be used to solve the problem of the backend database pressure caused by a hotspot data failure.

Write Through

Similar to the Read Through mode, when data is updated, SoR is updated first and cache is updated after success.

  1. Caffeine implements Write Through
Cache<String, String> cache = Caffeine.newBuilder()
        .maximumSize(100)
        .writer(new CacheWriter<String, String>() {
            @Override
            public void write(@NonNull String key, @NonNull String value) {
                //write data to database
                System.out.println(key);
                System.out.println(value);
            }

            @Override
            public void delete(@NonNull String key, @Nullable String value, @NonNull RemovalCause removalCause) {
                //delete from database
            }
        })
        .build();

cache.put("name", "silently9527");
Copy the code

Caffeine implements Write Through using the CacheWriter, which synchronously listens for cache creation, changes, and deletions, and updates the cache only when a Write is successful

  1. EhCache implements Write Through
CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder().build(true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder().heap(10, MemoryUnit.MB); CacheConfiguration<String, String> cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource) .withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofMinutes(10))) .withLoaderWriter(new CacheLoaderWriter<String, String>(){ @Override public String load(String key) throws Exception { return "silently9527"; } @Override public void write(String key, String value) throws Exception { //write data to database System.out.println(key); System.out.println(value); } @Override public void delete(String key) throws Exception { //delete from database } }) .build(); Cache<String, String> cache = cacheManager.createCache("userInfo", cacheConfig); System.out.println(cache.get("name")); cache.put("website","https://silently9527.cn");Copy the code

When we call cache.put(” XXX “,” XXX “) to write to the cache, EhCache will delegate the write operation to the CacheLoaderWriter. Cacheloaderwriter.write is responsible for writing data sources

Write Behind

This pattern typically writes data to the cache before asynchronously writing to the database for data synchronization. This design can not only reduce the direct access to the database, reduce the pressure, at the same time, the database can be modified multiple times to merge operations, greatly improve the system carrying capacity. However, this model also has risks, such as the possibility of data loss when the cache machine goes down.

  1. Caffeine can implement Write Behind atCacheLoaderWriter.writeMethod to send data to MQ for asynchronous consumption, which can keep the data secure, but to implement merge operations need to be extended to more powerfulCacheLoaderWriter.
  2. EhCache implements Write Behind
/ / 1 defines a thread pool PooledExecutionServiceConfiguration testWriteBehind = PooledExecutionServiceConfigurationBuilder .newPooledExecutionServiceConfigurationBuilder() .pool("testWriteBehind", 5, 10) .build(); CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder() .using(testWriteBehind) .build(true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder().heap(10, MemoryUnit.MB); / / set the maximum number of cached items / / 2 set back the pattern configuration WriteBehindConfiguration testWriteBehindConfig = WriteBehindConfigurationBuilder .newUnBatchedWriteBehindConfiguration() .queueSize(10) .concurrencyLevel(2) .useThreadPool("testWriteBehind") .build(); CacheConfiguration<String, String> cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder(String.class, String.class, resource) .withLoaderWriter(new CacheLoaderWriter<String, String>() { @Override public String load(String key) throws Exception { return "silently9527"; } @Override public void write(String key, String value) throws Exception { //write data to database } @Override public void delete(String key) throws Exception { } }) .add(testWriteBehindConfig) .build(); Cache<String, String> cache = cacheManager.createCache("userInfo", cacheConfig);Copy the code

The first to use PooledExecutionServiceConfigurationBuilder defines the thread pool configuration; Then use WriteBehindConfigurationBuilder setting will write mode configuration, including newUnBatchedWriteBehindConfiguration said without batch write operations, because are written asynchronously, so need to write into the first in the queue, QueueSize sets the queueSize, useThreadPool specifies which thread pool to use; ConcurrencyLevel Sets the number of concurrent threads and queues to use for Write Behind

EhCache also makes it easy to batch write

  • First turn on thenewUnBatchedWriteBehindConfiguration()replacenewBatchedWriteBehindConfiguration(10, TimeUnit.SECONDS, 20)If the number of batches reaches 20, the batch will be processed if the number of batches does not reach 20 within 10 seconds
  • Second inCacheLoaderWriterWirteAll and deleteAll are implemented in batch processing

If you need to combine operations on the same key to record only the last time, you can enable the merging by enableCoalescing()

To the last point of attention, do not get lost

There may be more or less deficiencies and mistakes in the article, suggestions or opinions are welcome to comment and exchange.

Finally, white piao is not good, creation is not easy, I hope friends can like the comments pay attention to three even, because these are all the power source I share ๐Ÿ™

Source code address: github.com/silently952…

Public number: Beta learning JAVA

๐Ÿ† technology project issue 8 chat | magical function and problems of cache