Spring Cache, from beginner to delicious

Drought cavity be out of tune

This is two days ago WHEN I went to visit the West Lake, play to the sunset time. So I sat on the edge of the West Lake, quietly watching the sun set, waiting for the sky to turn black, waiting for the lights to light up.

Personally, I prefer places with water, especially lakes. I was in Hangzhou once a few years ago, and I also walked through the West Lake once. I didn’t graduate at that time, and I came here to see a job opportunity. The day of the west Lake under a light rain, I a person along the bai Dike, around the West Lake to go a circle, misty rain hazy, like a landscape painting scroll.

I did not expect that a few years later, I will come to this city again, working and living in Hangzhou, The West Lake is still the west Lake, I am still that I, but this time, become two people to see the West Lake.

I didn’t travel or go home on National Day. I spent the holiday in a relaxed rhythm by playing around for a day and resting for a day. Both body and mind were very relaxed, but also figured out some puzzling things for a long time, on the whole, it was not wasted.

Today’s article is a bit more “tutorial” article. But also from shallow to deep, careful analysis of the source code, and introduced some common problems and solutions in the use of Spring Cache, is certainly more in-depth than a simple introduction to the document, I believe you will have some harvest after reading.

Why cache

The other day IN my article “How I Took a 15-minute Program and Made it 10 seconds,” I talked about some ways to optimize performance at the code level. The first of these is the use of caching.

Using caching is a cost-effective way to optimize performance, especially for applications with a large number of repeated queries. Typically, in a WEB back-end application, there are two areas that are time-consuming: database lookup and API calls to other services (because other services end up doing time-consuming operations like database lookup as well).

There are also two types of repeated queries. One is that we don’t code very well in our applications, and we write a for loop that repeats the query each time. In this case, a more intelligent programmer would refactor this code, using a Map to temporarily store the found items in memory, and then check the Map before going to the database. In fact, this is a kind of caching idea.

Another type of repeated query is the result of a large number of identical or similar requests. For example, the list of articles on the home page of information websites, the list of goods on the home page of e-commerce websites, the articles searched by social media such as Weibo and so on. When a large number of users request the same interface and the same data, if they check the database every time, it is an unbearable pressure for the database. So we usually cache high-frequency queries, which we call “hot”.

Why use Spring Cache

As mentioned above, there are many benefits to caching, so everyone is ready to add caching capabilities to their applications. However, a search on the web reveals that there are so many caching frameworks, each with its own advantages, such as Redis, Memcached, Guava, Caffeine, etc.

If our program wants to use caching, it needs to be coupled to these frameworks. Smart architects are already using interfaces to reduce coupling, taking advantage of object-oriented abstractions and polymorphisms to decouple business code from concrete frameworks.

But we still need to explicitly call cache-specific interfaces and methods in our code, insert data into the cache when appropriate, and read data from the cache when appropriate.

Think about the application scenarios of AOP. Isn’t that what AOP was born to do?

For Spring AOP, please refer to my previous article “Ten Soul Questions about Spring AOP”.

Yes, Spring Cache is one such framework. It uses AOP to realize the caching function based on annotations, and carries on the reasonable abstraction, the business code does not care what caching framework is used at the bottom, just need to add a simple annotation, can realize the caching function. Spring Cache also provides a lot of default configurations, and users can use a nice Cache in three seconds.

If you have such good wheels, why not use them?

How to use Spring Cache

The three seconds above are no exaggeration. There are three simple steps to using SpringCache: add dependencies, enable caching, and add caching annotations.

Git source code: github.com/spring-guid…

1 and rely on

Gradle:

implementation 'org.springframework.boot:spring-boot-starter-cache'
Copy the code

maven:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>
Copy the code

2 Enabling cache

To enable the use of caching, add the @enablecaching annotation to the startup class.

@SpringBootApplication
@EnableCaching
public class CachingApplication {

	public static void main(String[] args) { SpringApplication.run(CachingApplication.class, args); }}Copy the code

Add cache annotations

You can cache the return value of a method by annotating it with @cacheable.

@Override
@Cacheable("books")
public Book getByIsbn(String isbn) {
    simulateSlowService();
    return new Book(isbn, "Some book");
}

// Don't do this at home
private void simulateSlowService(a) {
    try {
        long time = 3000L;
        Thread.sleep(time);
    } catch (InterruptedException e) {
        throw newIllegalStateException(e); }}Copy the code

test

@Override
public void run(String... args) {
    logger.info("... Fetching books");
    logger.info("isbn-1234 -->" + bookRepository.getByIsbn("isbn-1234"));
    logger.info("isbn-4567 -->" + bookRepository.getByIsbn("isbn-4567"));
    logger.info("isbn-1234 -->" + bookRepository.getByIsbn("isbn-1234"));
    logger.info("isbn-4567 -->" + bookRepository.getByIsbn("isbn-4567"));
    logger.info("isbn-1234 -->" + bookRepository.getByIsbn("isbn-1234"));
    logger.info("isbn-1234 -->" + bookRepository.getByIsbn("isbn-1234"));
}
Copy the code

If you test it, you can see. The first and second calls to getByIsbn (with a different argument than the first) wait three seconds, while the next four calls all return immediately.

Commonly used annotations

Spring Cache has several common annotations: @cacheable, @cacheput, @cacheevict, @Caching, and @Cacheconfig. All but the last one, CacheConfig, can be used on a class or method level. If used on a class, it applies to all public methods of that class.

@Cacheable

The @cacheble annotation indicates that the method is cached. The return value of the method is cached. The next time the method is called, it checks to see if it already has a value in the cache. If not, the method is called and the result is cached. This annotation is typically used on query methods.

@CachePut

A method annotated with @cacheput puts the return value of the method in a cache for use elsewhere. It is usually used for new methods.

@CacheEvict

Methods using the CacheEvict annotation empty the specified cache. Usually used to update or delete.

@Caching

The Java annotation mechanism dictates that only one identical annotation can be valid on a method. It is sometimes possible for a method to operate on multiple caches (this is more common for delete caches and less common for add).

Spring Cache also takes this into account, and the @Caching annotation is designed to address this, as you can see from its source code.

public @interface Caching {
	Cacheable[] cacheable() default {};
	CachePut[] put() default {};
	CacheEvict[] evict() default {};
}
Copy the code

@CacheConfig

The four annotations mentioned above are common Spring Cache annotations. Each annotation has a number of properties that can be configured, which we’ll explain in more detail in the next section. However, these annotations are usually applied to methods, and some configurations may be class-specific. In this case, you can use @CacheconFig, which is a class-level annotation. You can configure cacheNames, keyGenerator, cacheManager, and cacheResolver at the class level.

Configuration of common annotations

This part is best viewed in conjunction with the source code to better understand how these configurations work.

Source code: parsing annotation time

This section is mainly source code analysis, a little obscure, the source code is not interested in students can skip. But if you want to understand how Spring Cache works, it’s recommended.

The annotations @cacheable, @cacheput, @cacheevict, and @Cacheconfig mentioned earlier all have configurable properties. These configured properties can be found in the abstract class CacheOperation and its subclasses. They go something like this:

See here have to admire, this inheritance use, wonderful ah.

Parse each code can be found in SpringCacheAnnotationParser class notes, such as parseEvictAnnotation method, in that,

builder.setCacheWide(cacheEvict.allEntries());
Copy the code

Is it “allEntries” in the annotations, but “CacheEvictOperation” is “cacheWide”? I look at the authors, there are multiple authors, but the first author is a buddy named Costin Leau, I’m still a little confused about the name… It seems that the big guys write code will also have the problem of naming inconsistency

When this SpringCacheAnnotationParser is called? It’s very simple, we can just make a breakpoint on a method of this class, and debug it, like the parseCacheableAnnotation method.

In the debug interface, can see the chain is very long, is familiar to us in front of the IOC Bean registration a process, until we see a called AbstractAutowireCapableBeanFactory the BeanFactory, The class then looks for advisors when creating the Bean. Just the Spring Cache in the source code defines such an Advisor: BeanFactoryCacheOperationSourceAdvisor.

The Advisor returns PointCut is a CacheOperationSourcePointcut, the PointCut autotype matches method, inside to get a CacheOperationSource, Call its getCacheOperations method. The CacheOperationSource is an interface, the main implementation class is AnnotationCacheOperationSource. In findCacheOperations method is invoked to our initial SpringCacheAnnotationParser said.

This completes annotation-based parsing.

Entry: AOP-based interceptor

So what happens when we actually call the method? As we know, beans that use AOP generate a proxy object that, when actually invoked, executes a series of interceptors from this proxy object. Spring Cache uses an interceptor called CacheInterceptor. If we add the corresponding annotation to the cache, we will go to the interceptor. This interceptor inherits from the CacheAspectSupport class and executes its execute method, which is the core method we’ll examine.

@ Cacheable sync

Let’s move on to the execute method mentioned earlier, which first checks if it’s synchronous. The sync configuration here uses the @cacheable sync attribute, which defaults to false. If synchronization is configured, multiple threads attempting to cache data with the same key will perform a synchronous operation.

Let’s take a look at the source code for the synchronous operation. If it is determined that the current operation (1) needs to be synchronized, it first checks whether the current condition meets the condition (2). Condition is also a configuration defined in @cacheable, which is an EL expression. For example, we can cache books with ids greater than 1 like this:

@Override
@Cacheable(cacheNames = "books", condition = "#id > 1", sync = true)
public Book getById(Long id) {
    return new Book(String.valueOf(id), "some book");
}
Copy the code

If the condition is not met, the cache is not used and the result is not put into the cache and the result is skipped to 5. Otherwise, try to get key (3). When retrieving a key, it checks whether the user has defined the key, which is also an EL expression. If not, use keyGenerator to generate a key:

@Nullable
protected Object generateKey(@Nullable Object result) {
    if (StringUtils.hasText(this.metadata.operation.getKey())) {
        EvaluationContext evaluationContext = createEvaluationContext(result);
        return evaluator.key(this.metadata.operation.getKey(), this.metadata.methodKey, evaluationContext);
    }
    return this.metadata.keyGenerator.generate(this.target, this.metadata.method, this.args);
}
Copy the code

We can manually specify keys like book-1 and book-2 based on id in this way:

@Override
@Cacheable(cacheNames = "books", sync = true, key = "'book-' + #id")
public Book getById(Long id) {
    return new Book(String.valueOf(id), "some book");
}
Copy the code

The key here is an Object Object, and if we do not specify a key on the annotation, the key generated by keyGenerator will be used. The default keyGenerator is SimpleKeyGenerator, which generates a SimpleKey object. The method is also very simple. If there is no input parameter, it returns an EMPTY object. Just use this parameter. (Note that I’m using the parameter itself, not the SimpleKey object. Otherwise, use a SimpleKey package with all input arguments.

Source:

@Override
public Object generate(Object target, Method method, Object... params) {
    return generateKey(params);
}

/** * Generate a key based on the specified parameters. */
public static Object generateKey(Object... params) {
    if (params.length == 0) {
        return SimpleKey.EMPTY;
    }
    if (params.length == 1) {
        Object param = params[0];
        if(param ! =null && !param.getClass().isArray()) {
            returnparam; }}return new SimpleKey(params);
}
Copy the code

There is no difference between the class name and the method name. If two methods have the same input parameter, it is not the same as the key.

You’re right. Here are two things you can try:

// Define a method where both arguments are String
@Override
@Cacheable(cacheNames = "books", sync = true)
public Book getByIsbn(String isbn) {
    simulateSlowService();
    return new Book(isbn, "Some book");
}

@Override
@Cacheable(cacheNames = "books", sync = true)
public String test(String test) {
    return test;
}

// Call both methods with the same argument "test"
logger.info("test getByIsbn -->" + bookRepository.getByIsbn("test"));
logger.info("test test -->" + bookRepository.test("test"));
Copy the code

You’ll notice that the key is the same, and when you call the test method, the console will report an error:

Caused by: java.lang.ClassCastException: class com.example.caching.Book cannot be cast to class java.lang.String (com.example.caching.Book is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
	at com.sun.proxy.$Proxy33.test(Unknown Source) ~[na:na] at com.example.caching.AppRunner.run(AppRunner.java:23) ~[main/:na] at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) ~ [spring - the boot - 2.3.2. RELEASE. The jar: 2.3.2. RELEASE]... 5 common frames omittedCopy the code

Book cannot be forcibly converted to String, because the first time we call getByIsbn, the key generated is test, and then the return value Book object is put into the cache. When the test method is called, the key generated is still test, and Book is fetched, but the test method returns a String, so we try to strong-cast to String, and find that strong-cast fails.

We can customize a keyGenerator to solve this problem:

@Component
public class MyKeyGenerator implements KeyGenerator {
    @Override
    public Object generate(Object target, Method method, Object... params) {
        return target.getClass().getName() + method.getName() + 
                Stream.of(params).map(Object::toString).collect(Collectors.joining(",")); }}Copy the code

You can then use the custom MyKeyGenerator in the configuration and run the program again without the above problems.

@Override
@Cacheable(cacheNames = "books", sync = true, keyGenerator = "myKeyGenerator")
public Book getByIsbn(String isbn) {
    simulateSlowService();
    return new Book(isbn, "Some book");
}

@Override
@Cacheable(cacheNames = "books", sync = true, keyGenerator = "myKeyGenerator")
public String test(String test) {
    return test;
}
Copy the code

Looking down, we can see that we have a Cache. This Cache is a CacheOperationContext that is new when we call the execute method of CacheAspectSupport. In the Context constructor, cacheResolver is used to parse the Cache in the annotations and generate the Cache object.

The default cacheResolver is SimpleCacheResolver, which obtains configured cacheNames from CacheOperation and uses cacheManager to get a Cache. The cacheManager here is used to manage the Cache of a container, the default is ConcurrentMapCacheManager cacheManager. ConcurrentHashMap (ConcurrentHashMap, ConcurrentHashMap)

So what is the Cache here? A Cache is an abstraction of a Cache container. It contains methods such as GET, PUT, EVict, and putIfAbsent that are used by caches.

Different cacheNames will correspond to different Cache objects. For example, you can define two cacheNames in one method. You can also use Value, which is the alias of cacheNames, but if you have multiple configurations, cacheNames is recommended. Because it’s much more readable.

@Override
@Cacheable(cacheNames = {"book", "test"})
public Book getByIsbn(String isbn) {
    simulateSlowService();
    return new Book(isbn, "Some book");
}
Copy the code

The default Cache is ConcurrentMapCache, which is also based on ConcurrentHashMap.

There is a problem, however, as we go back to the execute method code above and see that if sync is set to true, it fetches the first Cache and does not take care of the rest of the Cache. So if you set sync to true, only one cacheNames is supported. If you set multiple cacheNames, an error will be reported:

@Cacheable(sync=true) only allows a single cache on...
Copy the code

Further down, the Cache get(Object, Callcable) method is called. This method will first attempt to cache the value with key, and then call Callable if it doesn’t get it, and then add it to the cache. Spring Cache also expects the implementation class of the Cache to implement “synchronization” within this method.

So going back to the comment above the sync property in Cacheable, it says: Using sync to true has these limitations:

1. Unless is not supported, as you can see from the code, only condition is supported, not unless; Well, I’m not sure why… But that’s how Interceptor code is written.
2. There can only be one cache because the code is written to death. I guess this is to support synchronization better, and it implements synchronization in the Cache.
3. No other Cache operations are not supported, and only Cachable is supported in the code. I guess this is to support synchronization.

Other operating

What if sync is false?

Looking further down the execute code, it goes through the following steps:

1. Try to remove the cache before the method invocation. This is the beforeInvocation in the @cacheevict configuration, which defaults to false (if true will remove the cache at this step);
2. Try to get the cache;
3. If not, try fetching Cachable’s annotation to generate the corresponding CachePutRequest.
4. If you get it in step 2 and there is no CachPut annotation, get the value directly from the cache. Otherwise, the target method is called;
5. Parsing a CachePut annotation to also generate a CachePutRequest;
6. Perform all cacheputrequests;
7. Try removing the cache after the method invocation, if the @cacheevict configuration’s beforeInvocation is false the cache will be removed

At this point, we have explained the timing of all configurations in conjunction with the source code.

Use other caching frameworks

If you want to use other caching frameworks, what should you do?

As we know from the source code analysis above, if we want to use other caching frameworks, we only need to redefine the CacheManager and CacheResolver beans.

In fact, Spring will automatically detect if we have introduced the appropriate caching framework. If we have introduced spring-data-redis, Spring will automatically use the RedisCacheManager provided by Spring-data-Redis.

If we’re going to use Caffeine framework. With Caffeine introduced, Spring Cache uses Caffein Manager and Caffein Ache by default.

implementation 'org.springframework.boot:spring-boot-starter-cache'
implementation 'com.github.ben-manes.caffeine:caffeine'
Copy the code

Caffeine is a very high performance caching framework that uses the Window TinyLfu recycle strategy to provide a near optimal hit ratio.

Spring Cache also supports a variety of configurations, including special configurations for various major caching frameworks in the CacheProperties class. Such as Redis expiration time, etc. (default never expires).

private final Caffeine caffeine = new Caffeine();

private final Couchbase couchbase = new Couchbase();

private final EhCache ehcache = new EhCache();

private final Infinispan infinispan = new Infinispan();

private final JCache jcache = new JCache();

private final Redis redis = new Redis();
Copy the code

Problems with using caching

Double write inconsistent

There are many problems associated with using caches, especially with high concurrency, including cache penetration, cache breakdown, cache avalanche, and double-write inconsistencies. For a detailed description of the problems and common solutions, please refer to the article cache FAQ and Solutions on my personal website.

This is a common problem. A common solution is to delete the cache before updating the database. So the @cacheevict in Spring Cache has a beforeInvocation configuration.

But with caches, there are often inconsistencies between the data in the cache and the database, especially when you call a third-party interface and you don’t know when it updates the data. However, the business scenarios using cache do not require strong consistency of data in many cases. For example, we can disable the cache for one minute for the hot articles on the home page. In this way, it does not matter if the hot articles are not the latest within one minute.

Take up extra memory

This is inevitable. Because there has to be somewhere to put it down. Whether it’s ConcurrentHashMap, Redis or Caffeine, it takes up extra memory resources to slow down storage. But the idea of caching is to trade space for time, and sometimes taking up that extra space is worth the time optimization.

Note that SpringCache uses ConcurrentHashMap by default and does not automatically recycle keys, so if you use the default cache, the program will get bigger and bigger and will not be recycled. May eventually result in OOM.

Let’s simulate the experiment:

@Component
public class MyKeyGenerator implements KeyGenerator {
    @Override
    public Object generate(Object target, Method method, Object... params) {
        // A different key is generated each time
        returnUUID.randomUUID().toString(); }}// Adjust it 100w times
for (int i = 0; i < 1000000; i++) {
    bookRepository.test("test");
}
Copy the code

Then set the maximum memory to 20M: -xmx20m.

Let’s test the default ConcurrentHashMap based cache and see that it will be OOM soon.

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI TCP Connection(idle)"
Copy the code

We use Caffeine and configure its maximum capacity:

spring:
  cache:
    caffeine:
      spec: maximumSize=100
Copy the code

Run the program again and find that it works normally without any error.

So if you use a cache based on the same JVM memory, Caffeine is personally recommended, and the default ConcurrentHashMap based implementation is strongly discouraged.

So what’s a good case for a cache like Redis that requires a call to a third party process? If your application is distributed and one server queries and wants other servers to use the cache, Redis based caches are recommended.

The downside of using Spring Cache is that it hides the underlying Cache features. For example, it is difficult to have different expiration times for different scenarios (but it is not impossible to do this by configuring a different cacheManager). But on the whole, the benefits are greater than the disadvantages, we measure, suitable for their own good.

About the author

I’m Yasin, wechat official account: I made up a program

Personal website: Yasinshaw.com

Pay attention to my public number, grow up with me ~

There are learning resources, technical exchange group and big factory push oh