One, foreword

In project development, local cache is essential to improve system performance and reduce IO overhead. The most common local caches are Guava and Caffeine. This article introduces Caffeine.

Caffeine is based on Google’s Guava Cache design experience and is more efficient in terms of performance and hit ratio than Guava, which you can say is The Guava Plus.

It goes without saying that you should migrate your local cache from Guava to Caffeine as soon as possible. In this article, we will focus on comparing the performance of Guava with that of Caffeine, providing best practices for local cache, and migration strategies.

Second, PK Guava

2.1 features

In terms of functionality, Guava is quite complete and meets most of the local cache requirements. Caffine has added some extensions to what Guava already has.

2.2 performance

In Guava, the read and write operations are mixed with expiration time processing, which means that you may do a write-out operation during a put operation, so the read and write performance is affected.

Caffeine beats Guava in read and write operations, primarily because Caffeine operates on these events asynchronously, submitting them to a queue (using Disruptor RingBuffer), The default ForkJoinPool.commonPool(), or a self-configured thread pool, is then used to fetch the queue, and subsequent flushing and expiration operations are performed.

The following performance comparisons are from Caffeine:

(1) In this benchmark, eight threads read concurrently from the maximum size cache configured:

(2) In this benchmark, from the cache configured with the maximum size, 6 threads concurrently read and 2 threads concurrently write:

image.png

(3) In this benchmark, eight threads write concurrently from the maximum size cache configured:

image.png

2.3 percentage

The purpose of the cache obsolescence strategy is to predict which data is most likely to be used again in the short term, thereby increasing the cache hit ratio. Guava uses the least recently used algorithm of S-LRU segmentation, while Caffeine uses w-TinylFU, an algorithm combining the advantages of LRU and LFU, which is characterized by high hit ratio and low memory consumption.

2.3.1 LRU

Least Recently Used: Data is more likely to be accessed in the future if it has been accessed Recently. Each access puts this element at the head of the queue, and when the queue is full, the data at the end of the queue is eliminated, i.e., the data that has not been accessed for the longest time.

The need to maintain the access frequency information of each data item, each access needs to be updated, which is very expensive.

The disadvantage is that if a large amount of data arrives at any one time, it is easy to push hot data out of the cache, leaving behind data that is likely to be accessed only once and never again or very infrequently. For example, a sudden increase in the number of visits to takeout at noon, and a celebrity’s embarrassment on Weibo are sudden hot events. When the event is over, there may be no traffic, but due to its high frequency of visits, it will not be phased out for a long time to come.

2.3.2 LFU

Most Frequently Used: If the data has been accessed recently, it is more likely to be accessed in the future. That is, the data that has been accessed least often in a given period of time is eliminated (the principle of time locality).

If you need a Queue to hold access records, you can simply implement a cache based on the LRU algorithm using the LinkedHashMap.

The advantage is that it avoids the disadvantages of LRU, because there is no mass incoming squeezing out the old according to frequency obsolescence, and LFU can provide excellent hit ratio if the pattern of data access does not change over time.

The disadvantage is that the occasional and periodic batch operation will lead to a sharp drop in LRU hit ratio and serious cache contamination.

2.3.3 TinyLFU

TinyLFU, as its name suggests, is lightweight LFU and uses a smaller memory space to record access frequency than LFU algorithms.

TinyLFU maintains the frequency of recent access logs, unlike traditional LFU, which maintains access logs for the entire lifecycle, so it is well positioned to deal with sudden hot events (after a certain amount of time, the logs are no longer maintained). These access records act as a filter and are replaced only when the incoming New Item is accessed more frequently than the incoming Cache Victim. The process is as follows:

tiny-lfu-arch

Although it maintains recent access records, it is still very expensive. TinyLFU records frequency information through the count-min Sketch algorithm, which takes up small space and has a low false positive rate. About the count-min Sketch algorithm, please refer to the paper: pproximating Data with the Count-Min Data Structure

2.3.4 W – TinyLFU

W-tinylfu is a new algorithm proposed by Caffeine, which can solve the problem of inaccurate frequency statistics and attenuated access frequency. This method allows us to balance space, efficiency, and the error rate of hash collisions due to the length and width of the appropriate proof.

The following figure shows the hit ratio of various algorithms in a database service running ERP application. The experimental data comes from the author of ARC algorithm. For more performance tests in scenarios, please refer to the official website:

database

W-tinylfu algorithm is an optimization of TinyLFU algorithm, which can solve some sparse burst access elements well. TinyLFU will not be able to preserve these elements in a small number of scenarios with high bursts of traffic because they cannot accumulate at a high enough frequency in a short period of time to be filtered out. W-tinylfu temporarily puts the new record into the Window Cache. Only TinLFU inspection can enter the Main Cache. The general process is as follows:

W-TinyLFU

Best practices

3.1 practice 1

Configuration mode: Set maxSize and refreshAfterWrite, not expireAfterWrite

Problem: If the get cache interval exceeds refreshAfterWrite, the cache is refreshed asynchronously and the old values in the cache are obtained

Applicable scenarios:

  • A large amount of cache data limits the memory capacity occupied by the cache
  • The cache value changes and the cache needs to be flushed
  • It is acceptable to have old data in the cache any time

Set maxSize and refreshAfterWrite. Do not set expireAfterWrite

3.2 practice 2

Configuration mode: Set maxSize and expireAfterWrite. Do not set refreshAfterWrite

Problems: If the get cache interval exceeds expireAfterWrite, the thread that has obtained the lock will load the key synchronously, and other threads that have not obtained the lock will block and wait. If the execution delay of the thread that has obtained the lock is too long, other threads will block for a long time

Applicable scenarios:

  • A large amount of cache data limits the memory capacity occupied by the cache
  • The cache value changes and the cache needs to be flushed
  • Old data in the cache is not acceptable
  • Low synchronous loading data latency (using Redis etc.)

Set maxSize and expireAfterWrite. Do not set refreshAfterWrite

3.3 practice 3

Configuration mode: Set maxSize without refreshAfterWrite or expireAfterWrite. Data is asynchronously refreshed for scheduled tasks

Problem: The cache needs to be refreshed asynchronously by manually scheduled tasks

Applicable scenarios:

  • A large amount of cache data limits the memory capacity occupied by the cache
  • The cache value changes and the cache needs to be flushed
  • Old data in the cache is not acceptable
  • Synchronous loading of data can be very slow

g

Set maxSize without refreshAfterWrite or expireAfterWrite. Data is asynchronously refreshed for scheduled tasks

3.4 practice 4

Configuration mode: Set maxSize, refreshAfterWrite, expireAfterWrite, refreshAfterWrite < expireAfterWrite

Existing problems:

  • The get cache interval is between refreshAfterWrite and expireAfterWrite. The cache is refreshed asynchronously and the old values in the cache are obtained
  • The get cache interval is larger than expireAfterWrite. For this key, the thread that has obtained the lock synchronously executes the load, and other threads that have not obtained the lock block and wait. If the execution delay of the thread that has obtained the lock is too long, other threads block for a long time

Applicable scenarios:

  • A large amount of cache data limits the memory capacity occupied by the cache
  • The cache value changes and the cache needs to be flushed
  • It is acceptable to have old data in a finite time cache
  • Low synchronous loading data latency (using Redis etc.)

Set maxSize, refreshAfterWrite, and expireAfterWrite

4. Migration Guide

4.1 Switch to Caffeine

Introduce Caffeine dependency in POM files:

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
</dependency>
Copy the code

Caffeine is compatible with the Guava API and can be switched from Guava to Caffeine by changing cacheBuilder.newBuilder () to Caffeine.newBuilder().

4.2 Get the Exception

Note that when using Guava’s get() method, an ExecutionException is thrown when the cached load() method returns NULL. After switching to Caffeine, the get() method does not throw an exception, but allows a null return.

Guava also provides a getUnchecked () method, it does not need to show our to catch exceptions, but once the load () method returns null, will be thrown UncheckedExecutionException. After switching to Caffeine, the getUnchecked() method is no longer available, so it needs to be checked out.

Three things to watch ❤️

If you find this article helpful, I’d like to invite you to do three small favors for me:

  1. Like, forward, have your “like and comment”, is the motivation of my creation.

  2. Follow the public account “Java rotten pigskin” and share original knowledge from time to time.

  3. Also look forward to the follow-up article ing🚀

  4. [666] Scan the code to obtain the learning materials package