This is my 10th day of the November Gwen Challenge. The last Gwen Challenge in 2021. The core idea is to document the results of process data reuse operations. When a program needs to perform a complex and resource-consuming operation, we typically store the result of the run in the cache and read it from the cache the next time we need it. Caching works for data that doesn’t change often, or even never changes. Changing data is not a good place to cache. For example, GPS data from an airplane flight should not be cached, otherwise you will get the wrong data.
First, cache type
There are three types of caches:
- In-memory Cache: in-process Cache. The cache terminates when the process terminates.
- Persistent in-process cache: Back up the cache outside of process memory, either in a file, in a database, or elsewhere. If the process restarts, the cache is not lost.
- Distributed cache: Multiple machines share the cache. If one server holds a cache entry, other servers can use it.
Tip: In this article we will only cover in-process caching.
Second, the implementation
Let’s implement in-process caching step by step by caching avatars. The way we implemented caching in earlier versions of.NET was simple:
public class NaiveCache<TItem>
{
Dictionary<object, TItem> _cache = new Dictionary<object, TItem>();
public TItem GetOrCreate(object key, Func<TItem> createItem)
{
if(! _cache.ContainsKey(key)) { _cache[key] = createItem(); }return_cache[key]; }}Copy the codeHere’s how to use it:
var _avatarCache = new NaiveCache<byte[] > ();var myAvatar = _avatarCache.GetOrCreate(userId, () => _database.GetAvatar(userId));
Copy the codeWhen obtaining the user profile picture, only the first request really requests the database. After the request is received, the profile picture data is saved in the process memory, and all subsequent requests for the profile picture are extracted from the memory, saving time and resources. But this solution is not the best for a number of reasons. First, it is not thread-safe, and exceptions can occur when multiple threads use it. In addition, cached data will remain in memory forever, once the memory is cleared by various reasons, the data stored in memory will be lost. The disadvantages of this solution are summarized below:
- The cache occupies a large amount of memory, resulting in out-of-memory exceptions and crashes.
- High memory consumption can lead to memory stress, and garbage collectors can work more than they should to hurt performance;
- If the data changes, the cache needs to be flushed
To solve the above problem, the cache framework must have an expulsion strategy that removes items from the cache based on algorithmic logic. Common deportation policies are as follows:
- Expiration policy: Deletes items from the cache after a specified time;
- If an item is not accessed within a specified period of time, the sliding expiration policy removes the item from the cache. For example, if we set the expiration time to 1 minute, the item will remain in the cache as long as it is used every 30 seconds. But it will be deleted if it is not used for more than a minute.
- Size limiting policy: Limits the cache memory size.
To improve our code based on the strategies described above, we can use the solution provided by Microsoft. Microsoft has two solutions that provide two NuGet packages for caching. Microsoft recommends using Microsoft. Extensions. Caching. The Memory, because it can be integrated and Asp.NET Core, can be easily into Asp.NET in the Core. Use Microsoft. Extensions. Caching. The Memory of the sample code is as follows:
public class SimpleMemoryCache<TItem>
{
private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions());
public TItem GetOrCreate(object key, Func<TItem> createItem)
{
TItem cacheEntry;
if(! _cache.TryGetValue(key,out cacheEntry))
{
cacheEntry = createItem();
_cache.Set(key, cacheEntry);
}
returncacheEntry; }}Copy the codeHere’s how to use it:
var _avatarCache = new SimpleMemoryCache<byte[] > ();var myAvatar = _avatarCache.GetOrCreate(userId, () => _database.GetAvatar(userId));
Copy the codeFirst, this is a thread-safe implementation that can be safely called from multiple threads at once. Second, MemoryCache is allowed to join all expulsion policies. Here is an example of IMemoryCache with an expulsion policy:
public class MemoryCacheWithPolicy<TItem>
{
private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions()
{
SizeLimit = 1024
});
public TItem GetOrCreate(object key, Func<TItem> createItem)
{
TItem cacheEntry;
if(! _cache.TryGetValue(key,out cacheEntry))
{
cacheEntry = createItem();
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetSize(1)
.SetPriority(CacheItemPriority.High)
.SetSlidingExpiration(TimeSpan.FromSeconds(2))
.SetAbsoluteExpiration(TimeSpan.FromSeconds(10));
_cache.Set(key, cacheEntry, cacheEntryOptions);
}
returncacheEntry; }}Copy the code- SizeLimit is added to MemoryCacheOptions. This adds a cache size based policy to our cache container. The size of a mixed village has no units. We need to set the size on each cache entry;
- We can use.setPriority () to set what level of cache to remove when the size limit is reached. The levels are Low, Normal, High, and NeverRemove.
- SetSlidingExpiration(timespan.fromseconds (2)) sets the sliding expiration time to two seconds, and if an item is not accessed within two seconds, it will be deleted;
- SetAbsoluteExpiration(timespan.fromseconds (10)) sets the absolute expiration time to 10 seconds, within which the item will be deleted.
Do you think this implementation is ok? He had a problem:
- Although you can set a cache size limit, the cache does not actually monitor GC pressure.
- When multiple threads request the same project at the same time, the request will not wait for the first one to complete, and the project will be created multiple times. For example, if the avatars are being cached and it takes 5 seconds to get the avatars from the database, then within 3 seconds after the first request, another request to get the avatars will check if the avatars are cached, which is not, so it will also start accessing the database.
Let’s address the two issues mentioned above: First, regarding GC pressure, there are a number of techniques and heuristics that can be used to monitor GC pressure. The second problem is easier to solve using a MemoryCache:
public class WaitToFinishMemoryCache<TItem>
{
private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions());
private ConcurrentDictionary<object.SemaphoreSlim> _locks = new ConcurrentDictionary<object, SemaphoreSlim>();
public async Task<TItem> GetOrCreate(object key, Func<Task<TItem>> createItem)
{
TItem cacheEntry;
if(! _cache.TryGetValue(key,out cacheEntry))
{
SemaphoreSlim mylock = _locks.GetOrAdd(key, k => new SemaphoreSlim(1.1));
await mylock.WaitAsync();
try
{
if(! _cache.TryGetValue(key,out cacheEntry))
{
cacheEntry = awaitcreateItem(); _cache.Set(key, cacheEntry); }}finally{ mylock.Release(); }}returncacheEntry; }}Copy the codeUsage:
var _avatarCache = new WaitToFinishMemoryCache<byte[] > ();var myAvatar = await _avatarCache.GetOrCreate(userId, async() = >await _database.GetAvatar(userId));
Copy the codeThis implementation locks the creation of the project, and the lock is keyspecific. If we are waiting to get Joe’s avatar, we can still get Joe’s avatar cache on another thread. _locks stores all locks, and since regular locks are not suitable for async, await, we need to use SemaphoreSlim. The above implementation has some overhead and can only be used when:
- When the creation time of the project has some cost;
- When a project takes a long time to create;
- When you must ensure that each key creates an item.
Caching is a powerful but dangerous pattern with its own complexity. Too much caching can cause GC stress, and too little can cause performance problems.