preface

Caching is designed to reduce heavy I/O operations and increase system concurrency. Whether it’s multi-level CPU caches, page caches, or redis caches familiar to our business, the essence is to store a limited amount of hot data in a storage medium with faster access.

The cache design of the computer itself is that the CPU adopts multi-level caching. So for our service, can we also organize our cached data in this multi-level caching way? At the same time, redis access will go through network IO, so we can not put hot data directly stored in the process, by the process itself cache a recent hot batch of data?

This brings us to today’s topic: local cache, also known as process cache.

This article takes you through the process cache design in Go-Zero. Let’s go!

Quick start

As a process storage design, of course cruD has:

  1. So let’s initializelocal cache
// Initialize the local cache
cache, err = collection.NewCache(time.Minute, collection.WithLimit(10))
iferr ! =nil {
  log.Fatal(err)
}
Copy the code

The meanings of parameters are as follows:

  • expire: key Unified expiration time
  • CacheOption: Cache Settings. For example, set the upper limit of key
  1. Basic operation cache
// 1. add/update adds/modifies the API
cache.Set("first"."first element")

// 2. Get Obtains the value of the key
value, ok := cache.Get("first")

// 3. del Deletes a key
cache.Del("first")
Copy the code
  • Set(key, value)Set the cache
  • value, ok := Get(key)Read cache
  • Del(key)Delete the cache
  1. Advanced operation
cache.Take("first".func(a) (interface{}, error) {
  // Write to the local cache
  time.Sleep(time.Millisecond * 100)
  return "first element".nil
})
Copy the code

The previous Set(key, value) simply adds

to the cache; Take(key, setFunc) executes the fetch method passed in when the value of the key pair does not exist, handing the specific fetch logic to the developer to implement, and automatically putting the result into the cache.
,>

This is the core of the use of the code is basically covered, in fact, looks quite simple. You can also go to github.com/tal-tech/go… Look at the use in test.

The solution

First of all, cache is essentially a medium for storing limited hot data. It faces the following problems:

  1. Limited capacity
  2. Hotspot Data Statistics
  3. Multithreaded access

Let’s talk about these three aspects of our design practice.

Limited capacity

Limited means full to eliminate, this involves the elimination strategy. LRU (least recently used) is used in cache.

So how does that happen? There are a few options:

  1. Start a timer and loop through all keys until they expire by default. Execute a callback (in this case, delete keys from the map).
  2. Lazy deletion. Access determines whether the key is deleted. The downside is that if it is not accessed, it will increase space waste.

In cache, the first type of active deletion is adopted. However, the biggest problem encountered in active deletion is:

Loop over and over again, emptying CPU resources, even in additional coroutines, which is unnecessary.

The cache takes the time wheel to record additional expiration notifications, and then triggers a deletion callback when there is a notification in an expired channel.

Go-zero. dev/cn/timing-w…

Hotspot Data Statistics

For caching, we need to know if the cache is valuable if it uses extra space and code, and we want to know if we need to further optimize expiration times or cache sizes, all of which we rely on statistics capabilities, which are also provided by SQLC and Mongoc in Go-Zero. Therefore, we also added cache in cache to provide developers with the feature of local cache monitoring. When accessing ELK, developers can more intuitively monitor the distribution of cache.

The design is simple: Get() hits, add 1 to count.

func (c *Cache) Get(key string) (interface{}, bool) {
  value, ok := c.doGet(key)
  if ok {
    / / hit a + 1 bonus to hit
    c.stats.IncrementHit()
  } else {
    // Miss +1
    c.stats.IncrementMiss()
  }

  return value, ok
}
Copy the code

Multithreaded access

When multiple coroutines are accessed concurrently, the following problems are involved for caching:

  • Write-write conflict
  • LRUConflict with movement of elements in
  • Traffic impact or invalid traffic occurs when concurrently writing to the cache

In this case, the simplest way to resolve the write conflict is to add a lock:

// Set(key, value)
func (c *Cache) Set(key string, value interface{}) {
  // Add a lock and write 
      
        as a key-value pair to the map in cache
      ,>
  c.lock.Lock()
  _, ok := c.data[key]
  c.data[key] = value
  // lru add key
  c.lruCache.add(key)
  c.lock.Unlock()
  ...
}

// There is another place to operate on LRU: Get()
func (c *Cache) doGet(key string) (interface{}, bool) {
  c.lock.Lock()
  defer c.lock.Unlock()
  // When the key is present, the position in the LRU item is adjusted. This process is also locked
  value, ok := c.data[key]
  if ok {
    c.lruCache.add(key)
  }

  return value, ok
}
Copy the code

Concurrent execution of write logic, which is mostly imported by the developers themselves. And the process:

func (c *Cache) Take(key string, fetch func(a) (interface{}, error)) (interface{}, error) {
  // 1. Get the value in doGet()
  if val, ok := c.doGet(key); ok {
    c.stats.IncrementHit()
    return val, nil
  }

  var fresh bool
  SharedCalls are used to obtain the result shared by multiple coroutines
  val, err := c.barrier.Do(key, func(a) (interface{}, error) {
    // Double check prevents multiple reads
    if val, ok := c.doGet(key); ok {
      return val, nil}...// The important thing is that the cache setup function passed in is executed
    val, err := fetch()
    ...
    c.Set(key, val)
  })
  iferr ! =nil {
    return nil, err
  }
  ...
  return val, nil
}
Copy the code

SharedCalls, on the other hand, return results by sharing, saving the function from multiple executions and reducing coroutine contention.

conclusion

This article covers local cache design practices. From usage to design, you can also dynamically modify the expiration policy of the cache based on your business, adding the statistics you want, and implementing your own local cache.

It is even possible to combine local caching with Redis to provide multi-level caching for services, which will be left to our next article: multi-level caching in services.

For more articles on the design and implementation of Go-Zero, you can follow the “Microservices Practice” public account.

The project address

Github.com/tal-tech/go…

Welcome to Go-Zero and star support us!

Wechat communication group

Pay attention to the “micro service practice” public account and click into the group to obtain the qr code of the community group.