Please follow the public account “Go Xuetang” to learn more Go mistakes
01 background
This paper is based on a function of recording real-time requests based on REDis in the project. Due to the increase of traffic, the CPU of Redis is higher than 80% and the automatic alarm mechanism is triggered. By changing the way of writing requests to REDis in real time into the way of batch writing, the CPU usage is reduced by about 30%.
The business requirements are as follows: Requests received by our HTTP server are divided by geographical attributes. The goal is to control the total number of requests for a specific country. When the preset maximum number of requests is reached, the traffic is no longer processed and a 204 response is returned. If the maximum number of requests is not reached, +1 is required for real-time requests. As shown below:
02 Implementation version 1
First version is very simple, is the maximum in redis, then flow according to the time as a key, the flow of real-time counting record, each traffic came after the first query out the maximum flow properties, and the current number of real-time, and then compare, if real-time number has exceeded the maximum value, return directly, Otherwise, perform the +1 operation on the real-time number.
Take the traffic from China (denoted by CN) as an example. First, we have the following rules for keys in Redis:
- The key that represents the maximum number of requests a country can accept represents the rule: Country: Max :req
- Indicates the number of received requests from a country. Key indicates the rule: Country :YYYYMMDD: REQ and the validity period is N days.
The code for the first version is as follows:
func HasExceedLimitReq(a) bool {
key := "CN:max:req"
maxReq := redis.Get(key)
day := time.Now().Format("20060102")
dailyKey := "CN:"+day+":req"
dailyReq := redis.Get(dailyKey)
if dailyReq > maxReq {
return true
}
redis.Incr(dailyKey, dailyReq)
redis.Expire(dailyKey, 7*24*time.Hour)
return false
}
Copy the codeIn the above implementation, we don’t need to keep the dailyKey for a long time. In fact, after that day, the value of the key is useless, and we set the validity period of the dailyKey to 7 days for the purpose of querying historical data. However, the Incr operation of Redis does not carry an expiration time, so we add an Expire operation after the Incr operation.
Okay, so let’s see what’s wrong with this implementation. First, there’s no logical problem. When a request comes in, we have four operations on Redis without overloading: two queries and two writes (INCR and EXPIRE). In other words, Redis carries 4 times as many QPS as the traffic itself. If the traffic QPS increases, say, to 100,000, then Redis receives 400,000 requests. The CPU consumption of Redis comes naturally.
So where can we optimize? The first is that the Expire operation does not seem to be required every time. In theory, you only need to set the Expire time once, not every time, thus saving one write operation. The following implementation is version two
Implementation version 2: Reduces the number of times Expire is executed
We use a map type of hasUpdateExpire to record whether a key has been set to expire. As follows:
var hasUpdateExpire = make(map[string]struct{}) // Global variables
func HasExceedLimitReq(a) bool {
key := "CN:max:req"
maxReq := redis.Get(key)
day := time.Now().Format("20060102")
dailyKey := "CN:"+day+":req"
dailyReq := redis.Get(dailyKey)
if dailyReq > maxReq {
return true
}
redis.Incr(dailyKey, dailyReq)
ifhasUpdateExpire[dailyKey]; ! ok { redis.Expire(dailyKey,7*24*time.Hour)
hasUpdateExpire[dailyKey] = struct{}{}
}
return false
}
Copy the codeWe know that in Go, map is non-concurrent safe. The following code is concurrency safe:
ifhasUpdateExpire[dailyKey]; ! ok { redis.Expire(dailyKey,7*24*time.Hour)
hasUpdateExpire[dailyKey] = struct{}{}
}
Copy the codeThat is, it is possible that multiple coroutines will execute if hasUpdateExpire[dailyKey] at the same time, and all of them will get ok as false, so there will be multiple coroutines that execute the following two lines of code:
redis.Expire(dailyKey, 7*24*time.Hour)
hasUpdateExpire[dailyKey] = struct{} {}Copy the codeHowever, according to the scenario of our business, it does not matter if a few more Expire operations are performed. In the case of high QPS, setting several more Expire operations is negligible compared to the total number of requests.
What if the QPS continues to increase? That’s asynchronous bulk write. This type of writing is suitable for scenarios where accurate counts are not required. Let’s look at version three.
04 Implementation Version 3: Asynchronous batch write
In this version, instead of writing directly to Redis, our technique writes to an in-memory cache, a global variable, and starts a timer that writes the in-memory data in batches to Redis every period of time. As shown below:
So we define the following data structure:
import (
"sync"
"time"
"github.com/go-redis/redis"
)
const (
DefaultExpiration = 86400 * time.Second * 7
)
type CounterCache struct {
rwMu sync.RWMutex
redisClient redis.Cmdable
countCache map[string]int64
hasUpdateExpire map[string]struct{}}func NewCounterCache(redisClient redis.Cmdable) *CounterCache {
c := &CounterCache{
redisClient: redisClient,
countCache: make(map[string]int64),}go c.startFlushTicker()
return c
}
func (c *CounterCache) IncrBy(key string, value int64) int64 {
val := c.incrCacheBy(key, value)
redisCount, _ := c.redisClient.Get(key).Int64()
return val + redisCount
}
func (c *CounterCache) incrCacheBy(key string, value int64) int64 {
c.rwMu.Lock()
defer c.rwMu.Unlock()
count := c.countCache[key]
count += value
c.countCache[key] = count
return count
}
func (c *CounterCache) Get(key string) (int64, error) {
cacheVal := c.get(key)
redisValue, err := c.redisClient.Get(key).Int64()
iferr ! =nil&& err ! = redis.Nil {return cacheVal, err
}
return redisValue + cacheVal, nil
}
func (c *CounterCache) get(key string) int64 {
c.rwMu.RLock()
defer c.rwMu.RUnlock()
return c.countCache[key]
}
func (c *CounterCache) startFlushTicker(a) {
ticker := time.NewTicker(time.Second * 5)
for {
select {
case <-ticker.C:
c.flush()
}
}
}
func (c *CounterCache) flush(a) {
var oldCountCache map[string]int64
c.rwMu.Lock()
oldCountCache = c.countCache
c.countCache = make(map[string]int64)
c.rwMu.Unlock()
for key, value := range oldCountCache {
c.redisClient.IncrBy(key, value)
if_, ok := c.hasUpdateExpire[key]; ! ok { err := c.redisClient.Expire(key, DefaultExpiration)if err == nil {
c.hasUpdateExpire[key] = struct{}{}}}}}Copy the codeThe main idea here is to temporarily store data in the countCache of the structure while writing to it. Each CounterCache instance then starts a timer ticker that updates the countCache data to Redis at regular intervals. Let’s see how this is used:
package main
import (
"net/http"
"sync"
"time"
"github.com/go-redis/redis"
)
var counterCache *CounterCache
func main(a) {
redisClient := redis.NewClient(&redis.Options{
Addr: "127.0.0.1:6379",
Password: "",
})
counterCache = NewCounterCache(redisClient)
http.HandleFunc("/", IndexHandler)
http.ListenAndServe(": 8080".nil)}func IndexHandler(w http.ResponseWriter, r *http.Request) {
if HasExceedLimitReq() {
return
}
// Handle normal logic
}
func HasExceedLimitReq(a) bool {
maxKey := "CN:max:req"
maxCount, _ := counterCache.Get(maxKey)
dailyKey := "CN:" + time.Now().Format("20060102") + ":req"
dailyCount, _ := counterCache.Get(dailyKey)
if dailyCount > maxCount {
return true
}
counterCache.IncrBy(dailyKey, 1)
return false
}
Copy the codeThe usage scenario here is used when the count is not accurate. For example, if the server exits unexpectedly, the data temporarily stored in countCache will be lost before being flushed to Redis.
Another important thing to note is that the countCache variable is a map. As we know, map is a non-concurrent safe operation in Go, so be careful to add read/write locks.
05 summary
As service QPS grow, we will increase the utilization of all resources without limiting QPS. Our optimization ideas mainly reduce redis operations by caching. This counting method is used in situations where the counting requirements are not so accurate, such as the number of videos played, the number of tweets read, and so on.