How go-Zero Can Withstand the Impact of Traffic (1)

No matter in single service or micro service, the API interface provided by the developer to the front end has access limit. When the access frequency or concurrency exceeds its tolerance range, we must consider limiting the flow to ensure the availability of the interface or degrade the availability. The interface also needs to be equipped with a fuse to prevent the system from being overwhelmed by unexpected requests.

Go-zero incorporates a current limiter out of the box. There are two types of built-in current limiter, which also correspond to two usage scenarios:

Periodlimit is introduced in this article.

use

const (
    seconds = 1
    total   = 100
    quota   = 5
)
// New limiter
l := NewPeriodLimit(seconds, quota, redis.NewRedis(s.Addr(), redis.NodeType), "periodlimit")

// take source
code, err := l.Take("first")
iferr ! =nil {
    logx.Error(err)
    return true
}

// switch val => process request
switch code {
    case limit.OverQuota:
        logx.Errorf("OverQuota key: %v", key)
        return false
    case limit.Allowed:
        logx.Infof("AllowedQuota key: %v", key)
        return true
    case limit.HitQuota:
        logx.Errorf("HitQuota key: %v", key)
        // todo: maybe we need to let users know they hit the quota
        return false
    default:
        logx.Errorf("DefaultQuota key: %v", key)
        // unknown response, we just let the sms go
        return true
}
Copy the code

periodlimit

Go-zero uses the sliding window counting method to count the number of times a resource is accessed within a period of time. If the number exceeds the specified limit, the access is denied. Of course, if you are accessing different resources within a certain period of time, and each resource does not exceed the limit, this will allow a large number of requests to come in.

In a distributed system, there are multiple microservices providing services. So how do you get a counter to count properly in a distributed system when instantaneous traffic accesses the same resource at the same time? When computing resources are accessed simultaneously, multiple computations may be involved. How to ensure atomicity of computations?

• go-zeroWith the help ofredis 的 incrbyDo resource access counts
• usinglua scriptDo the whole window calculation to ensure the atomicity of the calculation

Let’s take a look at a few key properties controlled by Lua Script:

-- to be compatible with aliyun redis, 
-- we cannot use `local key = KEYS[1]` to reuse thekey
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
-- incrbt key 1 => key visis++
local current = redis.call("INCRBY", KEYS[1].1)
-- If it is the first access, set the expiration time => TTL = window size
Access is restricted for a period of time
if current == 1 then
    redis.call("expire", KEYS[1], window)
    return 1
elseif current < limit then
    return 1
elseif current == limit then
    return 2
else
    return 0
end
Copy the code

The above return code is returned to the caller. It is up to the caller to decide what follows from the request:

The following diagram depicts the incoming request and what happens when the request triggers limit:

Subsequent processing

If at some point in time, the service mass incoming requests, periodlimit short-term time limit threshold, and set the time range is far to reach. The handling of subsequent requests becomes a problem.

Periodlimit is not handled but instead returns a code. The handling of subsequent requests is left to the developer.

1. If not, the request is simply rejected
2. If you need to handle these requests, developers can helpmqBuffering requests takes the pressure off requests
3. usingtokenlimitAllows for temporary traffic shocks

So in the next article we’ll talk about TokenLimit

conclusion

The periodlimit scheme in Go-Zero is based on the Redis counter, which ensures the atomicity of the counting process by invoking the Redis Lua Script, and also ensures that the counting is normal in distributed mode.

However, this scheme also has disadvantages, because it has to record all the behavior records in the time window, if this amount is very large, the memory consumption can become very serious.

reference

• go-zero periodlimit
• Distributed service limiting actual combat, has lined up a pit for you

In the meantime, please use Go-Zero and join us at github.com/tal-tech/go…

If you think the article is good, welcome to github.com/tal-tech/go… A star 👏