The content of this article:
SentinelWhat is the performance costSentinelSource code analysis of workflowSentinelFuse downgrade source code analysis
What is the performance cost of Sentinel
From the previous article “Implementation Principle of QPS Statistics of Sentinel Flow Limiting Core Function”, we know that Sentinel statistical QPS uses time window +Bucket, and recycle Bucket to reduce the occupation of memory. In QPS statistics, the current timestamp is used to locate Bucket. LongAdder is used to count the number of successful requests, the number of failed requests, and the total time spent in the time window to optimize the concurrent locking, and the System is not used to obtain the current time each time by increasing the timestamp of the scheduled task. You can see the performance efforts that Sentinel has made to minimize its impact on the application as much as possible.
Sentinel creates an array of buckets with a one-minute window of one second and an array of buckets with a one-second window of 500ms for each resource (interface). The two arrays are wrapped as a Node to collect the request data for the interface. Each Bucket records the total number of requests in a time window, the total number of failed requests, the total time (the average time can be calculated by the total time), and the total number of requests that are restricted or blocked.
Therefore, the memory consumed by Sentinel is at least the total number of resources multiplied by the memory occupied by each Node for each resource, which is the memory occupied by a Bucket array of size 2 and a Bucket array of size 60.
Source code analysis of Sentinel workflow
Sentinel reduces memory consumption by reuse Bucket and reduces performance consumption of concurrent statistics by LongAdder. In addition to these, Sentinel realizes statistics, current limiting, fuse degradation and other functions through responsibility chain mode to achieve local lock-free.
Basic uses of Sentinel:
Sentinel realizes statistics, traffic limiting, fuse downgrading and other functions by one ProcessorSlot. For example, StatisticSlot is used to count the total number of requests and failures in the current time window of resources, and FlowSlot is used to determine whether current requests need traffic limiting. Determine whether a fuse degrades a request to achieve survival.
These processorSlots are packaged in a linked list in a strict order, such as StatisticSlot before FlowSlot and FlowSlot before DegradeSlot.
The ProcessorSlot entry method is called when a client request is received or before the client sends the request to the server, while the exit method is called when the server finishes processing the request (including exception completion) or when the client sends the request. Each ProcessorSlot signals down through the fireEntry method or fireExit method.
Netty is not unfamiliar with the use of this design pattern, Netty is also through the chain of responsibility pattern of the request Handler wrapped as a linked list, to achieve partial serial processing requests. But Sentinel’s ProcessorSlot differs from Netty’s Handler in that the exit method of ProcessorSlot is not passed from back to front like Netty’s.
The familiar Shiro is also implemented through a chain of responsibility (filter), so it is not hard to understand why Sentinel implements current limiting and fusing. In the case that cluster limiting is not considered. When the ENTRY method of SphU is invoked, it passes StatisticSlot, FlowSlot, and Anyhow. The sequence diagram is as follows.
When the Entry method of StatisticSlot is invoked, StatisticSlot obtains the Bucket of the current time window from the Node that obtains the resource based on the current timestamp, and then incrementing the total number of requests to the Bucket by 1. StatisticSlot catches exceptions in the Entry method. If the downstream ProcessorSlot throws a BlockException or a subclass of BlockException, the total number of Bucket exceptions is increased by one; otherwise, the total number of Bucket exceptions is increased by one.
When the Entry method of FlowSlot is invoked, the system checks whether the flow limiting rule configured for the current resource meets the flow limiting condition. If the flow limiting condition is met, a BlockException exception is thrown, indicating that the current request is restricted. Because Sentinel supports cluster traffic limiting, the implementation of traffic limiting is complicated, so we will not discuss it for the moment. If it is single-node current limiting, the implementation is similar to the implementation of fusible downgrading. This paper only introduces the implementation of fusible downgrading.
When the Survival of eslot entry method is invoked, the device checks whether the fuse degrade rule meets the requirements. If the degrade rule meets the requirements, a DegradeException exception is thrown.
Sentinel fuse downgrading implementation source code analysis
Sentinel creates a Node for each ResourceWrapper to collect request data (total number of requests, total number of exceptions, total number of traffic limiting or fusing, and total time) to support traffic limiting and fusing degradation.
The ResourceWrapper name is the resource name or the interface URL, but this is not correct. Resource names are also used when we configure traffic limiting rules or fuse downgrading rules.
The entryType of ResourceWrapper is the traffic type. The value can be IN or OUT. IN indicates the inflow type, that is, the server receives requests from the client. OUT indicates the outgoing type, that is, the client sends a request to the server.
The resourceType of ResourceWrapper is the resourceType, indicating whether it is a Servlet, RPC, API gateway, or database. Sentinel also supports traffic limiting and fusing.
The circuit breaker degradation function provided by Sentinel can be used not only on the client but also on the server. However, it is generally placed on the client for circuit breaker degradation of resources whose traffic type is OUT to ensure that it is not affected by the server and not dragged down by the server.
To determine whether Sentinel’s fuse degrade function can be implemented in the server, read the Guide to see if it only takes effect when entrytype.out is available.
The checkDegrade method of The Survival Manager is called when the Entry method is invoked to check whether the current request meets a fuse degrade rule.
To learn how to achieve a meltdown degradability with Sentinel, it is added to the General Manager to achieve the check logic.
It first gets configured fuse degrade rules based on the resource name. It returns a total because it is possible to configure multiple fuse degrade rules for the same resource. It then passes the fuse degrade rules and calls the passCheck method to achieve survival of all fuses. If multiple fuse degrade rules are configured for a resource, a fuse is triggered as long as one of the fuse degrade rules meets the condition.
The passCheck method for Survival ule is provided below.
Apart from the passCheck method for Survival ule, we don’t see anything that limits the availability of a fuse degrade to entrytype. OUT. Therefore, it can be used not only for clients but also for servers.
There are three types of fuse downgrading policies:
- 1. Average response time (
DEGRADE_GRADE_RT) - 2. Abnormal proportion (
DEGRADE_GRADE_EXCEPTION_RATIO) - 3. Number of anomalies (
DEGRADE_GRADE_EXCEPTION_COUNT).
The official document adds the following instructions in the section describing the DEGRADE_GRADE_EXCEPTION_COUNT policy: notice because the timeWindow for collecting statistics is minute, if the timeWindow is less than 60 seconds, the fuse may be in the fuse state again after the end.
It is easy to understand why it is separated into three groups: passCheck.
END
Sentinel provides adaptive modules for accessing Spring MVC, Dubbo RPC, API GATEWAY, etc. The adaptor implements nothing more than intercepting various requests or sending requests.