Flow limiting is a feature that is often integrated in back-end service development and is particularly useful in preventing systems from crashing due to stress. In multi-tenant systems, traffic limiting is also particularly useful for limiting the amount of resources used by a single tenant, which this article explores.

The problem

In a multi-tenant system, if some tenants use too many resources, other tenants may be affected. For example, the maximum capacity of the system for querying a resource is 300qps. Assume that the query water level of tenants is 10qps in normal cases. In this case, 30 tenants can be served simultaneously. All of a sudden, the query water level of a tenant rises to 100qps. If the system has no limited traffic, the system will slow down or even crash. If the system has limited traffic, all tenants may be affected by traffic limiting.

A sudden surge in the query water level of a tenant could be normal business or a malicious attack, but in either case it is unfair to other tenants. In this case, you can limit the traffic flow for different tenants. Here is how to solve the problem.

The principle of

The figure below illustrates the principle of limiting traffic for a multi-tenant system.

Tenants A, B, C, respectively, initiate A service request, the service first to identify user identity authentication through into the current limit inspection link, current-limiting inspections for each tenant, if did not meet current limiting threshold, you can through into the next, if the current limiting threshold, the termination of the processing and return an error to the tenant.

implementation

If you already understand the principle, you will be able to write the corresponding flow limiting control logic. However, for the sake of development efficiency, you should reuse existing components as much as possible. This logic is implemented here using fireflysoft.ratelimit. The demo code is based on ASP.NET Core, and you can use it in any other application. NET network services framework used, or even their own handwritten special processing procedures.

Install Nuget package

Using the Package Manager console:

Install-Package FireflySoft.RateLimit.AspNetCore
Copy the code

Or use the.NET CLI:

dotnet add package FireflySoft.RateLimit.AspNetCore
Copy the code

Or add it directly to the project file:

<ItemGroup>
<PackageReference Include="FireflySoft.RateLimit.AspNetCore" Version="2.*" />
</ItemGroup>
Copy the code

Using middleware

Register the traffic limiting service in startup. cs and use the traffic limiting middleware.

public void ConfigureServices(IServiceCollection services) { ... services.AddRateLimit(new InProcessFixedWindowAlgorithm( new[] { new FixedWindowRule() { Id = "1", ExtractTarget = context => {// Assume that the target of traffic limiting is the user Id of the tenant, Return (context as HttpContext).request.gettypedheaders ().get <string>("userId"); }, CheckRuleMatching = context => {// Assume that only the /Weather/GetToday interface requests are restricted var path = (context as) HttpContext).Request.Path.Value; if(path == "/Weather/GetToday"){ return true; } return false; }, Name=" Multi-tenant traffic limit ", LimitNumber=10, StatWindow= timespan.fromseconds (1), // Traffic limit time window, Here is 1 second StartTimeType = StartTimeType FromNaturalPeriodBeign}})); . } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { ... app.UseRateLimit(); . }Copy the code

After completing these two steps, the multi-tenant traffic limiting function can work normally.

Here for all tenants to use the same current limiting threshold, if the tenant have privileges to tenants and the difference between ordinary tenants, can respectively in the rules of the above current limiting privileges tenants and ordinary tenants can define different rules, specific can refer to this article: ASP.NET how to difference between the different types of users in the Core current limit.

Other technical issues

Note that the traffic limiting algorithm used here is the in-process fixed window traffic limiting algorithm. If you want to uniformly count traffic limiting in distributed environment, you can use FireflySoft.RateLimit’s built-in Redis fixed window traffic limiting algorithm, but you need to have a Redis service first.

The fixed window algorithm is relatively rigid. In actual situations, the requests are likely to be uneven, and it is difficult to set a reasonable traffic limiting threshold for the rigid algorithm. If you want to allow a certain amount of burst traffic, you can use sliding window, leaky bucket, token bucket, etc. FireflySoft.RateLimit has integrated these algorithms, you can use them directly.

If the service capability of the system is improved and a tenant is allowed to make 20 requests per second, you need to adjust the rules. FireflySoft.RateLimit supports dynamic adjustment of the rules

Well, that’s the main content of this article.