Zhang Chao: Senior engineer of Youpaiyun system development, responsible for updating and maintenance of relevant components of Youpaiyun CDN platform. Github ID: Tokers, active in open source communities such as the OpenResty community and the Nginx mailing list, focuses on server-side technology research; Has contributed source code for NGX_LUa, in Nginx, NGX_LUa, CDN performance optimization, log optimization has more in-depth research.
Child request, parent request, and master request
Most of the requests handled by Nginx are created after receiving HTTP request packets from clients. These requests are directly dealt with by clients, which are called master requests. This is contrasted with child requests, which, as the name implies, are created by other requests, such as the main request (which can also create children itself). When a request creates a child request, it becomes the parent of that child request. At the source level, the main request of the current request is obtained through the R ->main pointer, and the parent request is obtained through the R ->parent pointer.
The significance of using sub-request mechanism is that it can decentralize the processing logic that is concentrated in a single request, simplifying the task and greatly reducing the complexity of the request. For example, when we need to access both a MySQL cluster and a Redis cluster, we can create a sub-request to be responsible for the interaction with MySQL and another one to be responsible for the interaction with Redis, simplifying the business complexity of the main request. And the process of creating a sub-request does not involve any network I/O, just some memory allocation, and the cost is very manageable, so in my opinion, the sub-request mechanism is one of the most clever designs in Nginx.
Sub-request creation and driver
Module developers usually call ngx_HTTP_subrequest to create a child request. By default, the child request shares the parent request’s memory pool, variable cache, downstream connection, and HTTP request data. When a child request is created, it is attached to the R ->main-> Posted_requests list, which holds requests (not just child requests) that need to be deferred. Therefore, the child request will get the chance to run after the parent request completes local scheduling, which is usually the means for the child request to get the first chance to run.
We know that Nginx divides the processing logic of an HTTP request into 11 different stages. When a child request is created, it first runs the Find Config phase, which looks for an appropriate location, and then begins the subsequent logical processing. Typically, if a subrequest does not involve any network I/O operations or timer processing, a single dispatch can complete the current subrequest. If the sub-request needs to handle some network and timer events, the subsequent scheduling of the sub-request will be driven by these events, which makes its scheduling no different from ordinary main request.
Since subrequests may be driven by network events except for the first time, the scheduling of subrequests is out of order. Assuming that the current main request needs to request a resource of size 2MB from the backend, we obtain 0-1MB and 1MB-2MB respectively by generating two sub-requests and then send them downstream. Due to the uncertainty of the network, it is likely that the latter (1MB-2MB) will be acquired first and transmitted downstream. The downstream data is dirty.
To solve this problem, Nginx introduced another module called postpone_filter for the subrequest mechanism. The purpose of this module is to determine whether the current request to send data is “active.” If the current request is not “active,” the data it expects to send will be held until it is “active” before it can send it downstream.
How do you tell if a request is “active”? We need to understand the form of saving between parent and child requests. For the current request, its sub-requests are maintained as a linked list, and as mentioned earlier, sub-requests can also create sub-requests, so the complete preservation form between these requests can be understood as a hierarchical tree, as shown in the figure below.
In the figure above, each red circle represents a request, and each layer of request is a child of the previous layer of request. In tree traversal terms, which node should be processed first in such a tree? Combined with the practical significance of the sub-request mechanism, the sub-request is to share the processing logic of the parent request and reduce the business complexity. In other words, the parent request is dependent on the child request. For the most part, the parent request may have to wait until the current child request has finished running to do some finishing work based on the feedback from the child request. So what you need is something like a post-order traversal. The request in the bottom right corner of the figure is the first “active” request.
From the source code level, hierarchical tree save with the help of two data structures, r – > postponed and r – > the parent of these two Pointers, traverse r – > postponed to sequential access to the current request of (the brother of tree node in the tree); Traverse r->parent to access the parent request (the parent at the upper level of the tree).
The postpone_filter module will judge whether the request is “active” or not, and if it is not, its postponed link will be temporarily put to its r-> arent list. If arent active, arent will go through its r-> arent list and have to either send the temporarily blocked data or find the first child request, mark it as “active” and return. The parent request will be marked as “active” when the child request is finished. Thus, when the parent request runs to the postpone_filter module again, the parent can go through the r-> arent linked list until all requests or data processing is completed. Interested students can read the relevant source code (hg.nginx.org/nginx/file/…) .
A module that uses a subrequest mechanism
There are many examples of using subrequests throughout the Nginx ecosystem, most notably ngx_Lua’s subrequests and the official SLice_filter module.
The API provided by ngx_Lua to users (ngx.location.capture) is very flexible. Including whether to share variables is optional. In particular, when the child request of ngx_Lua runs, it blocks the parent request (suspending its corresponding Lua coroutine). Until the child request completes, the child request’s response header, response body (so if the response body is large, it will consume a lot of memory) and other information are returned to the parent request. Ngx_lua’s sub-requests do not pass through the postpone_filter module, which intercepts the sub-request response body in an earlier filter module (ngx_HTTP_luA_capture_filter).
The slice_filter module of Nginx can split a resource download into several HTTP Range requests. The biggest benefit of this is to spread out the hotspot. This module allows us to set the slice_size directive to set the Range size for subsequent Range requests. The module creates sub-requests (after the previous one completes) until the required resources are downloaded.
In addition, Nginx/1.13.1 introduced a mechanism called Background subrequests (for updating caches). Based on this mechanism, Nginx/1.13.4 introduced a mirror module that allows users to customize background tasks by creating sub-requests. For example, preheat some resources and put them directly into Nginx’s own proxy_cache.
Pitfalls and Pitfalls
As mentioned earlier, when the child request is created, some data from the parent request is reused, which introduces some potholes.
For example, variable caching, if a variable is accessed and cached in the child request, when later used in the parent request, we will get the previously cached data, which can cause engineers to spend a lot of time and effort to debug this problem.
In addition, I think a very significant defect is that the child request overuses the memory pool of the parent request. Take the slice_filter module for example, it divides an HTTP request into several sub-requests, and each sub-request launches HTTP Range requests at the back end. And configuration slice_size is relatively small, cause there are a large number of child request the creation of the whole resource download process may last for a long time, this leads to the parent request memory pool for a period of time did not release, if combined with concurrency is bigger, may be the cause of process memory usage are high, serious when may OOM, Services are affected. Therefore, these issues need to be weighed when considering usage, and you may need to modify the source code yourself if necessary to meet business needs.
Although some shortcomings are inevitable, but the sub-request mechanism greatly simplifies the request processing logic, its divide and conquer processing idea is very worthy of us to learn and use for reference, in any case, the sub-request mechanism will also be a great reference example for the subsequent system design.
Nginx in My Eyes series:
Nginx and bit arithmetic
HTTP/2 Dynamic table size update
Nginx variables and variable interpolation
Nginx in my eyes: What is making your Nginx service exit so slow?