This article is written by Zongwei, one of the team members. We have authorized the exclusive use of Doodle Front, including but not limited to editing, original labeling and other rights.
The thread pool
What a thread pool is
Let’s start with the first question, what is a thread pool. A technology is often created to solve a problem, and thread pools are no exception.
Let’s start with a hypothetical scenario.
Suppose you work in a jijiayuan website and your boss asks you to develop a recommendation service. The job is very simple. When a user visits your service, you need to recommend users according to their characteristics (such as gender). For example, when the monster visits, you need to recommend the rich lady.
When you’re given this task, the first thing to think about is that each incoming request from the user should be a separate thread. Because users are isolated from each other, what each thread does is a set of recommended actions.
Of course, if you’re good at it, you can create a new thread for each user request and destroy it when the response is complete, and all will be fine.
Burning goose, because your recommendation was so good, your website went viral, and a bunch of rich women were trying to sign up. So the question is, if 1000 rich women click at the same time, that means 1000 requests will come, do you have to create 1000 threads?
Will the threads be destroyed again after the series of requests? The most obvious overhead of thread creation is memory, and such frequent creation and destruction has a noticeable impact on performance, as well as a design that does not support instantaneous peak traffic.
Because of such a design, the rich women can not be satisfied, a jijia edge at risk.
This is where thread pools come in. As mentioned earlier, a technology is always created to solve some problem, so what problem does a thread pool solve? To summarize, this is thread lifecycle control. Let’s analyze it in detail.
Thread life cycle control
Let’s start with our problem, the frequent creation and destruction of threads. What’s the solution? It has to be thread reuse.
After a thread is created, even if the response ends this time, it is not allowed to be recycled, and the next time the request comes in, it is still allowed to process it.
The key point here is how to keep a thread from being recycled.
It seems amazing that a thread can continue to exist after completing an operation? So long? It’s actually very simple, just write an infinite loop. A thread is always in a loop, processing requests when they come in, waiting when they don’t, waiting when they come in, waiting when they’re done, skipping over and over again, repeating the loop indefinitely.
This leads to the second key point: how does a thread in an infinite loop know when it has a request to process?
Different languages don’t implement it exactly the same way, but it’s basically based on blocking wake up. When no task is available, all threads are blocked. When the task comes, idle threads compete for the task. The fetched threads begin to execute, and the unfetched threads continue to block.
You may have seen various thread pool interpretations on the web, but for a deeper understanding, start with the source code. (In front of the source code, all the bells and whistles are pale)
ThreadPool libuv threadPool libuv threadPool
We directly captured the core of the thread pool implementation, as noted in the following figure, and basically implemented as expected.
To clarify, uv_cond_signal is equivalent to waking up a blocked thread, and uv_cond_wait is equivalent to blocking the current thread.
Thread Pool summary
To summarize, thread pools use an infinite loop to keep threads from terminating, block while waiting for a task, and use a blocking wake to make threads receive tasks (essentially blocking wake based on semaphores) to achieve thread reuse, end the current task and enter the next loop, and repeat.
eventloop
Eventloop is what
Eventloop means exactly what it’s called: an eventloop.
A while(true) loop is a loop that checks if there are any pending tasks, and if there are no pending tasks, the next loop will continue.
The general process is shown below.
The idea of EventLoop is simple; it doesn’t care how your callback is implemented or when your IO ends.
What he does is to constantly take events, take them and execute them. So the key point here is where did he get the event?
The answer: Watcher. Every event loop has an observer, and every round of the loop goes to the observer and takes the event and executes it. The watcher is actually a queue for events.
How to understand watcher, nodeJS gives a very vivid metaphor.
In restaurants, the little girl at the front desk is often responsible for taking the guests’ orders while the chef cooks in the back. The younger sister will put the menu in the kitchen after getting the guests’ menu, while the chef only needs to keep looking at the menu, cooking, looking at the menu, cooking. He doesn’t care who ordered it or when it was ordered.
So this menu right here is the watcher, which is essentially a queue, and the chef is like an eventloop, which is constantly in a loop, and each loop is going to fetch the request from the queue, execute the callback if there’s a callback, and go to the next loop if there’s no callback.
Eventloop + Thread pool = asynchronous non-blocking
Thread pools and EventPoll were covered in some detail above, so let’s take a look at how you can use them to implement asynchronous non-blocking.
Let’s clear our minds step by step. First, you’re cute enough to initiate an IO call. As mentioned in the Large Front-end Advanced Node.js series of asynchronous non-blocking, an IO call either blocks or initiates the IO without blocking and then blocks when it needs to see the result. Obviously neither of these modes is desirable.
We want asynchronous non-blocking, so the IO call must not be executed in the main thread, which is when we think of the thread pool above.
The main thread can’t block, but the threads in the thread pool can, and the main thread can just hand over the IO calls to the pool and have fun, achieving our first goal: non-blocking.
What about asynchronous? How do I get calls in a thread pool to execute callbacks when they end? This is where EventLoop comes in.
After the thread pool has finished IO processing, it will actively put the finished request into the Watcher of the EventLoop, which is our queue. The EventLoop is in a continuous loop state. When the next loop check to queue has a request, It will pull it out and execute the callback, so we have the asynchrony we want.
The result, combined with thread pools and eventloop, is that when you make an I/O call, you don’t have to block and wait for the I/O to finish, and you don’t have to constantly poll when you want to use the RESULT of the I/O. The whole I/O process is non-blocking for the main thread, and the callback is automatically executed when it finishes. Achieve the asynchronous non-blocking we want.
Finally, we’ll use a diagram from Nodejs.
As shown in the figure above, after an asynchronous call is made, a request parameter is encapsulated, which contains the parameters and the callback to execute at the end.
The wrapped request is then thrown to the thread pool for execution, and any thread in the pool that is free will fetch the request from its queue and perform an I/O operation.
Notifies the IOCP upon completion of execution, essentially putting the Requeat into a queue that acts as a hub between the thread pool and the event loop.
The event loop finds a request in the queue as it loops, retrieves it and executes the appropriate callback, completing a perfect asynchronous non-blocking.
conclusion
This article has included making https://github.com/ponkans/F2E, there is a line of giant advanced guide, welcome to Star, continuously updated
After a careful look at the strange Node asynchronous non-blocking (top) (bottom) two series, still can not be hung up the interview questions you feel free to come to me ~
ThreadPool libuv threadPool libuv threadPool libuv threadPool
Finally, a pithy and perfect summary follows.
Node uses the thread pool to execute the IO call, avoid blocking the main thread, put the result and request into a queue after execution, use the event loop to retrieve the queue request, and finally execute the callback to achieve asynchronous non-blocking effect.
Biubiu ~
- Collection of dACHang front-end component library tools (PC, mobile, JS, CSS, etc.)
- “Large Front-end Advanced Node.js” series multi-process model underlying implementation (bytedance was asked)
- “Big front-end Advanced Node.js” series double eleven seconds kill system
- Large Front-end Advanced Node.js series asynchronous Non-blocking (Part 1)
Like the small partner to add a concern, a praise oh, Thanksgiving 💕😊