preface
Coding should be a lifelong career, not just 30 youth rice This article has included making https://github.com/ponkans/F2E, welcome to Star, continuously updated
Bytedance: How does the Node.js multi-process model and the underlying principle of multi-process listening on the same port work?
This article explores the underlying implementation of node.js multi-process architecture and shares it with you!!
Many partners to some foundation, especially the bottom is not very understanding, incidentally can also make up the basic principle of the bottom ha ~
I hope you can learn something from each article. This article covers the Node.js multi-process model from the beginning to the end. I hope you can learn something from it:
- Thoroughly understand the relationships between processes, threads, and coroutines
- Understand the multi-process model of Node.js inside out
- How to build a server with higher performance with limited computer resources
Operating system processes and threads
The Node multi-process model was mentioned in the previous node. js double eleven system, and this article will explain the details of Node process in detail
Processes and threads are a perennial topic of conversation.
As long as it is engaged in computer related partners, mention this most think like spring, multi-threaded ~ high concurrency ~ but various scattered concepts and cognition may be difficult to combine into a system of knowledge structure. Let’s start with a brief official explanation of these two concepts.
- Process: code in execution, a running program that includes not only object code, but also data, resources, state, and virtual computers.
- Threads: A line of execution within a program is called a thread. A more accurate definition is that a thread is “a sequence of control within a process”.
See the above two definitions, many small friends may frown a little, what @#¥toys. Here are some pictures to help you understand.
Feel the process
Processes are everywhere on our computers. The league of Legends that has just been destroyed, the little movie playing in the browser, and so on, are all processes in motion.
Process is actually in the execution of the floorboard of the program and related resources, it contains code to be executed, the required documents, ports, hardware resources, one theory is that a very common process are the smallest units of resource allocation, this sentence is more bluntly said, you want to run an executable code will need some resources, when this code to run, These resources must also be allocated to him.
So let’s summarize: running code + resources it occupies = process.
Feel the thread
After the process, some of you might be confused.
Process = running code + resources, so what’s the point of our thread? Why not just let the process run.
As mentioned above, the process is the smallest unit of resource allocation, meaning that the process and resource ratio is 1:1. The corresponding sentence is that the thread is the smallest unit of scheduling, and the process and thread are a 1:n relationship.
For example, it is not completely appropriate to compare a shopping mall to a computer, in which the stores are processes, they are the smallest units of the shopping mall resources, they have corresponding resources, but also in business activities, just like a process with resources and in operation.
Each shop inside the clerk is a thread, they in their own resources in each role, someone to pull customers, someone platform, someone watch. These are the smallest units of true dispatch.
We just think, if the resource allocation and scheduling is 1:1, it means that a store in the activities of people at the same time, there can be only one when you are at the time of soliciting, others may not be in the store, you at the time of the platform, others can only be waiting in the side, but you are in the same stores resources.
This is obviously not OK, so the process is the same, using multithreading in a process is to share the same set of resources together.
This can greatly reduce the overhead of process resource switching. When we are doing multiple operations, the lighter they are, the better they switch between them.
When you switch between these two operations, the lighter the better. You don’t need to turn off your hand and restart it, and then open the game again. Otherwise, the phone is too weak
Cost of process and thread switching
Now that we’re talking about process switching, we can explore the overhead of process switching. A process can monopolize a number of resources, such as registers, memory, files, etc.
When switching, it first saves the scene, a series of intermediate results of execution, the program’s code and data, its stack, the contents of general purpose registers, program counters, environment variables, and the set of open file descriptors, which are stored in memory. This state is called context.
He then needs to switch the resources back when he recovers. Obviously, the fewer resources you have to save when switching, the better the system performs, and that’s why threads exist. Threads have their own context, including unique integer thread ids, stacks, stack Pointers, program counters, general-purpose registers, and condition codes.
It can be understood that thread context is a subset of process context.
Thread, coroutine
Programs are always written to maximize performance. Threading makes switching between programs that share the same set of resources lighter. Is there anything lighter than threading?
This is made possible by the advent of coroutines, threads and processes optimized by operating system support, and coroutines are essentially application-level optimizations.
It’s as if threads and progression are born wizards of the game, super-god players, and coroutines are wizards who think they are not super-god enough and want to be super-ghost, because they have made an acquired effort.
Coroutines can be understood as special functions that can be suspended somewhere and resumed outside of the suspension. In simple terms, multiple such special functions can be run in a thread, but it must be made clear that multiple coroutines in a thread run serally.
On a multi-core CPU, multiple processes or threads within a process can run in parallel, but an in-thread coroutine is absolutely serial, no matter how many cores the CPU has.
After all, a coroutine is a special function, but it’s still a function. Multiple functions can be run in a thread, but these functions are run serially. While one coroutine is running, the others must be suspended.
Coroutines generally come from language support, such as Python, and here is a random snippet of py code for coroutines. And what it does is pretty simple. Yield is the syntax in Python.
When a function reaches the yield keyword, it pauses on that line (not blocking, but at the application level) and waits until the main thread calls send. The coroutine receives the data and continues execution. (The Python yield syntax is old; the new syntax uses async/await)
Here are the results.
Linux thread process
Linux design has always been a simplification of complexity, in addition to the familiar everything is a file, he is similar to the design of process threads, strictly speaking, there is no concept of threads on Linux, how to say?
Because whether it’s a process or a thread, you have to have a proof of existence, you say you exist in the world, how do you prove it?
The existence proof of the process is the process control block, Linux each process has its corresponding control block, which contains the process ID, the hardware resources needed, the code segment executed and so on. The same is true for threads. In Windows, there is a clear thread control block, and the operating system does thread scheduling.
Linux provides the pThread library to fork microprocesses. Multiple microprocesses can share resources, which is essentially the same as threads. But there is no special thread control, interested students can understand in detail.
Oh dear, does it feel weird? Don’t worry, continue to look ↓~
More CPU, shadow body ~
What skills do you need to be a big player in NB?
Interview hundreds of billions of concurrent, induction button style, ha ha ha.
There is the concept of concurrency, and the concept of parallelism is confusing and confusing.
Now we hear a lot about multi-core machines, multi-CPU machines. What does multiple cpus mean?
The first step is to figure out what the CPU does. The role of CPU is summed up in two words: computing.
All of our fancy code, when it’s compiled and executed, is just one word: compute. As mentioned above, the process must be running code, which must be running on the CPU.
The fact that we have several cpus means that we can have several programs running at the same time, which is called parallelism, just like when you were a kid and you wanted to have naruto’s shadow avatar, you could have one of them write math, one of them write Chinese, and one of them write English.
With more check should be helpless pain of the single-core computer today, like no shadow of me, this time also have more than one job to do, how to do? Half an hour to write Chinese, half an hour to write math, another half an hour to write Chinese, and another half an hour to write math. (forced time slice rotation) This is Chinese, mathematics and English are also written at the same time, but in fact only I helpless pain of a person, this is time sharing concurrent, but not parallel.
The summary is that parallelism must be concurrent, but concurrency is not necessarily parallel.
On the CPU scheduling process strategy, CPU execution code details, if interested can leave a message, the subsequent time can be arranged, here will not expand
The Node of the thread
The Node single-threaded
The first day I learned Node, I saw that Node is a single-process single-thread model, which is thread safe. Well, it is thread safe. But t on the back end it looks like a single guy saying I’m not going to get lost in love, crap because you’re not.
As we mentioned above, a single thread, no matter how beautiful it is, can only be used on a single CPU, which is definitely not acceptable for nodes that we want to scale the full stack.
Node multi-process model
Since a Node process can only have one thread, it would be a mistake to try to squeeze the CPU (similar to Java) with a single process and multiple threads, but Node supports the multi-process model.
Node provides the child_process module, which replicates processes using the child_process.fork() function.
As shown in the figure below, the master calls child_process.fork, and the forked process is worker.
The child_process module gives Node the ability to create child processes. The parent and child processes work in a master/worker mode.
This pattern is common in distributed systems, but it can be used to manage both parent and child processes. Node uses this pattern to manage both parent and child processes on a single Node. This pattern is similar to the classic reactor pattern (reactor is the primary process), and uses the parent process as the primary process. And dispatch the task to the worker process.
Normally blocking operations are assigned to the worker to perform (db lookup, file reading, process time calculation, etc.), and non-blocking code is written on the master as much as possible.
Node Communicates with multiple processes
Now that the master-slave process is mentioned, the inevitable problem is the communication between them.
Process communication postures are many, such as based on socket, based on pipes, based on MMAP memory mapping and so on. Here we mainly discuss Node communication, here and we first briefly explain two concepts: file descriptors, pipes.
File descriptor is one of the operating system used for file management concept, as shown in the above, each process can have a own file descriptor table, which contains a file descriptor logo and the file pointer, every process their own tables from zero beginning, then by the file pointer to point to the same system level open file table, The open file table records the file offset (where the file was read or written to) and the inode pointer.
The inode table is an entity that maintains the operating system file itself, including the file type, size, create time, and so on
The file descriptor in the system doesn’t necessarily point to a disk file, it can point to a socket on a network and so on. From Linux’s point of view, the operating system abstracts everything into files, network data, disk data, etc., all of which are maintained using file descriptors.
With file descriptors, we have a general sense that in order for a process to read something, it must need a medium to communicate between our parent and child processes.
Then we throw out the idea of pipes, which, as the name suggests, must be used to connect two things, like a pipe in a home, one inlet and one outlet.
Let’s examine how the two processes establish communication.
As mentioned earlier, the process has its own file descriptor table. When forking the process, the parent process copies its file descriptor to the child process. Let’s take a look at some pretty bad C code. (Remember when you first started studying C in college, Pointers gave you trouble?)
Let’s take a look at the code above, we don’t need to worry about the syntax of C, just pay attention to the pipeline construction process
Pipe (fd) is an empty array of size 2. Fd [0] is the file descriptor used for reading and fd[1] is the file descriptor used for writing.
At this point, we call vfork() in the current process and create a child process that holds the fd[].
If we determine that it is the child, we close its read file descriptor, and if it is the parent, we close its write file descriptor.
At this time, as shown in the figure below, we will implement a one-way communication, the operating system calls to pipe (create) pipe, will create a new piece of memory space, the memory is special communicate with two processes, the certificate should we said above, the system will take a lot of things the abstract into files, for example here is that a piece of Shared memory abstraction to get up, The child process then writes data to the memory area through fd[1], and the parent process reads data through FD [0], thus achieving a simplex communication.
Maybe the above is a little obscure, let’s take an not entirely appropriate chestnut, you live at the head of the Yangtze River, the sister lives at the tail of the Yangtze River, the river is like a pipeline between you, you want to communicate with her between what to do? Just write a letter down the river, and she’ll be there to receive it. It’s a one-way conduit between you.
But one-way is certainly not possible, how to achieve a duplex communication, very simple, with two pipes OK.
If the above explanation does not understand, please combine with the picture below, to understand again, or add group @answer water monster, provide you with one to one private service!!
Back to where we started, how nodes communicate with each other is really just that. Node itself abstracts the concept of Libuv, which has different underlying implementations for different operating systems, such as duplex pipe communication.
Ultimate optimization – Node handle passing
To truly understand why a server can withstand high concurrency and the core of the current service architecture, you need to understand every detail, from the network to the operating system.
Now that we’ve covered a number of obscure topics, let’s get down to the nitty-gritty. What do we write the server for?
The purpose, of course, is to get someone else to call it. If you think about the way we normally call a service, the simplest one is HTTP. We make a little movie request with the browser, and the little movie server receives it and returns the result, and then we start sleepless nights.
The essence of our request is to go to a small movie server, and the port of the server receives the request and then processes it and returns the result. The most unacceptable thing is Katton when watching a small movie. For example, when watching the great Cause of the Founding of the Party, I got stuck and beat my chest for a long time while listening to XXX manifesto.
So how can the server not jammed? How does our multi-process work above?
In this scenario, the master listens on the default port 80, and all requests are sent to port 80. Other child processes listen on a different port. When the parent receives the request, it writes to the port and the child processes it.
This seems to be possible, but actually it wastes too many file descriptors. As mentioned above, each process has a file descriptor table, and each socket reads and writes based on file descriptors. The file descriptors of the operating system are limited, which is obviously not elegant and extensible.
At this point, some people may ask, why not just let each process listen to 80, why do we have to turn once? That’s a good idea.
But, it will be found that only one process will succeed in preempting the port, and other processes will throw an exception indicating that the port is occupied. To solve this problem, Node uses a different architectural pattern. The diagram below.
At the beginning, the master process still listens to 80. After receiving the user request, the master does not directly throw the data to the worker, but generates the corresponding socket after port 80 receives the data. Then, the file descriptor corresponding to the socket is transmitted to the worker through the pipeline. A socket means a data channel between the server and the client, which means that the master transmits the data channel with the client to the worker.
As shown in the figure below, the master stops listening on port 80. since the file descriptor has been given to the worker, the worker can listen on the socket directly.
Hence the following mode, where multiple workers directly listen on the same port.
At this time partners may be very confused, why this time there is no port conflict?
The key here is two points.
First, Node sets SO_REUSEADRR for each port listener, indicating that this port can be listened on by multiple processes.
The second point is that you can only use this if every process that listens on this port listens on the same file descriptor.
As mentioned in the previous section on file descriptors, the file descriptor list is private to each process and not visible to each other, so they would also have their own file descriptor for this port, which would not take advantage of the SO_REUSEADRR feature.
Then why is it ok to send to worker through master?
Because when the master communicates with the worker, each child process receives the same file descriptor (through the master, please refer to the above explanation on duplex communication if you don’t understand), at this time, all child processes listen to the same socket file descriptor. This allows multiple processes to listen on the same port.
conclusion
This article has included making https://github.com/ponkans/F2E, welcome to Star, continuously updated 💧
Node uses master/worker mode to make use of multi-core resources, and uses SO_REUSEADRR and handle (file descriptor) transfer to make multiple processes listen on the same port at the same time, improving throughput.
Having an awareness of processes, threads, and CPUS is essential so that a project can understand every line of its code.
This article is only the entry post, the real Node kernel to be studied one by one, if you have a special interest in a piece can leave a message below, directly add group to discuss, strange I wait for you! ~
– We will soon write a series on Node.js, along with the same series of portals:
-
“Punch the Interviewer” series Node.js must know, must ask
-
“Punch the interviewer” series Node.js double eleven seconds kill system
Like the small partner to add a concern, a praise oh, Thanksgiving 💕😊
Contact me/public number
Wechat search “water monster” or scan the qr code below reply “add group”, I will pull you into the technical communication group. Honestly, in this group, even if you don’t talk, just reading the chat is a kind of growth. (Ali technical experts, aobing authors, Java3y, mogujie senior front end, Ant Financial security experts, all the big names).
The water monster will also be regular original, regular with small partners to exchange experience or help read the resume. Add attention, don’t get lost, have a chance to run together 🏃 ↓↓↓
