This paper is mainly divided into two parts, the first part is the foundation and architecture of Node.js, the second part is the implementation of the core module of Node.js.

  • A Node.js infrastructure and architecture
    • The composition of the Node. Js
    • Node.js code architecture
    • Node.js startup process
    • Node.js event loop
  • Two node.js core module implementation
    • Processes and interprocess communication
    • Threads and interthread communication
    • Cluster
    • Libuv thread pool
    • The signal processing
    • file
    • TCP
    • UDP
    • DNS

1. Nodejs

Node.js consists mainly of V8, Libuv, and third-party libraries:

  1. Libuv: cross-platform asynchronous IO library, but it provides more functions than IO, including processes, threads, signals, timers, interprocess communication, thread pools, etc.
  2. Third party libraries: Asynchronous DNS parsing (CARES), HTTP parser (old version uses HTTP_Parser, new version uses LLHTTP), HTTP2 parser (nGHTTP2), Decompression and compression library (Zlib), Encryption and Decryption Library (OpenSSL), etc.
  3. V8: Implements JS parsing, execution, and support for custom extensions. V8’s support for custom extensions is the reason for Node.js.

2. Node.js code architecture

The figure above shows the code structure of Node.js. The code of Node.js is divided into JS, C++ and C:

  1. JS is the module that we normally use (HTTP/FS).
  2. The C++ code is divided into three parts, the first part encapsulates Libuv functionality, the second part is independent of Libuv (the crypto API uses Libuv thread pool), such as the Buffer module, and the third part is V8 code.
  3. The code of the C language layer mainly encapsulates the functions of the operating system, such as TCP and UDP.

Now that we know what Node.js is made of and the code architecture, let’s take a look at what node.js does when it starts.

3. Node.js startup process

3.1 registering C++ modules

Node.js first registers C++ modules by calling registerBuiltinModules. This function calls a series of registerxxx functions that we can’t find in the node.js source code. Because these functions are implemented in each C++ module through macro definition, the macro expansion is the content of the yellow box above, each registerxxx function is to insert a node into the C++ module’s linked list, and finally form a linked list.

So how do you access C++ modules in node.js? In node.js, C++ modules are accessed using an internalBinding. The logic of internalBinding is simply to find the corresponding module in the module queue based on the module name. However, this function can only be used inside Node.js, not in user JS modules, which can be accessed via process.binding.

3.2 Environment objects and binding Context

After the C++ module is registered, the Environment object is created. Environment is the Environment object of node.js execution, which acts like a global variable. It records some common data of node.js at runtime. After creating the Environment, Node.js binds this object to V8’s Context. Why? The main purpose is to get the env object in V8’s execution Context, because V8 only has the Isolate and Context objects. If we want to get the contents of the Environment object in V8’s execution Environment, You can use Context to get the Environment object.

3.3 Initializing the module loader

  1. Node.js first passes in the C++ module loader and executes loader.js. Loader.js mainly encapsulates the C++ module loader and native js module loader and saves it in the env object.
  2. We then pass in C++ and the native JS module loader and execute run_main_module.js.
  3. Regular JS and native JS module loaders are passed in run_main_module.js to execute the user’s JS.

Assume the following user JS:

  • require(‘net’)
  • require(‘./myModule’)

The user module and the native JS module are loaded separately. Let’s look at the loading process.

  1. Node.js first determines if it is a native JS module. If not, it loads the user module directly. Otherwise, it loads the native JS module using the native module loader.
  2. When loading native JS modules, use internalBinding if C++ modules are used.

3.4 Execute user code, Libuv event loop

Node.js will then execute the user’s JS. Normally, the user’s JS will give the production task to the event loop and then enter the event loop. For example, when we listen to a server, we will create a TCP Handle in the event loop. Node.js will run through the event loop.

net.createServer(() => {}).listen(80)

4. Event loops

Let’s look at the implementation of the event loop. The event cycle is divided into seven stages. The Timer stage processes timer related tasks, the pending stage processes callbacks generated in the Poll IO stage, and the Check, Prepare, and Idle stages are customized stages. Tasks in these three stages will be executed in each event sequence cycle. The Poll IO stage mainly deals with network IO, signal, thread pool and other tasks, and the closing stage mainly deals with closed handles, such as closing the server.

  1. Timer phase: binary heap implementation, the fastest expiration in the root node.
  2. Pending stage: Handles callbacks generated in the Poll IO stage callbacks
  3. Check, Prepare, and Idle phases: Each event cycle is executed.
  4. The Poll IO phase: Handles file descriptor-related events.
  5. Closing phase: Executes the callback passed in when calling the uv_close function.

Let’s look at the implementation of each phase in detail.

4.1 Timer Phase

The underlying data structure of timer is binary heap, and the node with the fastest expiration is at the top. In the timer phase, the node is traversed one by one. If the node times out, its callback is executed. If the node does not time out, the following nodes do not need to judge, because the current node is the fastest to expire. After the node’s callback is executed, it is removed, and to support the setInterval scenario, if the repeat tag is set, the node is inserted back into the binary heap.

We see that the underlying implementation is a bit simpler, but the Node.js timer module implementation is a bit more complex.

  1. Node.js maintains a binary heap at the JS layer.
  2. Each node of the heap maintains a linked list in which the longest timeout is ranked next.
  3. Node.js also maintains a map where the key is the relative timeout and the value is the corresponding binary heap Node.
  4. All nodes of the heap correspond to a timeout node at the bottom.

When we call setTimeout, we first find the binary heap Node from the map according to the input parameter of setTimeout, and then insert the end of the linked list. If necessary, Node.js will update the timeout of the underlying Node according to the fastest js binary heap timeout. As the event loop processes the timer phase, Node.js will iterate through the JS binary heap, retrieve the expired nodes, and iterate through the list of expired nodes to determine whether a callback needs to be executed one by one, adjusting the timeout of the JS binary heap and the underlying layer if necessary.

4.2 Check, Idle, and Prepare phases

The check, Idle, and Prepare phases are relatively simple. A queue is maintained in each phase, and the callback of each node in the queue is executed during the corresponding phase. However, the nodes in the queue are not deleted after the check, idle, and Prepare phases are executed, but remain in the queue unless explicitly deleted.

4.3 Pending and Closing Stages

Pending stage: Callback generated in Poll IO callback. Closing phase: Perform the callback to close Handle. The pending and closing phases also maintain a queue, then execute callbacks for each node at the corresponding phase, and finally delete the corresponding node.

4.4 Poll IO Phase

The Poll IO phase is one of the most important and complex phases, so let’s look at the implementation. First let’s look at the data structure at the core of the Poll IO phase: IO observers, which encapsulate file descriptors, events of interest, and callbacks, primarily used in ePoll.

When we have a file descriptor that needs to be listened on by epoll

  1. We can create an IO observer.
  2. Call uv__io_start to insert an IO observer queue into the event loop.
  3. Libuv records the mapping between file descriptors and IO observers.
  4. In the Poll IO phase, the IO observer queue is traversed and epoll is manipulated to do the corresponding processing.
  5. When returned from epoll, we can get which file descriptor events were triggered, and finally find the corresponding IO observer based on the file descriptor and execute his callback.

In addition, we see that the Poll IO phase may block, whether or not and for how long depends on the current state of the event loop system. When a block occurs, to ensure that the timer phase is executed on time, the ePOLL block time must be set to the time when the timer node expires the fastest.

5. Process and interprocess communication

5.1 Creating a Process

Processes in Node.js are created using fork+exec. Fork copies the data of the main process, and exec loads the new program to execute. Node.js provides both asynchronous and synchronous process creation modes.

  1. asynchronous

In asynchronous mode, after a process is created, the main process and child processes run independently. In the data structure of the master process, as shown in the figure, the master process records information about the child process, which is used when the child process exits

  1. synchronously

Synchronously creating a child causes the main process to block

  1. A new event loop structure is created in the main process, and a child process is created based on the new event loop.
  2. The main process then executes in the new event loop, and the old event loop is blocked.
  3. When the child process terminates, the new event loop ends and the old event loop returns.

5.2 Inter-process Communication

Now let’s look at how parent processes communicate with each other. In the operating system, virtual addresses between processes are independent, so there is no way to communicate directly based on process memory, which requires kernel memory. There are many ways to communicate between processes: pipes, signals, shared memory, and so on.

Node.js selects the Unix domain for interprocess communication. Why does Node.js select the Unix domain? Because file descriptor passing is supported only by Unix domains, file descriptor passing is a very important capability.

In an operating system, when a process opens a file, it forms a fd->file->inode, which is inherited when a child process is forked.

But what if the main process opens a file after forking the child and wants to tell the child? If you simply pass the number corresponding to the file descriptor to the child process, the child process has no way of knowing the file corresponding to that number. If sent over a Unix domain, the system copies the file descriptor to the child process as well.

The specific implementation

  1. Node.js creates two file descriptors using socketpair. The main process takes one of the file descriptors and encapsulates the send and ON meesage methods for inter-process communication.
  2. The main process then passes another file descriptor to the child process via an environment variable.
  3. Child processes also encapsulate interfaces that send and receive data based on file descriptors. Then the two processes can communicate with each other.

6. Threads and interthread communication

6.1 Threading Architecture

Node.js is single-threaded. In order to make it easier for users to handle time-consuming operations, Node.js supports multi-threading in addition to multi-process. The architecture of multiple threads in Node.js is shown below. Each child thread is essentially an independent event loop, but all threads share the underlying Libuv thread pool.

6.2 Creating a Thread

Let’s look at the process of creating a thread.

When we call the new Worker to create the thread

  1. The main thread first creates two data structures for communication, and then sends a message to the peer to load the JS file.
  2. A thread is then created by calling the underlying interface.
  3. The child thread is then created and initializes its own execution environment and context.
  4. Then read the message from the communication data structure, and then load the corresponding JS file to execute, and finally enter the event cycle.

6.3 Communication between threads

So how do threads in Node.js communicate? Threads are different from processes in that the address space of a process is independent and cannot communicate directly, but the address of a thread is shared, so it can communicate directly based on the memory of the process.

Let’s take a look at how Node.js implements interthread communication. Before we look at node.js communication between threads, let’s take a look at some core data structures.

  1. Message stands for a Message.
  2. MessagePortData encapsulates the operation Message and carries the Message.
  3. MessagePort represents the endpoint of communication and encapsulates MessagePortData.
  4. MessageChannel represents the two ends of communication, namely the two Messageports.

We see that the two ports are related to each other, and when a message needs to be sent to the peer, we simply insert a node into the peer message queue. Let’s look at the specific process of communication

  1. Thread 1 calls postMessage to send the message.
  2. PostMessage serializes the message first.
  3. It then takes the lock on the peer message queue and inserts the message into the queue.
  4. Once the message is successfully sent, the thread on which the message receiver belongs needs to be notified.
  5. The message receiver processes the message during the Poll IO phase of the event loop.

7. Cluster

We know that Node.js is a single-process architecture and does not take advantage of multi-core architecture. The Cluster module enables Node.js to support a multi-process server architecture. Node.s supports polling (main process Accept) and sharing (child process Accept) modes, which can be set using environment variables. The multi-process server architecture usually has two modes. The first mode is that the main process processes the connection and then distributes the connection to the child process. The second mode is that the child process shares the socket and obtains the connection for processing through competition.

Let’s take a look at how the Cluster module is used.

This is an example of using the Cluster module

  1. The main process calls fork to create the child process.
  2. The child process starts a server. In general, multiple processes listening on the same port will cause an error. Let’s see how node.js handles this.

7.1 Main process Accept

Let’s first look at the accept mode for the main process.

  1. First, the main process forks several child processes.
  2. Listen is then called in each child process.
  3. When listen is called, the child sends a message to the main process.
  4. The main process creates a socket, binds the address, and puts it in the listening state.
  5. When a connection arrives, the main process is responsible for receiving the connection and then distributing it to the child process via file descriptor passing.

7.2 Subprocess Accept

Let’s take a look at the process Accept pattern.

  1. First, the main process forks several child processes.
  2. Listen is then called in each child process.
  3. When listen is called, the child sends a message to the main process.
  4. The main process creates a socket and binds the address. Instead of putting it in a listening state, the socket is returned to the child process as a file descriptor.
  5. When a connection arrives, it is processed by one of the child processes.

8. Libuv thread pool

Why use thread pools? File IO, DNS, and CPU intensive tasks are not suitable to be handled in the main thread of Node.js. These tasks need to be handled in child threads.

Before we get to the thread pool implementation, let’s look at Libuv’s asynchronous communication mechanism. Asynchronous communication refers to the communication mechanism between Libuv’s main thread and other child threads. For example, the Libuv main thread is executing a callback, and the child thread completes a task at the same time, so how to notify the main thread, this needs to use the asynchronous communication mechanism.

  1. Libuv maintains an asynchronous queue, into which an Async node is inserted when asynchronous communication is needed
  2. Libuv also maintains an IO observer for asynchronous communication
  3. When an asynchronous task is completed, the pending field of the corresponding Async node is set to 1, indicating that the task is completed. And notify the main thread.
  4. The main thread performs callbacks to handle asynchronous communication during the Poll IO phase. In the callback, the node whose pending is 1 is executed.

Let’s look at the thread pool implementation.

  1. A thread pool maintains a queue of pending tasks from which multiple threads mutually remove tasks for processing.
  2. When a task is submitted to the thread pool, a node is inserted into the queue.
  3. When the child thread finishes processing a task, it inserts the task into the event loop itself to maintain a completed task queue, and notifies the main thread through asynchronous communication.
  4. The main thread performs the corresponding callback during the Poll IO phase.

9. The signal

Above is a representation of signals in an operating system that uses a long type to represent the information received by the process and an array to mark the corresponding handler. Let’s look at how the signal module is implemented in Libuv.

  1. Libuv maintains a red-black tree that inserts a new node whenever we listen for a new signal
  2. When the first node is inserted, Libuv encapsulates an IO observer registered with ePoll to listen for signals that need to be processed
  3. When a signal occurs, the corresponding Handle is found in the red-black tree based on the signal type and notified to the main thread
  4. The main thread performs callbacks one by one during the Poll IO phase.

In Node.js, signals are listened for by listening for events called newListener, which is a mechanism of hooks. Every time an event is listened on, if the newListener event is listened on, the newListener event will be raised. So when process.on(‘ SIGINT ‘) is executed, startListeningIfSignal (the handler of the newListener event) is called to register a red-black tree node. The subscription is saved in the Events module, and when the signal is triggered, process.emit(‘ SIGINT ‘) is executed to notify the subscriber.

10. The file

10.1 File Operations

In Node.js, file operations are divided into synchronous and asynchronous modes. The synchronous mode refers to directly calling the API of the file system in the main process, which may cause the process to block. The asynchronous mode uses the Libuv thread pool to transfer the blocking operation to the sub-thread, and the main thread can continue to process other operations.

10.2 File Monitoring

File listening in Node.js provides two modes, polling and subscribe-based publishing. Let’s take a look at the implementation of polling mode, polling mode is relatively simple, he is using a timer to achieve, Node.js will periodically perform a callback, in the callback to compare the current file metadata and the last obtained is not the same, if so, it indicates that the file has changed.

The second listening mode is the more efficient inotify mechanism, which is subscription-based and avoids inefficient polling. Let’s start by looking at the inotify mechanism of the operating system. Inotify and epoll are used similarly:

  1. The file descriptor for an inotify instance is first obtained through the interface.
  2. The inotify instance is then manipulated by adding, deleting, modifying, and reviewing the interface. For example, when you need to listen for a file, the interface is called to add a subscription to the inotify instance.
  3. When a file changes, we can call the read interface to find out which files have changed, and inotify is usually used in conjunction with epoll.

Let’s look at how file listening is implemented in Node.js based on the inotify mechanism.

  1. Node.js first registers inotify instance file descriptors and callbacks as IO observers in ePoll
  2. When listening for a file, Node.js calls a system function to insert an entry into the inotify instance and get an ID. Node.js then encapsulates the ID and file information into a structure and inserts it into the red-black tree.
  3. Node.js maintains a red-black tree, where each Node records a list of listened files or directories and callbacks when events are triggered.
  4. If an event is triggered, the corresponding callback will be executed in the Poll IO stage. In the callback, it will determine which files have changed, and then find the corresponding interface from the red-black tree according to the ID, so as to execute the corresponding callback.

11. TCP

We typically start a server with a call to http.createserver (cb).listen(port), so what does this process actually do? Listen encapsulates the network API:

  1. First get a socket.
  2. Then bind the address to the socket.
  3. Then call listen to change the socket to listen state.
  4. Finally, register the socket with epoll and wait for the connection to arrive.

So how does Node.js handle connections? When a TCP connection is established, Node.js performs the corresponding callback in the Poll IO phase:

  1. Node.js will call Accept to take down a TCP connection.
  2. It then calls the C++ layer, which creates an instance of an object representing communication with the client.
  3. The JS layer is then called back, and JS will also create an object representing an instance of the communication, mainly for use by the user.
  4. Finally, register to wait for readable events, waiting for the client to send data.

This is how Node.js processes a connection. After a connection is processed, Node.js will determine whether single_ACCEPT is set. If so, it will sleep for a while and let other processes process the rest of the connection to avoid responsibility imbalance to some extent. Node.js will continue to attempt to process the next connection. This is how Node.js handles connections.

12. UDP

Because UDP is a non-connected and unreliable protocol, it is relatively simple to implement and use. Here is the process of sending UDP data. When we send a UDP packet, Libuv will first insert the data into the waiting queue, and then register in epoll to wait for writable events. When a writable event is triggered, Libuv will traverse the waiting queue and send each node one by one. After a successful send, Libuv will move the node to the send success queue and insert a node into the pending phase. Libuv then performs the send completion call for each node in the queue to notify the caller that the send is finished.

13. DNS

Because the API for looking up IP by domain name or domain name by IP is blocking, both functions are implemented with Libuv’s thread pool. When a lookup is initiated, Node.js mentions a task to the thread pool and proceeds to do other work, while the subthreads of the pool call the underlying function to perform a DNS query. When the query is complete, the subthreads pass the result to the main thread. So that’s the whole search.

Other DNS operations are implemented through CARES, which is an asynchronous DNS library. We know that DNS is an application layer protocol, and CARES implements this protocol. Let’s take a look at how Node.js uses Cares to implement DNS operations.

  1. First, when Node.js is initialized, the Cares library is initialized, the most important of which is setting up the callback for socket changes. We’ll see how this callback works in a moment.
  2. When we initiate a DNS operation, Node.js calls the Cares interface, which creates a socket and initiates a DNS query, then passes the socket to Node.js via a state change callback.
  3. Node.js registers the socket with epoll and waits for the query result. When the query result is returned, Node.js calls the cares function to parse it, and finally invokes the JS callback to notify the user.

14. To summarize

This article introduces the implementation of Node.js as a whole, and also introduces the implementation of some core modules. From this article, we’ve also seen a lot of low-level content. Node.js is the JS runtime created by combining V8 with the capabilities of the operating system. By understanding the principle and implementation of Node.js in depth, you can better use Node.js.

For more information: github.com/theanarkh/u…