This article was first published on the public account of Maoyan front end team.

preface

If you use PM2 to manage node.js processes, you’ll notice that it supports a cluster mode. After the Cluster mode is enabled, you can create multiple node. js processes. If the Instances in Cluster mode are set to Max, they also create Node processes based on the number of CPU cores on the server.

In fact, PM2 uses node.js Cluster module to achieve, this module is to solve the node.js instance single thread running, can not use the advantages of multi-core CPU appeared. So how does a Cluster work internally? How do multiple processes communicate with each other? How can multiple processes listen on the same port? How does Node.js distribute requests to processes? If you’re not sure about the above, read on.

Core principles

Node.js worker processes are created by the child_process.fork() method, which also means that there are parents and children. The code looks like this:

const cluster = require('cluster');
const os = require('os');

if (cluster.isMaster) {
  for(var i = 0, n = os.cpus().length; i < n; i += 1) { cluster.fork(); }}else{// start program}Copy the code

For those of you who have studied operating systems, the system call fork() is familiar. The child and parent have the same code segment, data segment, and stack, but they do not share memory space. The parent process (the master process) listens on the port and dispatches new requests to the worker processes below. There are three issues involved: parent-child communication, load balancing policy, and port listening for multiple processes.

Note: On Linux, fork() supports copy-on-write. A copy of the parent process is copied to the child process only when the contents of each segment of the process space are changed. So the child process and the parent process start out sharing the same memory space.

Process of communication

The master process creates child processes through process.fork(), and they communicate with each other through the IPC (internal process communication) channel. The interprocess communication modes of the operating system are as follows:

  • Shared memory Different processes share the same memory space. Semaphore mechanisms are often introduced to enable synchronization and mutual exclusion.
  • Message transfer In this mode, information is synchronized between processes by sending and receiving messages.
  • Semaphore A semaphore is simply a state value assigned to a process by the system. An uncontrolled process is forced to stop at a specific place to wait for a signal that it can continue. If the semaphore has only 0 or 1, it is also called a mutex. This mechanism is also widely used in various programming modes.
  • Pipeline A pipeline is itself a process that connects two processes, taking the output of one process as the input of the other. Pipes can be created using the PIPE system call. We often use “|” command line is in using the pipeline system.

Node.js provides an event mechanism for communication between parent and child processes to pass messages. The following example implements the parent process passing TCP Server object handles to the child process.

const subprocess = require('child_process').fork('subprocess.js'); // Start the server object and send the handle. const server = require('net').createServer();
server.on('connection', (socket) => {
  socket.end('Processed by parent');
});
server.listen(1337, () => {
  subprocess.send('server', server);
});
Copy the code
process.on('message', (m, server) => {
  if (m === 'server') {
    server.on('connection', (socket) => {
      socket.end('Quilt processing'); }); }});Copy the code

Then again, if there is no parent-child relationship between processes, in other words, how do we implement communication between arbitrary processes? Check out this article: Alternative implementations of interprocess communication

Load Balancing Policy

As mentioned above, all requests are distributed through the master process, so it is necessary to ensure that the server load is evenly distributed to all worker processes, which involves the load balancing strategy. By default, Node.js adopts the round-robin time-slice rotation policy.

Round-robin is a very common load balancing algorithm, and it is also used as a load balancing strategy on Nginx. The principle is very simple, each time the request from the user is assigned to each process in turn, starting from 1 to N(number of worker processes), and then starting the loop again. The problem with this algorithm is that it assumes the same performance from process to process, or from server to server, but if the request is processed at long intervals, it can easily lead to load imbalance. So we usually use another algorithm on Nginx: WRR, weighted rotation. Assign a certain weight to each server, select the server with the largest weight each time, and reduce its weight by 1. After all the weights are 0, polling is performed according to the sequence generated at this time.

Load balancing policies can be modified by setting the NODE_CLUSTER_SCHED_POLICY environment variable or by using cluster.setupmaster (options). We can use Nginx to do load balancing on multi-machine clusters, and then use Node.js Cluster to achieve load balancing on single-machine multi-process.

Port listening for multiple processes

On the original Node.js, multiple processes listened on the same port, competing with each other for newly accepted connections. As a result, the load of each process was very uneven, so the round-robin strategy mentioned above was used later. The idea is that the master process creates the socket, binds the address and listens. The SOCKET fd is not passed to worker processes. When the master process obtains a new connection, it decides to distribute the accepted client connection to the specified worker. To put it simply, the master process listens on the port and forwards the connection to the worker process through some distribution policy (such as round-robin). Since only the master process receives client connections, the problem of load imbalance caused by contention is solved. But this design requires that the master process be stable enough.

conclusion

In this paper, PM2 Cluster Mode as a starting point, to introduce the core principle of node.js Cluster to achieve multi-process. It emphasizes three aspects: process communication, load balancing and multi-process port monitoring. Through the study of cluster module, it can be found that many underlying principles or algorithms are universal. For example, the round-robin algorithm is also used in the underlying process scheduling of the operating system. For example, master-worker architecture is also familiar in Nginx multi-process architecture; Mechanisms such as semaphores and pipes can also be found in various programming modes. There are a lot of new technologies on the market today, but the core is that all changes are the same. Once you understand the basics, the rest can be understood by analogy.

Reference links:

  1. What are we talking about when we talk about clusters (2)
  2. Node.js advanced: Cluster module in-depth analysis
  3. Alternative implementation of interprocess communication