Multiple processes

After several generations of development, cluster is now more effective. Using cluster, the sub-process worker can automatically complete the task of allocating request, so it is no longer necessary to write code to assign tasks to each worker in the master process robin style.

const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} died`);
  });
} else {
  // Workers can share any TCP connection
  // In this case it is an HTTP server
  http.createServer((req, res) => {
    res.writeHead(200);
    res.end('hello world\n');
  }).listen(80);
}
Copy the code

The above simple code realizes the creation of multiple workers according to the number of cpus. As for the actual project, it depends on the situation whether one worker should be checked or 2 or 3 workers should be checked. If it takes a long time for other servers (such as databases) to respond in a project, it would be better to have more than 2 workers.

But generally speaking, one CPU is fine for one worker.

 

So, the architecture looks something like this:

 

What the Master process needs to do is to monitor the life cycle of the worker. If the worker is found to be down, it will restart the worker and prepare the corresponding log.

There are no major challenges to the architecture, just details such as reboots, logging, 5 second heartbeat packets, etc.

Multi-process architecture, compared to the original single-process + PM2 restart benefits are much more, the entire Node service will be more stable, will not suddenly completely hang.

In addition, compared with the PM2 multi-process, it also has advantages. The logic of the master is in the hands of the developer, and the developer can customize the log and email and SMS alarms.

Use curl 127.0.0.1:12701 to send a simple HTTP GET request to the master nodeJS server. For example, use curl 127.0.0.1:12701 to send a simple HTTP GET request to the master nodeJS server.

For example, if XXX passes heapdump, it can be used as a command to capture memory snapshots.

 

Load balancing

When it comes to multi-processing, the goal is to maximize the use of multi-core cpus and increase the load capacity of a single machine.

However, in practical projects, the load of all processes is not completely balanced due to the influence of the processing time of business logic and the scheduling of system CPU.

Officials also said:

This basically means that 70% of requests end up on 2 workers, and these 2 workers consume more CPU resources.

Then in actual project deployment, we can try a further step: binding the CPU. With 4-core CPU, we fork out 4 workers, and each worker is bound to CPU #1-#4 respectively.

Node doesn’t provide a ready-made interface, but you can use the Linux command taskset

In Node, we can use child_process to execute the shell.

cp.exec('taskset -cp ' + (cpu) + ' ' + process.pid,{ timeout: 5000 },function(err,data,errData){ if(err){ logger.error(err.stack); } if(data.length){ logger.info('\n' + data.toString('UTF-8')); } if(errData.length){ logger.error('\n' + errData.toString('UTF-8')); }});Copy the code

 

According to the actual situation, the effect is good.

Smooth restart

Every time a new version is released, the server must be restarted.

Simply kill the main process, restart it all, and there will be a period of service interruption.

For small businesses, ok, can be arranged in the early morning restart, but for large companies, large products, can not be so rude.

Then we need to smooth the restart so that the service is not interrupted during the restart.

The strategy isn’t complicated, but it works:

1. Worker processes take turns to restart at intervals;

2. The worker process does not restart directly, but closes new request listening first, and then restarts after all current requests return.

  try {
        // make sure we close down within 30 seconds
        var killtimer = setTimeout(() => {
          process.exit(1);
        }, 30000);

        // stop taking new requests.
        server.close();

        // Let the master know we're dead.  This will trigger a
        // 'disconnect' in the cluster master, and then it will fork
        // a new worker.
        cluster.worker.disconnect();

      } catch (er2) {
      }
Copy the code

After a smooth restart is implemented, the throughput of the server will be much smoother.