Preliminary knowledge
Before learning cluster, you need to know the knowledge related to process. If you do not know it, you are advised to read the process module and child_process module first.
Cluster uses the fork() method of the child_process module to create child processes. An IPC channel is established between the child processes created by fork and their parent processes, supporting bidirectional communication.
The Cluster module first appeared in Node.js v0.8
Why does the Cluster module exist?
Node.js is single-threaded, so if you want to take advantage of the server’s multi-core resources, you should create multiple processes that provide services. If you use the following methods to start multiple services, a message is displayed indicating that the port is occupied.
const http = require('http');
http.createServer((req, res) => {
res.writeHead(200);
res.end('hello world\n'); }).listen(8000); // Start the first service node index.js & // start the second service node index.js & throw er; // Unhandled'error'event ^ Error: listen EADDRINUSE :::8000 at Server.setupListenHandle [as _listen2] (net.js:1330:14) at listenInCluster (net.js:1378:12) at Server.listen (net.js:1465:7) at Object.<anonymous> (/Users/xiji/workspace/learn/node-basic/cluster/simple.js:5:4) at Module._compile (internal/modules/cjs/loader.js:702:30) at Object.Module._extensions.. js (internal/modules/cjs/loader.js:713:10) at Module.load (internal/modules/cjs/loader.js:612:32) at tryModuleLoad (internal/modules/cjs/loader.js:551:12) at Function.Module._load (internal/modules/cjs/loader.js:543:3) at Function.Module.runMain (internal/modules/cjs/loader.js:744:10)Copy the codeThere is no problem if you use cluster instead
const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
// Fork workers.
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
});
} else {
// Workers can share any TCP connection
// In this case it is an HTTP server
http.createServer((req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
console.log(`Worker ${process.pid}started`); } // node index.js starts a main process and 8 child processes (the number of child processes is the same as the number of CPU cores). Master 11851 is running Worker 11852 started Worker 11854 started Worker 11853 started Worker 11855 started Worker 11857 started Worker 11858 started Worker 11856 started Worker 11859 startedCopy the codeHow does cluster implement multi-process port sharing?
A cluster can create two types of processes: a parent process and a child process. The parent process has only one and the child process has multiple processes (usually created based on the number of CPU cores).
- The parent process is responsible for listening for the port to accept the request and then distribute it.
- The child process is responsible for processing the request.
There are three questions to answer:
- Why does listen call not bind ports for child processes
- When the parent process created the TCP Server
- How does the parent process complete distribution
Why does listen call not bind ports for child processes?
The Listen method in net.js source code is used to distinguish the parent process from the child process through the listenInCluster method. The differences of different processes are reflected in the listenInCluster method
function listenInCluster(server, address, port, addressType, backlog, fd, excluseive) {
if (cluster.isMaster || exclusive) {
server._listen2(address, port, addressType, backlog, fd);
return; } const serverQuery = { address: address ...... }; cluster._getServer(server, serverQuery, listenOnMasterHandle);functionlistenOnMasterHandle(err, handle) { server._handle = handle; server._listen2(address, port, addressType, backlog, fd); }}Copy the codeWhen the child process calls the LISTEN method, it first executes _getServer, then specifies the value of server._handle via callback, and then calls the _listen2 method.
cluster._getServer = function(obj, options, cb) {
...
const message = util._extend({
act: 'queryServer',
index: indexes[indexesKey],
data: null
}, options);
message.address = address;
send(message, (reply, handle) => {
if (handle)
shared(reply, handle, indexesKey, cb); // Shared listen socket.
elserr(reply, indexesKey, cb); // Round-robin. }); . };Copy the codeThe _getServer method sends a queryServer message to the main process, and the parent process calls the callback function when it finishes. The parent process calls the shared method or rr method based on whether it returns handle. The main function of the RR method is to forge TCPWrapper to call the Net listenOnMasterHandle callback
function rr(message, indexesKey, cb) {
var key = message.key;
function listen(backlog) {
return 0;
}
function close() {
if (key === undefined)
return;
send({ act: 'close', key });
delete handles[key];
delete indexes[indexesKey];
key = undefined;
}
function getsockname(out) {
if (key)
util._extend(out, message.sockname);
return 0;
}
const handle = { close, listen, ref: noop, unref: noop };
handles[key] = handle;
cb(0, handle);
}
Copy the codeSince the child’s server gets a fake TCPWrapper and does nothing when it calls LISTEN, it does not bind ports and therefore does not report an error.
When the parent process created the TCP Server
When the child process sends queryServer message to the parent process, the parent process will detect whether TCP Server has been created. If not, the parent process will create TCP Server and bind the port, and then record the child process for subsequent users to request worker distribution.
How does the parent process complete distribution
The parent process is bound with a port number, so it can capture the connection request. The onConnection method of the parent process will be triggered, and the TCP object parameter will be passed when the onConnection method is triggered. Since the parent process recorded all the workers before, the parent process can select the worker to process the request. Then, the child process sends the message with act as newCONN to the worker and passes the TCP object. After listening to the message, the child process encapsulates the TRANSMITTED TCP object into a socket, and then triggers the Connection event. This enables the child process to handle user requests without listening on the port.
How does a Cluster implement load balancing
Load balancing relies directly on Cluster’s request scheduling policy, which prior to V6.0 was cluster.sched_none (OS dependent), SCHED_NODE performs best on paper (Ferando Micalli wrote about Cluster and Iptables and Nginx performance in Node.js 6.0 here) but in practice, There will be uneven request scheduling (perhaps two or three of the eight child processes will process 70% of the connection requests). So cluster.sched_rr (round robin) was added to Node.js in version 6.0 and is now the default scheduling policy (except on Windows)
Scheduling policies can be modified by setting the NODE_CLUSTER_SCHED_POLICY environment variable
NODE_CLUSTER_SCHED_POLICY='rr'
NODE_CLUSTER_SCHED_POLICY='none'
Copy the codeOr set the schedulingPolicy property of the cluster
cluster.schedulingPolicy = cluster.SCHED_NONE;
cluster.schedulingPolicy = cluster.SCHED_RR;
Copy the codeNode. Js round – robin
Node.js internally maintains two queues:
- The free queue records the currently available workers
- Handles queues record TCP requests that need to be processed
When a new request arrives, the parent process temporarily holds the request in the handles queue, enlists one worker from the free queue, and enters the worker processing (Handoff) phase. The key logic is as follows:
RoundRobinHandle.prototype.distribute = function(err, handle) {
this.handles.push(handle);
const worker = this.free.shift();
if(worker) { this.handoff(worker); }};Copy the codeIn the worker processing phase, a request is processed from the Handles queue, and then the child worker is notified to process the request through the process communication. When the worker receives the communication message, it sends an ACK message and continues to respond to the request in the Handles queue. When the worker cannot accept the request, the worker sends an ACK message to the child worker. The parent process is responsible for rescheduling the worker for processing. The key logic is as follows:
RoundRobinHandle.prototype.handoff = function(worker) {
const handle = this.handles.shift();
if (handle === undefined) {
this.free.push(worker); // Add to ready queue again.
return;
}
const message = { act: 'newconn', key: this.key };
sendHelper(worker.process, message, handle, (reply) => {
if (reply.accepted)
handle.close();
else
this.distribute(0, handle); // Worker is shutting down. Send to another.
this.handoff(worker);
});
};
Copy the codeNote: IPC is established between the main process and its children, so the main process can communicate with its children, but the children are independent of each other (cannot communicate).
The resources
medium.com/@fermads/no…