This is the fifth day of my participation in the August More text Challenge. For details, see: August More Text Challenge
Worker Threads
Multithreading in Node is a new feature introduced in Node VERSION V10.5.0. Worker Thread has been an experimental feature for a long time. So far, the stable version of Node has reached 14.17.4, and this feature has become stable and available.
What is worker_thread
Let’s start with the description on the website
The `worker_threads` module enables the use of threads that
execute JavaScript in parallel.
Copy the codeThe worker_Thread module allows JavaScript to be executed in parallel using threads.
We used to have Javascript that was single threaded and then we had an event loop that was constantly listening for functions coming in from the execution stack. For worker_Thread, it can be understood that there are multiple javascript worker threads in an event loop. Creating a thread is equivalent to creating a new JS execution environment. Multithreading in action is shown below
And child_process
Child_process can create a new node process. The difference between worker_thread and child_process is that worker_thread can share memory, and common data can be shared between threads. Child_process can only pass data through JSON.
Also, since threads are within a process, creating a thread is less expensive than creating a process
Worker_thread Range
Worker_thread is useful for CPU-intensive JS operations, but it doesn’t provide much performance improvement for IO intensive operations. Instead, Node’s built-in one-step IO operations are more useful than worker threads.
Using worker_thread
Having introduced the concept of worker_thread, it’s time to introduce its use
Creating a worker thread
const { isMainThread, Worker, parentPort } = require('worker_threads');
if (isMainThread) {
createChildThread();
} else {
parentPort.on('message'.(msg) = > {
console.log('parent thread listen:', msg);
})
parentPort.postMessage('hello child thread')}// Create the thread method
function createChildThread() {
const worker = new Worker(__filename)
worker.on('message'.(val) = > {
console.log('child thread listen:', val);
worker.postMessage('hello parent thread'); })}Copy the codeThe above code calls the method of creating a worker thread when the main thread is created, creates a worker thread and adds another one. After the worker thread is created, it calls the port of the main thread to send a message to the worker thread, and the worker thread receives the message and responds to the main thread. The general flow chart is as follows
Return after execution
Before creating a thread
After a thread is created, the number of threads is +1
Interthread communication
As mentioned above, worker_Threads can be shared through ArrayBuffer or SharedArrayBuffer. Let’s take a look at how this is implemented in code.
const { isMainThread, Worker, parentPort } = require('worker_threads');
if (isMainThread) {
createChildThread();
} else {
// Create a SharedArrayBuffer of 4 bytes
const shareBuf = new SharedArrayBuffer(4);
// Create an array of 8-bit unsigned integers
const bufInt = new Uint8Array(shareBuf);
parentPort.on('message'.() = > {
console.log('parent thread listen:', bufInt);
});
// Send the shared memory created integer
parentPort.postMessage({ bufInt });
}
// Create the thread method
function createChildThread() {
const worker = new Worker(__filename);
worker.on('message'.({ bufInt }) = > {
console.log('child thread listen:', bufInt);
bufInt[0] = 11;
worker.postMessage('finished');
});
}
Copy the codeWhen you return from running, you can see that you created one in the child threadSharedArrayBufferIt is a shared Buffer between threads, so the main thread can also see the data modified in the child thread.
As shown in the figure, useSharedArrayBufferThe created value is allocated to shared memory, which can be shared by all threads.
Interthread communication
We have learned to create a thread and the use of Shared memory, you can see from the above code, the thread is sending a message from the main thread, and then the child thread to the main thread to reply message, there is no way to let two child threads directly communication, if I want two child thread communication directly, that need to use MessageChannel this class, The detailed use of MessageChannel can be found here. Now let’s implement direct communication between threads at once.
const {
isMainThread,
Worker,
parentPort,
MessageChannel,
threadId,
} = require('worker_threads');
if (isMainThread) {
createChildThread();
} else {
parentPort.on('message'.({ port }) = > {
// Configure the communication port method after the main thread receives the port
port.on('message'.(msg) = > {
console.log(`port${threadId} listen:`, msg);
});
port.postMessage(`hello, im thread ${threadId}`);
});
parentPort.postMessage('hello');
}
// Create the thread method
function createChildThread() {
const { port1, port2 } = new MessageChannel(); // Create a MessageChannel
const worker1 = new Worker(__filename); // Create child thread 1
const worker2 = new Worker(__filename); // Create child thread 2
worker2.postMessage({ port: port1 }, [port1]); // Port 1 that sends the Channel to the main thread
worker1.postMessage({ port: port2 }, [port2]); // Port 1 that sends the Channel to the main thread
}
Copy the codeRun the code and return
As you can see from the code implementation, the final step to establish direct communication between the child threads is in the message event of the main thread. The flowchart for establishing communication is as follows
In actual combat
If you have a million pieces of data in an array that requires MD5 encryption, compare the speed of the implementation with or without a worker thread
const {
isMainThread,
Worker,
parentPort,
threadId,
} = require('worker_threads');
const { createHash } = require('crypto');
const ARR_NUM = 1000000; // Array length
const WORKER_NUM = 1; / / the number of threads
const size = Math.ceil(ARR_NUM / WORKER_NUM); // The amount of data that each thread needs to process
if (isMainThread) {
createChildThread();
} else {
parentPort.on('message'.({ status, index, startTime }) = > {
if (index === WORKER_NUM) {
const usedTime = Date.now() - startTime;
console.log(`finish bussiness time: ${usedTime}ms`); process.exit(threadId); }});const data = addHasCode(threadId, size, (threadId - 1) * size);
/ / is completed
parentPort.postMessage({
business: 'finish work',
data,
});
}
// Create the thread method
function createChildThread() {
let finishNumBuf = new SharedArrayBuffer(4);
let finishNum = new Uint8Array(finishNumBuf);
const startTime = Date.now();
for (let x = 0; x < WORKER_NUM; x++) {
const worker = new Worker(__filename, {});
worker.on('message'.({ business, data }) = > {
if (business === 'finish work') {
finishNum[0] + +; worker.postMessage({status: `finish worker ${x}`.index: finishNum[0].startTime: startTime, data, }); }}); }console.log(`${WORKER_NUM} thread start working`);
}
// Encryption method
function addHasCode(index, size, limit) {
const result = [];
for (let x = limit, num = index * size; x < num; x++) {
result.push(createHash('md5').update('hello world').digest('base64'));
}
return result;
}
Copy the codeuse1The average number of threads is calculated2700msOr so
use5The average number of threads is calculated2000msOr so
use20The average number of threads is calculated2500msOr so
use60The average number of threads is calculated3800msOr so
It can be seen that more threads is not the better, too many threads may increase too much system overhead, speed is not as good as the single thread time.
summary
This article introduced the concept of worker_Threads in nodeJs, the difference between multiple processes, and its benefits. How worker_Threads works, shared memory, and communication between child threads. Finally, we use an example to test the efficiency of a child process to show worker_Threads versus a single thread.
If there are any inaccuracies or errors in this article, please note them in the comments section
reference
- Worker Theread