The introduction

Some time ago, wrote a front-end to heavy blog. In the process of tens of millions of data, even without considering compatible selection of the most efficient way, it will take a few seconds to complete the process (the specific time is mainly affected by the data repetition rate), which will cause the result of page stuck. Because JavsScript is single-threaded, subsequent tasks wait while the data is being processed. Today to solve the task is too large, caused by the page stuck how to solve.

Single-threaded processing of big data demo

Single-threaded code removal

// Create a 5 million, duplicate 10% of the data
var arr = Array.from(new Array(5000000), function (item2, i) {
  return i % 10 < 0.1 || Math.random();
})

// remove the function
function distinct(arr) {
  const newArray = [];
  const newMap = new Map(a);for (let i = 0, len = arr.length; i < len; i++) {
    if(! newMap.get(arr[i])) {// newMap.has(arr[i])
      newMap.set(arr[i], 1); newArray.push(arr[i]); }}return newArray;
}

// Outputs the elapsed data function
function printTime(arr) {
  console.log('Data length:${arr.length}`)
  console.time("Take")
  var newArr = distinct(arr)
  console.timeEnd("Take")
  console.log('Amount of data repetition:${arr.length - newArr.length}`)}// Perform de-duplication and follow the rest of the code
printTime(arr)
console.log('De-duplicated code execution')
Copy the code

Dynamic graph demonstration

Multi-threaded processing of big data demonstration

Multithreading to redo code

// Create a 5 million, duplicate 10% of the data
var arr = Array.from(new Array(5000000), function (item2, i) {
  return i % 10 < 0.1 || Math.random();
})

// remove the function
// The function that will be put into the Worker thread, post the data after processing, and then listen on the main thread to capture it.
function distinct(e) {
  const arr = e.data;
  const newArray = [];
  const newMap = new Map(a);for (let i = 0, len = arr.length; i < len; i++) {
    if(! newMap.get(arr[i])) {// newMap.has(arr[i])
      newMap.set(arr[i], 1);
      newArray.push(arr[i]);
    }
  }
  self.postMessage(newArray); 
}

// Create Worker thread
var blob = new Blob([`
    self.addEventListener('message', ${distinct.toString()}, false);
`]); // The internal string content will be executed in the Worker thread, listening for the data sent by the main thread, and processed using the unredo function above
var url = window.URL.createObjectURL(blob);
var worker = new Worker(url);

// The main thread listens for data returned by the worker thread
worker.addEventListener('message'.function (event) {
  console.timeEnd("Take")
  console.log('Amount of data repetition:${arr.length - event.data.length}`)
  worker.terminate() // Close the Worker after using it
})

// Perform de-duplication and follow the rest of the code
console.log('Data length:${arr.length}`)
console.time("Take")
worker.postMessage(arr) // Send data to the Worker thread for processing

console.log('De-duplicated code execution')
Copy the code

Dynamic graph demonstration

conclusion

Here we mainly demonstrate how to keep the page free of lag when dealing with computation-intensive or high-latency tasks. For more ways to use Worker, you can learn from Teacher Ruan’s tutorial, or go to MDN to see the Worker API and Blob used above. The above de-duplication code uses the Map de-duplication method in this blog. Those of you who are interested can go and have a look.

Please correct any mistakes or omissions, thank you.