Optimize code using Web Workers

Some time ago, THERE was a demand for front-end export excel. In general, it’s best to leave the function of exporting large amounts of data to the back end, but the back end doesn’t want to do it (and loses the argument) and has to do it by itself.

Front-end export Excel itself has a very mature library, such as JS-XLSX, Js-export-Excel, so it is not difficult to implement. However, when the exported data reaches tens of thousands of entries, you will find that the page has a significant lag. The reason is simple: We usually generate Excel based on json data returned from the back end, but the data returned from the back end cannot be used to generate data directly. We need to do some formatting:

const list = await request('/api/getExcelData');

const format = list.map((item) = > {
  // Format the returned JSON data
  item.time = moment(item.time).format('YYYY-MM-DD HH:mm');
  / /... Omit various other operations
});

// Generate Excel from JSON
const toExcel = new ExportJsonExcel(format).saveExcel();
Copy the code

Caton occurs when a map operation is performed on a large amount of data. As JS is single-threaded, it will monopolize the main thread during a large number of complex operations, resulting in other events on the page cannot respond in time, resulting in the phenomenon of page suspension.

Can we put complex loops in a single thread? This is where the Web worker comes in

Web Worker

Let’s start with a simple example

<button id="btn1">js</button>
<button id="btn2">worker</button>
<input type="text">
Copy the code

index.js

const btn1 = document.getElementById('btn1');

btn1.addEventListener('click'.function () {
    let total = 1;

    for (let i = 0; i < 5000000000; i++) {
      total += i;
    }
    console.log(total);
})
Copy the code

When you click on btn1, js will do a lot of calculations and you’ll see that the page freezes. Clicking on Input will not do anything

We use the Web Worker to optimize the code:

worker.js

onmessage = function(e) {
  if (e.data === 'total') {
    let total = 1;

    for (let i = 0; i < 5000000000; i++) { total += i; } postMessage(total); }}Copy the code

index.js

if (window.Worker) {
  const myWorker = new Worker('worker.js');

  myWorker.onmessage = function (e) {
    console.log('total', e.data);
  };

  const btn1 = document.getElementById('btn1');
  const btn2 = document.getElementById('btn2');

  btn1.addEventListener('click'.function () {
    let total = 1;

    for (let i = 0; i < 5000000000; i++) {
      total += i;
    }

    console.log('total', total);
  })

  btn2.addEventListener('click'.function () {
    myWorker.postMessage('total');
  });

}
Copy the code

When you click btn2, the page does not freeze and you can enter the input normally

We enable a single worker thread for complex operations, communicating between the two threads via postMessage and onMessage.

Optimize export excel table

Having seen the previous example, we can use web workers for complex map operations in the same way

worker.js

onmessage = function(e) {
  const format = e.data.map((item) = > {
  // Format the returned JSON data
  item.time = moment(item.time).format('YYYY-MM-DD HH:mm');
  / /... Omit various other operations
});

postMessage(format);
}
Copy the code

const myWorker = new Worker('worker.js');

myWorker.onmessage = function (e) {
  // Generate Excel from JSON
  const toExcel = new ExportJsonExcel(e.data).saveExcel();
};
const list = await request('/api/getExcelData');
myWorker.postMessage(list);
Copy the code

Of course, the actual project is usually packaged with Webpack, and some special processing is needed to use worker-loader. You can refer to the article “How to Use Web Worker in ES6+ WebPack” for learning.

Further optimization

In the code changes above, we just optimized the map operations in the business logic. Because THE JS library I use is JS-export-Excel, it can be seen from its source code that for the data we pass in, it will also be a forEach loop operation again for binary data conversion. Therefore, the forEach loop of this step can theoretically be operated in the Web worker.

The easiest way to think about it is:

worker.js

onmessage = function(e) {
  const format = e.data.map((item) = > {
    // Format the returned JSON data
    item.time = moment(item.time).format('YYYY-MM-DD HH:mm');
    / /... Omit various other operations
  });

  // Generate Excel directly in worker
  const toExcel = new ExportJsonExcel(format).saveExcel();
}
Copy the code

Generate Excel directly from worker.js. However, the saveExcel method needs to use the Document object, but in worker, we cannot access global objects like Window Document.

Therefore, can only magic change the source code…

The actual use of the document object is the source line:

// saveAs and Blob use document
saveAs(
  new Blob([s2ab(wbout)], {
    type: "application/octet-stream"
  }),
  _options.fileName + ".xlsx"
);
Copy the code

The saveExcel method simply changes to:

// Do not generate Excel, only return data
return s2ab(wbout);
Copy the code

worker.js

onmessage = function(e) {
  const format = e.data.map((item) = > {
    // Format the returned JSON data
    item.time = moment(item.time).format('YYYY-MM-DD HH:mm');
    / /... Omit various other operations
  });

  // saveExcel returns only BLOB data
  const blob = new ExportJsonExcel(format).saveExcel();
  postMessage(blob);
}
Copy the code

index.js

myWorker.onmessage = function (e) {
  // Generate Excel in the main thread
  saveAs(
    new Blob([e.data], {
      type: "application/octet-stream"
    }),
   "test.xlsx"
  );
};
Copy the code

The principle is: we only put the data conversion in the worker, and the final generation of Excel is still done in the main thread.

At this point, optimization is complete!

conclusion

We can place some performance intensive operations (such as large file uploads) in the worker thread so that the main thread can respond to user actions without stuttering. It should be noted that the running time of complex calculation in worker does not become shorter, and sometimes even longer. After all, starting worker also requires certain performance consumption.