Introduction to the


  • Puppeteer is configured on the server
  • Screenshots of key code implementation
  • Screenshot Service pressure test

Rely on

  • puppeteer
  • nestjs

  • pm2

background

🐟 has been developing and optimizing a front-end heavy project of the company for more than a year, which is similar to a webpage version of PPT (as shown below, here keynote is a demonstration, after all, the internal project of the company can not be revealed). It has many functions, and performance has been the bottleneck of the project that is difficult to break through 😿. With the business function from 0 to 1, Functions have been satisfied with the business of the company. From 11 students in the front end of the group, there are only 4 students (plus me) to maintain the products. The leaders have changed one after another. Without further ado, in order to display the outline on the left as an image, reduce the DOM amount of the page, and images can be used for many purposes, such as PDF generation, preview, etc. So a screenshot service was born



Pros and cons of front-end and server screenshots

Front-end html2Canvas and other frameworks can also be used in front-end pages for HTML screenshots

Advantages: Screenshots run on the client side, there is no need to worry about server resources and concurrency and other related problems, it is an easy project for pure front-end partners.

Disadvantages: The first disadvantage is that html2Canvas (based on foreignObject and canvas) is cross-domain and some elements that cannot be serialized cannot generate the correct screenshots; The second disadvantage is that JS is single-threaded, which will occupy the main thread when taking a screenshot, making the page into a state of suspended animation and unable to carry out UI operations.

The server invokes the Chromium screenshot method using puppeteer

Advantages: Compared with the front-end implementation, the interface request server is an asynchronous event and does not occupy the main thread resources.

Disadvantages: Screenshot service generation requires rendering time, and the time depends on the complexity of the page, so the screenshot cannot be generated synchronously. The problem with asynchronously generating screenshots is that the client periodically polls for the presence of the resource (of course you can generate screenshots using a long connection) and then displays it. And consider server concurrency and task scheduling.

Preparations for the server

Install the puppeteer – core

Find the chromium version you want to use on the Puppeteer website github.com/puppeteer/p…

I’m using “Puppeteer “: “3.3.0” for Chromium version 83

How do I download the chromium in Puppeteer

The download link is at puppeteer\ SRC \ browserfetcher.js, as shown below:





The corresponding version is in package.json, as shown below:



Complete links

https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/756035/chrome-linux.zipCopy the code

Server Configuration Chromium

Download the chromium

wget https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/756035/chrome-linux.zip
Copy the code

Detect missing dependency packages

ldd chrome | grep notCopy the code

If the dependency package is missing, see the following figure



Then run the following installation:

# dependent libraries
yum install pango.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXi.x86_64 libXtst.x86_64 cups-libs.x86_64 libXScrnSaver.x86_64 libXrandr.x86_64 GConf2.x86_64 alsa-lib.x86_64 atk.x86_64 gtk3.x86_64 -y
# font
yum install ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc -yCopy the code


Commissioning chromium

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch({ headless: false,
        executablePath : '/home/homework/local-chrome/chrome-linux/chrome'
   });
  const page = await browser.newPage();
  await page.goto("https://google.com");
  await page.screenshot({ path: "example.png"}); await browser.close(); }) ();Copy the code


Screenshots of key code implementation

The process is introduced



Preloading logic

To save Browser OpenPage time, preload 10 page instances;

let ready = false;
let domCaptureBrowser: CreateBrowserPage;
(async () => {
  const time = +new Date().getTime();
  const filePath = 'domcapture.html';
  domCaptureBrowser = new CreateBrowserPage(10);
  const init = await domCaptureBrowser.init();
  if(! init) { Log({data:'Initialize DomCapture preopen error'.logType: 'error'});
    return; } const html = await getFile({serverWebroot: config.serverWebroot, filePath}); const htmlFile = path.join(config.outputPath, `/domcapture.html`); try { fs.writeFileSync(htmlFile, html); } catch (error) {Log({data: 'init domCapture preopen HTML write error',logType: 'error'});
  }
  await (async () => {
    const len = domCaptureBrowser.pageList.length;
    for (let i = 0; i < len; i++) {
      try {
        await domCaptureBrowser.pageList[i].page.goto(`${config.serverProtocol}${htmlFile}`, {
          waitUntil: ['load'.'networkidle0'], timeout: 1000 * 3 * 60, }); } catch (error) {Log({data: 'Initializes domCapture preopen page No.${i} error`, logType: 'error'}); }}}) (); ready =true; Log({data: 'initializing domCapture preopen success, newPage time:${(+ new Date().getTime() - time) / 1000}s`}); }) ();export {
  ready,
  domCaptureBrowser,
};Copy the code


Screenshot Service pressure test

Stress tests use stress, a service built by the company’s testing gurus.

It is recommended to test the service, because the local test single point does not trigger concurrency problems, because the code page scheduling problem was still found during the test.



Finally, show the log generation



conclusion

Generally speaking, it was a very simple process to develop and build the screenshot service. However, there were many detours when building the Node service for the first time. For example, the server was not connected to the network. The variable was not configured, some configurations of the newly applied machine were not initialized, and finally I sought op to solve these problems, but found that these problems wasted some time, and pM2 multi-process caused taskId generation exception, etc.

Pm2 and the use of NestJS will be covered in other articles, thanks for reading