Nodejs +nginx+ Puppeteer

Technology stack

Node.js/TypeORM/Apollo/GraphQL/MySQL

Front end: Vue/nuxt.js/Vuetify

Both front and back ends use TypeScript and Docker deployment.

background

A web on the mobile side needs to generate and download PDF files from the data.

Considering that apollo-server + GraphQL return data stream is not convenient, we used front-end generation initially.

Front-end PDF generation

Generating methods

(HTML2Canvas + JSPDF) method generation, and then dynamically generate a tag to download, generated PDF page time directly truncated and poor picture quality, in addition, in ios, the default will open PDF file preview, can only be shared by the browser to wechat what can barely be used.

Problems encountered

Who knows what customers need to incorporate this site into applets and who knows when it was introduced into their APP as a module? The preview PDF, which was originally shared or downloaded by the browser, is now nested in something else, so you go to the preview page and do nothing at all. So the front-end generated method is dead!

The back end generates PDF

So it’s going to have to be generated at the back end.

Generate solutions

  1. Write buffer directly to response.body via server (set content-type: The application/octet-stream header, in conjunction with the Content-Disposition header to specify the file name for the download, automatically triggers the browser file download. Binary stream data can be written directly into the HTTP response without base64 encoding.
  2. For ali cloud OSS, the server uploads the generated PDF file to OSS and returns the download address provided by OSS to the front end.
  3. Nginx can configure a static file directory, the server can write the generated PDF file to the local file, put in the specified static file directory, return static download address.

Scheme selection

It looks like plan 1 is the easiest, basically just setting the response header after generating the file, but!! The backend framework uses Apollo-server + graphQL, which is not a good way to return file data. So if you want to do that, you probably have to change the framework to something like KOA + Apollo-server-KOA or Express + Apollo-server-Express which is a combination of framework and middleware, and I tried to change that, Due to my lack of knowledge of the NodeJS framework and Apollo, OPTION 1 was abandoned for the time being.

Does plan 2 have to apply for a service from the boss? This seems to increase the implementation time, and this problem is quite urgent.

Nginx should be the easiest, but I won’t go into it here, because I won’t… (Since I can’t configure it in Docker, it’s good that there are leaders in the group to match it, thank you, and promise to learn later).

The implementation process

html-pdf

The original method of generating PDF in the back end is htML-PDF library, which dynamically generates HTML strings to be rendered according to the data. Using library toFIle method to achieve the generation of PDF files to the local. The road to success is not always smooth sailing, encountered the damn Error: Write EPIPE problem, back and forth N related issue, install fontConfig and rebuild PhantomJs-Prebuilt, Or could it be a problem someone mentioned with the Alpine PhantomJS-Prebuilt that won’t run? Basically the above method to achieve a time, the problem is still not solved. While I was spending a lot of time trying to troubleshoot the problem, I simply tried another library.

Play the Dockerfile at this point

FROM node:12.20.1 AS builder
RUN sed -i 's#http://deb.debian.org#https://mirrors.aliyun.com#g' /etc/apt/sources.list && \
  apt update && \
  apt add fontconfig
WORKDIR /builder
COPY package.json .
COPY yarn.lock .
RUN yarn
COPY.
RUN yarn run build


FROM node:12.20.1
WORKDIR /app
COPY --from=builder /builder/package.json .
COPY --from=builder /builder/yarn.lock .
COPY --from=builder /builder/node_modules ./node_modules
COPY --from=builder /builder/dist ./dist
COPY --from=builder /builder/ormconfig.js .
EXPOSE 4000
CMD ["yarn"."run"."start"]
Copy the code
puppeteer

I thought that changing the library would not have that kind of program running failure problem, the result is the same, this time is chromium run failure…

Problems encountered

Chromium startup failure

This is not a problem when run locally, but it will throw an exception when the export PDF function is triggered after deployment:

Error: Failed to launch Chrome! Spawn/app/node_modules/puppeteer. Local – chromium/Linux – 609904 / chrome – Linux/chrome ENOENT”

The solution

Github Issue has learned that the cause of this is that the puppeteer cannot be found on the server to execute it. This can be resolved by assigning the resource address to the puppeteer. Where does the resource come from? Since the installation failed, we need to manually install the system in Dockerfile.

Dockerfile

Add installation commands to Dockerfile

RUN apk add --nocache udev ttf-freefont chromium git
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
ENV CHROMIUM_PATH /usr/bin/chromium-browser

// npm install, etc.
Copy the code
pdf.js

Specify the Chromium resource address for puppeteer when it is instantiated

const puppeteer = require('puppeteer')

const pdf = puppeteer.launch({
  executablePath: process.env.CHROMIUM_PATH,
  args: ['--no-sandbox'].// This was important. Can't remember why
});
Copy the code

But be careful hereDockerfileWhere the installation command is written ！！！！！！！！！！！

Dockerfile does not resolve the problem
FROM node:12.14.1-alpine AS builder
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories && \
  apk add --no-cache udev ttf-freefont chromium git
RUN yarn config set registry https://registry.npm.taobao.org/
WORKDIR /builder
COPY package.json .
COPY yarn.lock .
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
RUN yarn
COPY.
RUN yarn run build


FROM node:12.14.1-alpine
WORKDIR /app
COPY --from=builder /builder/package.json .
COPY --from=builder /builder/yarn.lock .
COPY --from=builder /builder/node_modules ./node_modules
COPY --from=builder /builder/dist ./dist
COPY --from=builder /builder/ormconfig.js .
RUN mkdir /usr/share/fonts/win/
COPY --from=builder /builder/src/assets/simhei.ttf /usr/share/fonts/win/simhei.ttf
RUNchmod 777 /usr/share/fonts/win/simhei.ttf && \ fc-cache -f
EXPOSE 4000
CMD ["yarn"."run"."start"]

Copy the code
Solve problem with Dockerfile
FROM node:12.14.1-alpine AS builder
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories && \
  apk add --no-cache udev ttf-freefont chromium git
RUN yarn config set registry https://registry.npm.taobao.org/
WORKDIR /builder
COPY package.json .
COPY yarn.lock .
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
RUN yarn
COPY.
RUN yarn run build


FROM node:12.14.1-alpine
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories && \
  apk add --no-cache udev ttf-freefont chromium git && \
  apk add fontconfig
WORKDIR /app
COPY --from=builder /builder/package.json .
COPY --from=builder /builder/yarn.lock .
COPY --from=builder /builder/node_modules ./node_modules
COPY --from=builder /builder/dist ./dist
COPY --from=builder /builder/ormconfig.js .
RUN mkdir /usr/share/fonts/win/
COPY --from=builder /builder/src/assets/simhei.ttf /usr/share/fonts/win/simhei.ttf
RUNchmod 777 /usr/share/fonts/win/simhei.ttf && \ fc-cache -f
EXPOSE 4000
CMD ["yarn"."run"."start"]

Copy the code

See the difference? It’s the location of the installation code that took me so long. The syntax of Dockerfile is not clear.

Error: Write EPIPE has not been solved because fontConfig was installed in the wrong place.

The printed document is garbled in Chinese

The reason for this is that there is no Chinese font in the container system, so we can add it to the Dockerfile as well.

RUN mkdir /usr/share/fonts/win/
COPY --from=builder /builder/src/assets/simhei.ttf /usr/share/fonts/win/simhei.ttf
RUNchmod 777 /usr/share/fonts/win/simhei.ttf && \ fc-cache -f
Copy the code

This means:

  1. Created on the service/usr/share/fonts/win/Directory, to ensure the existence of directory;
  2. Put the project in the root directory/src/assets/simhei.ttfFont file copied to the service/usr/share/fonts/win/simhei.ttfLocation (this requires you to download firstsimhei.ttfFile in the project directory, the specific location of their own choice);
  3. Refresh font;

How to install Chinese fonts for Alpine system in dcoker container