background

“Screenshot” refers to converting an applet or H5 page into an image that can be easily forwarded or shared. Compared with ordinary structured sharing (link sharing), screenshot sharing has the advantages of richer visual representation and more information bearing, and has been applied in many projects. For example:

Related technologies

At present, the popular “screen capture” implementation scheme can be divided into client side screen capture and server side screen capture according to the way of generating pictures.

Puppeteer or other Headless browsers are generally used to render pages and generate images on the server side. This method avoids front-end compatibility problems caused by multiple browsers and terminals, but consumes a large number of server resources (starting the Headless browser consumes system resources, Page rendering needs to load external resources), and has low performance in weak network environment; In addition, if the front end organizes the content of the page to be rendered and then transmits it to the server, there may be a certain security risk.

The client screen capture usually uses the graphics API (such as Canvas and SVG) provided by the client to redraw the page and generate pictures. This method does not rely on the server and consumes less resources. However, redrawing the page using the graphics API provided by the client is equivalent to reimplementing the rendering process of the client, which is undoubtedly complicated and troublesome. Therefore, current client-side screenshot components often choose to support only partial CSS properties to reduce complexity and avoid the compatibility problems caused by the wide variety of browsers and terminals.

html2canvas

Taking the most popular HTML2Canvas component as an example (as of September 2020, it has 21.6 K Star on Github), its official website Features states that CSS properties are not supported at present as follows:

However, html2Canvas does not perform well when using the supposedly supported CSS properties.

  • [Fixed] Text rendering is offset

Text rendering DEMO

In the figure above, it is obvious that the text after the html2Canvas screenshot (the third part of the red box in the figure) has a downward offset compared with the original text (the first one in the figure).

  • Text gradient is not supported.

Text gradient DEMO

In the figure above, the first one is the effect of the text gradient on the original page, and the third part in the red box is the effect of the screenshot of the text gradient on the page by HTML2Canvas.

  • The inheritance relationship of some CSS properties (transform, opacity, filter, etc.) is not perfect.

Attribute Inheritance DEMO

In the page, p element has three parent elements, which are set with opacity, transform, filter and other attributes respectively. Under the combined action of these parent element attributes and p element’s own attributes, the final rendering effect of the page is shown as the first one in the figure. The effect of HTML2Canvas on the screenshot of P element is the third one in the figure, which only considers the influence of CSS attributes of P element.

The above is only a simple example, when we really use HTML2Canvas may encounter more pits; This is the problem with html2Canvas’s attempt to completely re-implement the browser rendering process; Even as THE CSS standard is updated and components with this mindset can’t keep up, the problems will increase.

In addition, there is a Canvas API similar to the browser in the applet, but html2Canvas cannot be used in the applet due to some minor differences. Therefore, there are small program components wXML-to-Canvas, Painter, etc. In fact, these components can be regarded as the implementation of the weakened version of HTML2Canvas in the small program, which supports fewer CSS properties and increases the cost of learning to use a lot.

dom-to-image

Is there a slightly simpler way to think about client screenshots? For example, if only the client provided HTML + CSS to generate images directly!

The answer is yes!

Elements that use other XML namespaces can be embedded in SVG’s foreignObject element, This means that we only need to specify the internal elements of namespace is http://www.w3.org/1999/xhtml can use HTML + CSS in SVG, a simple example:

<svg xmlns='http://www.w3.org/2000/svg' width='100' height='100'> <foreignObject width='100%' height='100%'> <div class='test' xmlns='http://www.w3.org/1999/xhtml' style='width:100px; height:100px; background-color:red; '> 1 😁 1 < / div > < / foreignObject > < / SVG >Copy the code

Create an image in a browser and specify the type as SVG + XML.

let svg = ""; svg += "<svg xmlns='http://www.w3.org/2000/svg' width='1' height='1'>"; svg += "<foreignObject width='100%' height='100%'>"; svg += "<div xmlns='http://www.w3.org/1999/xhtml' style='width:100%; height:100%; background-color:red; '></div>"; svg += "</foreignObject>"; svg += "</svg>" let self = this; let img = new Image(); img.src = "data:image/svg+xml; charset=utf-8," + encodeURIComponent(svg);Copy the code

To put it simply, we first convert HTML + CSS to SVG and then convert SVG to images. In this way, we indirectly realize the generation of images through HTML + CSS.

The biggest part of the solution is converting the page elements to SVG’s foreignObject elements. In theory, as long as this conversion process is fine and the client supports foreignObject, the resulting screenshot should look exactly like the original page.

However, the reality is always cruel (opportunity), take the current representative of the project DOM-to-Image (by September 2020, 6.5K star on Github) component for example, its conversion process has many problems; The dom-to-image screenshot (the last blue box) is also shown in the above illustration of the HTML2Canvas problem. Exhibit the following problems:

  • The screenshot effect is blurred;
  • Text gradient is not supported.
  • The inheritance relationship of some CSS properties is not handled properly.

To make matters worse, even the most basic box model and layout of CSS are problematic:

  • Abnormal box model screenshot;

The box model DEMO

In the figure above, the first one is the effect of the margin box model of the original page, and the fourth blue box part is the effect of the DOM-to-image after screenshots of the margin box model. It can be obviously seen that the effect after screenshots has downward offset.

  • The layout screenshot is abnormal.

Layout of the DEMO

In the figure above, the fourth part is the absolute positioning layout effect of the original page, and the third part in the blue box is the effect of the DOM-to-image after taking a screenshot of the absolute positioning layout. It can be obviously seen that there are blank anomalies in the effect after taking a screenshot.

In addition to the implementation problems of DOM-to-image itself, there are also some problems of the scheme itself. There are compatibility issues with foreignObject and Data: Image/SVG + XML, and in some cases these features may not be supported by clients (applets and some browsers).

Front and rear end mixed screen capture solution

The advantages and disadvantages of various schemes such as server side screen capture and client side screen capture are summarized above. Based on the principle of simple implementation, easy to use, stable and reliable, not limited to a single technology, it is possible to get a more perfect solution by combining the server-side and client-side SVG screenshots.

SimpleScreenshot can be implemented as follows.

The core code of the project includes two parts:

├── Web │ ├─ ql-App │ ├── App │ ├─ Server │ ├── Tools │ ├─ Web │ ├─ Web │ ├─ QL-App │ ├── Guitar │ └ ─ ─ format_xml. HTML ├ ─ ─ the index - web. Js / / web SimpleScreenshot code ├ ─ ─ the index - qq. Js / / qq small program SimpleScreenshot code └ ─ ─ // wechat small program SimpleScreenshot codeCopy the code

The server directory is the screenshot service, which needs to be deployed separately (Docker image deployment is supported), and the build directory is the packaged and constructed client code, which needs to be introduced when used in actual projects.

SimpleScreenshot supports all CSS properties. What you See is what you get. Developers just need to code the page normally and then do a simple initialization:

let screenshot = new SimpleScreenshot({ debug: isPub ? ImgType: imgType, // puppeteerServer: puppeteerServer: "Https://dom2img.lione.me/simple-screenshot", / / screenshot service puppeteerGlobalFont: PingFang, / / global font devicePixelRatio screenshots service: Function (MSG) {console.log(MSG); function (MSG) {console.log(MSG); console.log(msg.svg); }, error: function (err) { console.log(err); }});Copy the code

Dom2img. Lione. Me/simple – scre… For example services deployed on cloud servers, do not use them in a production environment!!

After initialization, pass in the element selector to be captured and execute the SimpleScreenshot component’s toIMG method to complete the screenshot:

screenshot.toIMG(".mps-content", function (img) {
  // img.base64
  // img.canvas
});
Copy the code

The result of a screenshot is obtained in the callback function (a client screenshot can get base64 images and a Canvas Canvas, but a server screenshot can only get Base64 images).

Web sample

SimpleScreenShot also supports screenshots of applets, similar to the Web. The only difference is that after you have written the applets page, you need to copy the code of the elements to SimpleScreenShot’s formatting tools:

The reason for the above operation is that the API function provided by the applet is limited. After selecting the element node through the selector, the type, child node information and some key attributes (such as SRC attribute of image element, inline text, etc.) of the element node cannot be obtained. This causes the generated SVG code to be missing critical information and the resulting screenshot image to be abnormal.

In order to solve the above problems, this technical solution provides a formatting tool that can be used to parse the code of the applets page and set the relevant information needed to generate SVG code in advance into the element’s custom attributes, so that it can be directly obtained when running the code.

Where data-name indicates the element type, data-level indicates the element level (0 indicates the root node, 0-0 indicates the first child node of the root node, 0-1 indicates the second child node of the root node, and so on), data-len indicates the number of child elements (including inline text) of the current element, Data-texts represent the inline text information of the current element (there is no concept of inline text in the small program); In addition, in order to select all the element nodes that generate the image at once, we need to annotate each element node with a specific class name.

Finally, replace the original code with the formatted code.

Example of wechat applets

SimpleScreenshot screenshots are also shown in the html2Canvas/DOM-to-image problem illustration illustration above (yellow box).

Summary and Prospect

This paper proposes and realizes SimpleScreenshot, which adopts a relatively simple solution and makes full use of the resources and capabilities of the client under the premise of ensuring stability and reliability. Unified screenshot requirements are implemented in browsers and applets in a way that minimizes learning and development costs.

At present, the scheme has been implemented in a few projects, but the application scenarios are few, so there are other undiscovered problems. Personal energy is limited, welcome to work together to build.