By GGvswild

1. The background

Saving a web page as a picture (hereinafter referred to as a snapshot) is an effective way for users to record and share information about a page, especially in the form of interest tests and marketing campaigns.

The snapshot is usually at the end of the page interaction process, summarizing the user’s final participation results and directly affecting the user’s complete experience of the activity. Therefore, producing a high-quality page snapshot is very important for the spread of the campaign and the transformation of the brand.

This article discusses the practical exploration of turning web pages into high-quality images from the aspects of content integrity, clarity and conversion efficiency based on the practice of cloud music’s past quality activities (such as “About Your Painting”, “Game of Thrones” and “Your Instruction Manual”).

2. Application scenario

  • It is suitable for turning pages into pictures, especially for scenes requiring high real-time performance.
  • Scenarios where you want to show cross-domain image resources in the snapshot.
  • In view of the incomplete, fuzzy or slow conversion process of the generated picture, we seek effective solutions to the scene.

3. Brief analysis of the principle

3.1 Scheme Selection

Based on whether the image is generated locally, snapshots can be processed in either front end or back end modes.

Since the back-end generated solution relies on network communication, there are inevitable communication overhead and wait delays, as well as some maintenance costs for template and data structure changes.

Therefore, for the comprehensive consideration of real-time and flexibility, we give priority to the front-end processing.

3.2 Basic Principles

The snapshot processing on the front end is essentially a process of converting view information contained in DOM nodes into picture information. This process can be implemented with the help of Canvas’s native API, which is the basis for the feasibility of the solution.

Specifically, the transformation process is to draw the target DOM node to the Canvas canvas, which is then exported as a picture. This can be simply marked as the draw phase and the export phase:

  • Drawing stage: Select the DOM node you want to draw according tonodeTypeDraw the target DOM node to the Canvas canvas by calling the corresponding API of the Canvas object<img>Draw usingdrawImageMethods).
  • Export stage: Finally achieve the export of canvas content through external interfaces such as Canvas toDataURL or getImageData.

3.3 Native Examples

Specifically, a snapshot of a single element can be generated as follows:

HTML:

<img id="target" src="./music-icon.png" />
Copy the code

JavaScript:

// Get the target element
const target = document.getElementById('target');

// Create canvas
const canvas = document.createElement('canvas');
canvas.width = 100;
canvas.height = 100;
const ctx = canvas.getContext("2d");

// Export phase: export the new image from canvas
const exportNewImage = (canvas) = > {
    const exportImage = document.createElement('img');
    exportImage.src = canvas.toDataURL();
    document.body.appendChild(exportImage);
}

// Draw phase: draw the canvas after the image content is loaded
target.onload = (a)= > {
    // Draw the contents of the picture into the canvas
    ctx.drawImage(target, 0.0.100.100);

    // Export the canvas contents as a new image
    exportNewImage(canvas);
}
Copy the code

DrawImage is the instance method of the Canvas context object and provides a variety of ways to draw a CanvasImageSource source to the canvas. ExportNewImage is used to export the view information in the Canvas as a data URI containing the image display.

4. Basic plan

In the previous section, we saw that the related basic API based on canvas provides the possibility for page snapshot processing on the front end.

However, specific business applications are often more complex, and the “low-spec” example above clearly fails to cover most practical scenarios, such as:

  • The canvasdrawImageThe method only acceptsCanvasImageSourceAnd theCanvasImageSourceDo not includeText nodeAnd ordinarydivAnd so on, will not<img>Drawing elements to the Canvas requires specific processing.
  • Hierarchical priority processing can be complicated when there are multiple DOM elements to draw.
  • Need to pay attention tofloat,z-index,positionAnd so on layout positioning processing.
  • The calculation of style synthesis rendering is complicated.

Therefore, based on the consideration of the comprehensive business scenario, we adopt the highly recognized solution in the community: HTML2Canvas and Canvas2Image as the basic library to implement the snapshot function.

4.1 html2canvas

Provides the ability to draw the DOM to the canvas

This community artifact simplifies the process of drawing one DOM at a time to the canvas. In a nutshell, the basic principle is:

  • Recursively traversing the target node and its child nodes to collect the style information of the node;
  • Compute the hierarchical relationship of nodes and draw them to the canvas one by one according to a certain priority policy.
  • This process is repeated, and finally the entire content of the target node is drawn.

Html2canvas exposes an executable function whose first argument is to receive the target node to be drawn (mandatory). The second parameter is an optional configuration item that sets the parameters involved in the Canvas export:

// Element draws the target node. Options is an optional parameter
html2canvas(element[,options]);  
Copy the code

The following is an example of a simple call:

import html2canvas from 'html2canvas';

const options = {};

// Enter the body node to return the Canvas object containing the body view contents
html2canvas(document.body, options).then(function(canvas) {
    document.body.appendChild(canvas);
});
Copy the code

4.2 canvas2image

Provides a variety of ways to export image information from Canvas

Compared with the complex drawing process undertaken by HTML2Canvas, canvas2Image is much simpler to do.

Canvas2image is only used to convert and store the input Canvas object in a specific format. These two types of operations support PNG, JPEG, GIF and BMP image types:

// Format conversion
Canvas2Image.convertToPNG(canvasObj, width, height);
Canvas2Image.convertToJPEG(canvasObj, width, height);
Canvas2Image.convertToGIF(canvasObj, width, height);
Canvas2Image.convertToBMP(canvasObj, width, height);

// Save the image as the specified format
Canvas2Image.saveAsPNG(canvasObj, width, height);
Canvas2Image.saveAsJPEG(canvasObj, width, height);
Canvas2Image.saveAsGIF(canvasObj, width, height);
Canvas2Image.saveAsBMP(canvasObj, width, height);

Copy the code

In essence, Canvas2Image only provides secondary encapsulation for canvas’s basic API (such as getImageData and toDataURL), and does not depend on HTML2Canvas.

In terms of usage, because the author does not provide the ES6 version of Canvas2Image (V1.0.5) at present, we cannot directly import this module.

For projects that support modern builds (such as WebPack), developers can clone the source code and manually add export to get ESM support:

Support for ESM export:

// canvas2Image.js
const Canvas2Image = function () {
    ...
}();

// The following is a custom addition
export default Canvas2Image;
Copy the code

Example call:

import Canvas2Image from './canvas2Image.js';

// Where canvas represents the incoming Canvas object, width and height are respectively the width and height of the exported image
Canvas2Image.convertToPNG(canvas, width, height)
Copy the code

4.3 the combinations

Next, we implement a basic snapshot generation solution based on the above two tools. It is also divided into two stages, corresponding to the basic principles in Section 3.2:

  • Step one, passhtml2canvasImplement DOM node drawing to canvas object;
  • The second step is to pass in the Canvas object returned from the previous stepcanvas2imageTo export snapshot information as required.

Specifically, we encapsulate a convertToImage function that inputs the target node and configuration item parameters and outputs the snapshot image information.

JavaScript:

// convertToImage.js
import html2canvas from 'html2canvas';
import Canvas2Image from './canvas2Image.js';

/** * Basic snapshot solution * @param {HTMLElement} Container * @param {object} Options html2Canvas related configuration */
function convertToImage(container, options = {}) {
    return html2canvas(container, options).then(canvas= > {
        const imageEl = Canvas2Image.convertToPNG(canvas, canvas.width, canvas.height);
        return imageEl;
    });
}
Copy the code

5. Advanced optimization

Using the example in the previous section, we implemented a more versatile basic page snapshot scheme based on HTML2Canvas and Canvas2Image than the native scheme. However, in the face of complex application scenarios, the snapshot results generated by the above basic solutions are often unsatisfactory.

On the one hand, the difference of the snapshot effect is that the view information exported by HTML2Canvas is the result of secondary drawing through the compound calculation of various DOM and canvas API (not one-key rasterization). Therefore, different host environments have different implementations of related apis, which may lead to multiple inconsistencies or abnormal display of the generated images.

On the other hand, business factors, such as the wrong configuration of html2Canvas or improper page layout for developers, will also cause deviations in the results of snapshot generation.

There are also common discussions in the community about the quality of generated snapshots, such as:

  • Why is some content displayed incomplete, incomplete, white or black?
  • Why is the image blurred like frosted glass when the original page is clearly distinguishable?
  • The process of converting a page to an image is very slow and affects subsequent operations. Is there any good way?
  • .

Let’s take a closer look at high-quality snapshot solutions in terms of content integrity, clarity optimization, and conversion efficiency.

5.1 Content Integrity

First problem: Ensure that the view information of the target node is fully exported

Because of the compatibility of the real machine environment and the difference of business implementation, the snapshot content is often inconsistent with the original view in some html2Canvas process. The common self-check checklist for incomplete content is as follows:

  • Cross-domain problem: Cross-domain images contaminate the Canvas.
  • Resource Loading: When a snapshot is generated, the related resources are not loaded.
  • Scrolling problem: There is an offset in the scrolling elements on the page, causing blank space at the top of the generated snapshot.

5.1.1 Cross-domain problems

This is often seen in scenarios where the imported picture material is cross-domain relative to the deployment project. Such as deployment are introduced in the https://st.music.163.com/ page source for https://p1.music.126.net images, this kind of picture is belong to the picture of the cross-domain resources.

Due to the canvas’s homogeneity restriction on image resources, the canvas can be polluted if it contains cross-domain image resources (Tainted canvases), resulting in messy images or the html2Canvas method not executing.

For cross-domain image resource processing, you can start from the following aspects:

(1) useCORS configuration

Enable the useCORS configuration item for HTML2Canvas as shown in the following example:

// doc: http://html2canvas.hertzen.com/configuration/
const opts = {
    useCORS   : true.// Cross-domain images are allowed
    allowTaint: false   // Do not allow cross-domain images to contaminate the canvas
};

html2canvas(element, opts);
Copy the code

The useCORS configuration is set to true in the html2Canvas source code, which essentially injects the tag in the target node with crossOrigin as anonymous, thus allowing the loading of CORS compliant image resources.

AllowTaint is false by default and may not be explicitly set. Even if this item is set to true, canvas’s restrictions on cross-domain images cannot be bypassed, because invoking the Canvas’s toDataURL will still be prohibited by the browser.

(2) CORS configuration

The configuration of useCORS in the previous step only allows the to receive cross-domain image resources, but to unlock the cross-domain image drawing on the canvas and export, the image resources themselves need to provide CORS support.

Here are some precautions for using CDN resources for cross-domain images:

To verify that the image resource supports CORS cross-domain, the Chrome Developer tool shows that the image request response header should contain the Access-control-Allow-Origin field, commonly referred to as the cross-domain header.

For example, a sample response header from a CDN image resource:

// Response Headers
access-control-allow-credentials: true
access-control-allow-headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type
access-control-allow-methods: GET,POST,OPTIONS
access-control-allow-origin: *
Copy the code

Different CDN service providers configure cross-domain headers in different ways. For details, consult the CDN service provider.

In special cases, some CDN providers may have the situation that the image cache does not contain the CORS cross-domain header. To ensure normal snapshot display, you are advised to contact the CDN for technical support first. You are not recommended to forcibly return the image link to the source by means of a postfix time stamp to avoid affecting the source performance and CDN charging.

(3) Server forwarding

In third-party apps such as wechat, the image resources such as user avatars on the platform do not directly provide CORS support. In this case, the server needs to be used as proxy forwarding to bypass the cross-domain restriction.

That is, the server requests the profile picture address of the platform user and sends it to the client (browser). Of course, the server interface itself should be the same as the page or support CORS.

For simplicity, it is assumed that the front end and the back end make the following conventions for cross-domain picture forwarding, and the interface and the front-end project are deployed under the same domain name:

Request the address Request way Incoming parameters Return information
/api/redirect/image GET Redirect, the original address Content-Typeforimage/pngImage resources of

sends a GET request to the image resource by stitching the/API /redirect/image with the redirect query parameter. Since the interface is the same as the page, cross-domain restrictions are not triggered:

<img src="/api/redirect/image? redirect=thirdwx.qlogo.cn/somebody/avatar" alt="user-pic" class="avatar" crossorigin="anonymous">
Copy the code

For a server-side interface implementation, here is a simple example based on KOA:

const Koa = require('koa');
const router = require('koa-router') ();const querystring = require('querystring');

const app = new Koa();

/** * Image forwarding interface * - Receive the redirect input, that is, the URL of the image to be requested * - Return image resources */
router.get('/api/redirect/image'.async function(ctx) {
    const querys = ctx.querystring;
    if(! querys)return;

    const { redirect } = querystring.parse(querys);

    const res = await proxyFetchImage(redirect);
    ctx.set('Content-Type'.'image/png');
    ctx.set('Cache-Control'.'max-age=2592000');
    ctx.response.body = res;

})

/** * Request and return image resource * @param {String} URL Image address */
async function proxyFetchImage(url) {
    const res = await fetch(url);
    return res.body; 
}

const res = await proxyFetchImage(redirect);

app.use(router.routes());
Copy the code

In the view of the browser, the image resources requested by the page are still resources under the same domain name, and the forwarding process is transparent to the front end. It is recommended to understand the source of image resources before the development of requirements, and make it clear whether server support is required.

In cloud Music’s early activity “Game of Thrones”, a similar scheme was used to achieve the complete drawing and snapshot export of users’ avatar in wechat platform.

5.1.2 Resource loading

Incomplete resource loading is a common cause of incomplete snapshots. If some of the resources are not fully loaded when you create a snapshot, the resulting content is not complete.

In addition to setting a certain delay, if you want to ensure that the resource has been loaded, you can implement it based on Promise.all.

Load image:

const preloadImg = (src) = > {
    return new Promise((resolve, reject) = > {
        const img = new Image();
        img.onload = (a)= > {
            resolve();
        }
        img.src = src;
    });
}
Copy the code

Ensure that snapshots are generated after all loads:

const preloadList = [
    './pic-1.png'.'./pic-2.png'.'./pic-3.png',];Promise.all(preloadList.map(src= > preloadImg(src))).then(async () => {
    convertToImage(container).then(canvas= > {
        // ...})});Copy the code

In fact, the above method is only to solve the page image display problem. In a real scenario, even if the picture on the page is complete, the content may still be blank after the snapshot is saved. The reason is that html2Canvas library internal processing, the image resources will still do a load request; If the loading fails, the section is empty after the snapshot is saved.

The following describes how to transfer image resources to BLOBs. Ensure that the image addresses are local to avoid loading failures during snapshot conversion. The Blob object in question represents an immutable, file-like object that represents binary raw data and is used in certain usage scenarios.

Blob:

 // Return the image Blob address
const toBlobURL = (function () {
    const urlMap = {};

    // @param {string} URL Passes in the image resource address
    return function (url) {
        // Filter duplicate values
        if (urlMap[url]) return Promise.resolve(urlMap[url]);

        return new Promise((resolve, reject) = > {
            const canvas = document.createElement('canvas');
            const ctx = canvas.getContext('2d');
            const img = document.createElement('img');

            img.src = url;
            img.onload = (a)= > {
                canvas.width = img.width;
                canvas.height = img.height;
                ctx.drawImage(img, 0.0);

                / / key 👇
                canvas.toBlob((blob) = > {
                    const blobURL = URL.createObjectURL(blob);

                    resolve(blobURL);
                });
            };
            img.onerror = (e) = >{ reject(e); }; }); }; } ());Copy the code

The above toBlobURL method implementation turns the resource link loaded with into blobURL.

Further, through the convertToBlobImage method, the in the incoming target node is batch processed into Blob format.

// Batch processing
function convertToBlobImage(targetNode, timeout) {
    if(! targetNode)return Promise.resolve();

    let nodeList = targetNode;

    if (targetNode instanceof Element) {
        if (targetNode.tagName.toLowerCase() === 'img') {
            nodeList = [targetNode];
        } else {
            nodeList = targetNode.getElementsByTagName('img'); }}else if(! (nodeListinstanceof Array) && !(nodeList instanceof NodeList)) {
        throw new Error('[convertToBlobImage] must be of Element or NodeList type ');
    }

    if (nodeList.length === 0) return Promise.resolve();

    // only consider 
    return new Promise((resolve) = > {
        let resolved = false;

        // Timeout processing
        if (timeout) {
            setTimeout((a)= > {
                if(! resolved) resolve(); resolved =true;
            }, timeout);
        }

        let count = 0;

        // Replace the  resource addresses one by one
        for (let i = 0, len = nodeList.length; i < len; ++i) {
            const v = nodeList[i];
            let p = Promise.resolve();

            if (v.tagName.toLowerCase() === 'img') {
                p = toBlobURL(v.src).then((blob) = > {
                    v.src = blob;
                });
            }

            p.finally((a)= > {
                if (++count === nodeList.length && !resolved) resolve();
            });
        }
    });
}

export default convertToBlobImage;
Copy the code

Using aspects, convertToBlobImage should be executed before calling the snapshot-generating convertToImage method.

5.1.3 Rolling Problem

  • Typical features: There is an empty area at the top of the snapshot.
  • Reason: This is usually caused by saving long images (more than one screen) and the scroll bar is not at the top (common in SPA-like applications).
  • Solution: In the callconvertToImageBefore I do that, I’m going to record thescrollTop, and then callwindow.scroll(0, 0)Move the page to the top. After the snapshot is generated, the call can be invokedwindow.scroll(0, scrollTop)Restore the original longitudinal offset.

Example:

// The target node to be saved (modified as required 👇)
const container = document.body;
// Actual scrolling elements (actual modifications 👇)
const scrollElement = document.documentElement;
// Record the longitudinal offset of the scrolling element
const scrollTop = scrollElement.scrollTop;

// Put the body first for the scrolling element
window.scroll(0.0);
convertToImage(container)
    .then((a)= > {
        // ...
    }).catch((a)= > {
        // ...
    }).finally((a)= > {
        // Restore the offset
        window.scroll(0, scrollTop);
    });

Copy the code

In particular, if there is a local scroll layout, you can also set the scroll element to the top to avoid the empty top of the container.

5.2 Definition optimization

Clarity is the watershed of snapshot quality

The following is a snapshot of two “Game of Thrones” results pages before and after optimization. You can see the left image before optimization, both in the text edge and image details, compared with the optimized sharpness there is a significant difference.

The sharpness of the resulting snapshot depends primarily on the sharpness of the canvas converted from the DOM in the first step.

Here are five effective ways to optimize clarity.

5.2.1 Use px units

In order to give explicit integer values to html2Canvas and avoid stretch blur caused by decimal rounding, it is recommended to change the element style of %, VW, Vh or REM units used in the layout to px.

good:

<div style="width: 100px;"></div>
Copy the code

bad:

<div style="width: 30%;"></div>
Copy the code

5.2.2 Use img tag to display pictures

In many cases, the exported image blur is caused by the image in the original view being displayed as a BACKGROUND in the CSS. Background-size does not reflect a specific width and height value, but uses enumerations such as contain and cover to represent the image zoom type. Compared to the tag, the background method results in a blurry image. Changing background to will improve the sharpness of the image. For scenarios where background must be used, see the solution in Section 5.25.

good:

<img class="u-image" src="./music.png" alt="icon">
Copy the code

bad:

<div class="u-image" style="background: url(./music.png);"></div>
Copy the code

5.2.3 Configuring a high-powered Canvas

For high resolution screens, Canvas can achieve some degree of clarity improvement by aligning CSS pixels with the physical pixels of the high resolution screen (both types of pixels are described and discussed in detail here).

In action, create an image enlarged by devicePixelRatio and then use CSS to zoom it out by the same multiple, effectively improving the clarity of the image drawn to the canvas.

When using HTML2Canvas, we can configure an enlarged canvas for drawing imported nodes.

// convertToImage.js
import html2canvas from 'html2canvas';

// Create the base canvas for drawing
function createBaseCanvas(scale) {
    const canvas = document.createElement("canvas");
    canvas.width = width * scale;
    canvas.height = height * scale;
    canvas.getContext("2d").scale(scale, scale);

    return canvas;
}

// Generate a snapshot
function convertToImage(container, options = {}) {
    // Set the magnification
    const scale = window.devicePixelRatio;

    // Create the base canvas for drawing
    const canvas = createBaseCanvas(scale);

    // Pass the original width and height of the node
    const width = container.offsetWidth;
    const height = container.offsetHeight;   

    // html2Canvas configuration item
    const ops = {
        scale,
        width,
        height,
        canvas,
        useCORS: true.allowTaint: false. options };return html2canvas(container, ops).then(canvas= > {
        const imageEl = Canvas2Image.convertToPNG(canvas, canvas.width, canvas.height);
        return imageEl;
    });
}
Copy the code

5.2.4 Disabling antialiasing

ImageSmoothingEnabled imageSmoothingEnabled is a property used by the Canvas 2D API to set whether the image is smooth or not. True means that the image is smooth (the default) and false means that the Canvas anti-aliasing is disabled.

By default, canvas anti-aliasing is turned on. By turning off anti-aliasing, you can sharpen the image to some extent and improve the sharpness of the line edges.

Accordingly, we upgrade the above createBaseCanvas method to:

// Create the base canvas for drawing
function createBaseCanvas(scale) {
    const canvas = document.createElement("canvas");
    canvas.width = width * scale;
    canvas.height = height * scale;

    const context = canvas.getContext("2d");

    // Turn off antialiasing
    context.mozImageSmoothingEnabled = false;
    context.webkitImageSmoothingEnabled = false;
    context.msImageSmoothingEnabled = false;
    context.imageSmoothingEnabled = false;

    context.scale(scale, scale);

    return canvas;
}
Copy the code

5.2.5 Sharpen specific elements

Inspired by canvas scaling, we can also adopt similar optimization operations for specific DOM elements, that is, set the width and height of the element to be optimized to 2 times or devicePixelRatio, and then control the display size of the element to remain unchanged through CSS scaling.

For example, for elements that must use a background image, the sharpness of the snapshot can be significantly improved by:

.box {
    background: url(/path/to/image) no-repeat;
    width: 100px;
    height: 100px;
    transform: scale(0.5);
    transform-origin: 0 0;
}
Copy the code

Among them, width and height are 2 times of the actual display width and height, and the element size is scaled through transform: Scale (0.5), and transform-Origin is set according to the actual situation.

5.3 Conversion efficiency

The snapshot conversion efficiency is directly related to the waiting time of users. We can optimize it in the target node incoming phase and the snapshot export phase.

5.3.1 Incoming phase

The leaner the view information that is passed into the node, the less computation is required to generate the snapshot

The following methods are applicable to “slim down” incoming view information:

  • To reduceDOM sizeAnd reduce thehtml2canvasThe amount of computation required to recursively iterate.
  • Compress the volume of the image material itself, using tools such as Tinypng or ImageOptim to compress the material.
  • If you are using custom fonts, use the Fontmin tool to crop the text as needed to avoid introducing megabytes of invalid resources.
  • Pass in appropriatescaleValue to scale the Canvas canvas (Section 5.2.3). Under normal circumstances, 2~3 times has met the general scene, it is unnecessary to pass too large magnification.
  • The image resource transfer to BLOB mentioned in Section 5.1.2 can localization the image resource, avoiding the secondary image loading process of HTML2Canvas when the snapshot is generated. Meanwhile, the generated resource link has the advantages of shorter URL length and so on.

5.3.2 Export optimization

Canvas2image provides several apis for exporting image information, as described above. Include:

  • convertToPNG
  • convertToJPEG
  • convertToGIF
  • convertToBMP

Different export formats have significant impact on the size of the snapshot file. Generally for image materials that do not require transparency, you can use the JPEG export format. In our practice, JPEG even saves more than 80% of the file size compared to PNG.

The export format of the picture in the actual scene can be selected according to the business requirements.

6. Summary

Based on HTML2Canvas and Canvas2Image, this paper introduces the solution of producing high quality snapshot from the aspects of snapshot content integrity, clarity and conversion efficiency.

Due to the complexity of practical application, the above scheme may not be able to cover every specific scenario. Welcome to communicate and discuss.

7. Reference links

  • Based on HTML2Canvas, web pages are saved as pictures and picture definition is optimized
  • Wechat WAP page generated to share the poster function stomp pit experience
  • The H5 realizes the preservation of the mining records of pictures
  • Realize the realization of wechat H5 webpage long press to save the picture and identify the TWO-DIMENSIONAL code
  • MDN: Allowing cross-origin use of images and canvas

This article is published by netease Cloud Music front end team. Any unauthorized reprint of the article is prohibited. We’re always looking for people, so if you’re ready to change jobs and you love cloud music, then join us!