Canvas frame animation realizes background video

preface

A few days ago, a friend asked me how to optimize the frame animation effect. The idea is to make the background video look like a Web page. For this effect, you can refer to the world of Warcraft cool official website. Open the debugger and you can see that the video element is located at the bottom. So, if your project doesn’t consider compatibility with older browsers, or you’re not interested in frame animation, this article may not be for you.

On the rendering

The solution

According to the original requirements, it was necessary to achieve a similar effect to the world of Warcraft website. But the technology used is Canvas frame animation: drawing pictures of each frame through canvas to achieve the effect of video playback.

steps

1. Export the video as an image frame

In this case, I’m using Adobe Premiere

Open Premiere -> Import Video -> Clip video (it is recommended not to be too long, the video in this example is about 5 seconds) -> Export (CTRL + M) -> select JPG, and finally decide to export

If you don’t know much about the app, ask the UI girl for help

2. Build the basics

Let’s look at the structure of the project

Part of the HTML

/index.html

<canvas id="cvs"></canvas>
<script src="./main.js"></script>
Copy the code

Js part

Look at the basic properties of the original video and make a note of them for later use.

/main.js

const FRAME_LENGTH = 137; // The number of exported image frames
const VIDEO_FPS = 23.976; // The frame rate of the original video, which affects the smoothness of the video
const VIDEO_WIDTH = 480; // Video size
const VIDEO_HEIGHT = 360;
Copy the code

Then, we want to load all the image resources before playing. The resource load task needs to be handled asynchronously.

Let’s define an asynchronous function that loads images

function loadImage(url) {
  return new Promise((r) = > {
    const img = new Image();
    img.onload = () = > r(img);

    // Special processing is done here
    // If the image fails to load, resolve is still returned, but the content is empty
    img.onerror = () = > r();
    img.src = url;
  });
}
Copy the code

In this example, the file names of all images are sorted by ordinal order. So I defined a function to get the path of the image resource to facilitate the loading of the image. This function depends on the actual requirements and is not necessary.

function getImageSrcByIndex(index = 0) {
  // The image exported by PR is automatically numbered and is 3 bits long
  const idx = 00 `${index}`.slice(-3);
  return `./frames/frame-${idx}.jpg`;
}
Copy the code

Next, let’s initialize some basic functionality

// /main.js

async function init() {
  const cvs = window.document.getElementById("cvs");
  const ctx = cvs.getContext("2d");
  cvs.width = VIDEO_WIDTH;
  cvs.height = VIDEO_HEIGHT;
}

window.addEventListener("load", init);
Copy the code

Then, load all the images

// Load all image resources first
const loadTasks = Array(FRAME_LENGTH)
  .fill(0)
  .map((v, i) = > loadImage(getImageSrcByIndex(i)));
const frames = await Promise.all(loadTasks);
Copy the code

In this step,frames will be an array of Image objects (Image[])

Next, let’s define a function that plays the video

function runAnimation(ctx = undefined, frames = []) {
  function draw(timestamp) {
    console.log("Map");
    window.requestAnimationFrame(draw);
  }

  window.requestAnimationFrame(draw);
}
Copy the code

Here, we use the requestAnimationFrame API. This API does the same thing as window.setInterval, drawing content over and over again, but with better performance.

Then you can start drawing pictures

function runAnimation(ctx = undefined, frames = []) {
  let currFrameIndex = -1; // Index of the current image to draw

  function draw(timestamp) {
    currFrameIndex = (currFrameIndex + 1) % frames.length;
    // There may be some image loading failure, need to judge
    if (frames[currFrameIndex]) {
      ctx.drawImage(frames[currFrameIndex], 0.0);
    }

    window.requestAnimationFrame(draw);
  }

  window.requestAnimationFrame(draw);
}
Copy the code

Call this function in the main function

// /main.js

async function init() {
  const cvs = window.document.getElementById("cvs");
  const ctx = cvs.getContext("2d");
  cvs.width = VIDEO_WIDTH;
  cvs.height = VIDEO_HEIGHT;

  // Load all image resources first
  const loadTasks = Array(FRAME_LENGTH)
    .fill(0)
    .map((v, i) = > loadImage(getImageSrcByIndex(i)));
  const frames = await Promise.all(loadTasks);

  runAnimation(ctx, frames);
}

window.addEventListener("load", init);
Copy the code

At this point, you should be able to see the video. But that’s not all. You’ll notice that the video doesn’t play smoothly, or even change speed. That’s because frame rates aren’t considered. Since the frame rate of the video is 23.976, but requestAnimationFrame will try to call the GPU at an interval of around ’16ms'(depending on the GPU performance), which will not match the frame rate of the video, The interval between each frame of the video should be 1000 ms / 23.976 FPS. In addition, the arguments passed in the requestAnimationFrame callback will tell us when the callback will be executed.

To fix the framerate problem, let’s tweak the runAnimation.

function runAnimation(ctx = undefined, fps = 30, frames = []) {
  const tpf = Math.floor(1000 / fps); // The interval between each frame of the current video
  let lastRenderTime = 0; // The time when the last picture frame was drawn
  let currFrameIndex = -1;

  function draw(timestamp) {
    // Determine whether the next frame needs to be drawn by calculating the call time difference
    const shouldRender = timestamp - lastRenderTime >= tpf;

    if (shouldRender) {
      lastRenderTime = timestamp;
      currFrameIndex = (currFrameIndex + 1) % frames.length;

      if (frames[currFrameIndex]) {
        ctx.drawImage(frames[currFrameIndex], 0.0); }}// Loop
    window.requestAnimationFrame(draw);
  }

  // First draw
  window.requestAnimationFrame(draw);
}
Copy the code

Finally, the complete code

const FRAME_LENGTH = 137;
const VIDEO_FPS = 23.976;
const VIDEO_WIDTH = 480;
const VIDEO_HEIGHT = 360;

function loadImage(url) {
  return new Promise((r) = > {
    const img = new Image();
    img.onload = () = > r(img);
    img.onerror = () = > r();
    img.src = url;
  });
}

function getImageSrcByIndex(index = 0) {
  const idx = ` 000${index}`.slice(-3);
  return `./frames/frame-${idx}.jpg`;
}

/ * * * *@param {object} ctx canvas.context
 * @param {number} FPS video frame rate *@param {Image[]} Frames The image object, the image of each frame *@returns* /
function runAnimation(ctx = undefined, fps = 30, frames = []) {
  if (
    !ctx ||
    typeofctx.drawImage ! = ="function" ||
    fps <= 1 ||
    !Array.isArray(frames) ||
    frames.length === 0
  ) {
    return;
  }

  const tpf = Math.floor(1000 / fps);
  let lastRenderTime = 0;
  let currFrameIndex = -1;

  function draw(timestamp) {
    const shouldRender = timestamp - lastRenderTime >= tpf;

    if (shouldRender) {
      lastRenderTime = timestamp;
      currFrameIndex = (currFrameIndex + 1) % frames.length;

      if (frames[currFrameIndex]) {
        ctx.drawImage(frames[currFrameIndex], 0.0); }}// Loop
    window.requestAnimationFrame(draw);
  }

  // First draw
  window.requestAnimationFrame(draw);
}

async function init() {
  const el = window.document.getElementById("content");
  const cvs = window.document.getElementById("cvs");
  const ctx = cvs.getContext("2d");
  cvs.width = VIDEO_WIDTH;
  cvs.height = VIDEO_HEIGHT;
  el.style.width = VIDEO_WIDTH + "px";
  el.style.height = VIDEO_HEIGHT + "px";

  // Load all image resources first
  const loadTasks = Array(FRAME_LENGTH)
    .fill(0)
    .map((v, i) = > loadImage(getImageSrcByIndex(i)));
  const frames = await Promise.all(loadTasks);

  / / to draw
  runAnimation(ctx, VIDEO_FPS, frames);
}

window.addEventListener("load", init);
Copy the code

The above

I hope this article may help some of you.