Currently, the hottest short video app is Douyin, which has attracted many young users with its rich video effects. Today, let’s take a look at how Douyin effects can be realized on the Web.

Core principles

The implementation principle is relatively simple, summed up in the following three steps:

  • 1. Use pre-rendered canvas to draw each frame of the video.
  • 2. Upload the pre-rendered canvas as a texture to video memory.
  • 3, shader program for post-processing of the texture.

The steps seem simple, but the second and third steps involve some WebGL and shader knowledge, so don’t worry if you don’t understand them.

The main implementation

Each special effect needs the support of JS program and shader program. I will explain the JS program and shader program respectively.

The following special effects are intended to demonstrate the power and charm of the shader program, so the effects are mainly implemented on the shader side, so what does the JS program do?

JS is mainly used to read every frame of video, render to a temporary Canvas, and upload the temporary Canvas as a texture to video memory. The JS part of the following special effects is basically the same, so let’s take a look at the JS logic first.

1. Take a look at the HTML structure first

  <canvas id="canvas"></canvas>
  <canvas id="canvasBg" width="256" height="256"></canvas>
  <video id="video" src=".. /img/movie.mp4" controls></video>
Copy the code

We have placed two canvases, among which the first Canvas is used to display special effects and the second Canvas is used to draw video frames, which is relatively simple.

2, then look at the core code of JS logic

  • Read the video frame, render it to a temporary Canvas, and pass the temporary Canvas as a texture to video memory.
  ctxBg.drawImage(video, 0.0.256.256);
  gl.activeTexture(gl.TEXTURE0);
  gl.bindTexture(gl.TEXTURE_2D, texture);
  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, canvasBg);
  gl.texParameterf(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
  gl.texParameterf(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
  gl.uniform1i(u_Texture, 0);
Copy the code
  • Start a timer that transfers one time variable to video memory at a time.
function loop() {
      if (video.ended) {
        return;
      }
      computeFrame();
      time = time + interval;
      gl.uniform1f(u_Time, time);
      // Clear the canvas
      gl.clear(gl.COLOR_BUFFER_BIT);
      if (positions.length <= 0) {
        return;
      }
      // Draw primitives set to triangles.
      var primitiveType = gl.TRIANGLES;
      // Because we are drawing 6 points, we do 6 vertex drawing operations.
      gl.drawArrays(gl.TRIANGLES, 0, positions.length / 3);
      timer = requestAnimationFrame(loop);
    }
Copy the code

3, the real implementation of special effects is part of the shader program, of course, shader source code is also from JS to video memory, this part of the code we do not press the table, just take you feel each special effect.

The image is somewhat empty, you may not see clearly, but you can go to the sample page to feel ~

  • The zoom

  • Flash of white

  • The ghosting

  • Soul /

  • burr

conclusion

The code implementation of the above example contains some basic WebGL operations, which are quite lengthy. If you want to see the example, you can go to my Github to see it.

Tips: it is best to view it on PC, but there are still some experience problems in mobile browsing

Finally, the little hand point, pay attention to the public number [front-end E stack] bai ~