A brief introduction to the Canvas API

  • Video_and_audio_content
  • ImageData

The ImageData object stores the actual pixel data of the Canvas object. It contains the following read-only properties:

Data: A one-dimensional array of type UINT8ClampeDarray containing RGBA integer data, ranging from 0 to 255 inclusive.

The data property returns a UINT8ClamPedArray, which can be used to view the initial pixel data. Each pixel has four 1Bytes (red, green, blue, and transparent values in order; This is the “RGBA” format) to represent. Parts of each color value are represented by 0 to 255. Each segment is assigned a contiguous index in the array, with the red portion of the upper-left pixel at index 0 in the array. Pixels are processed from left to right, and then down through the array.

Simply put, we need to extract each pixel in groups of fourrgbavalue

Then we combined the features of Canvas used to operate video to carry out green screen image matting for background.

First on the renderings:

Code address: Gitee preview address: ali cloud, githubPage (Gitee recently redress gitePage, first with github of the, if you can not open the download source code local static service view)

Implementation approach

Video ==> Video Screenshot ==> processing green pixel transparent ==> map to the top of the background image to take a screenshot of the video, and then the pixel block in the video pixel green into transparent and then put the processed image on the top of the prepared background image

implementation

1. Prepare the video and canvas

<body onload="processor.doLoad()"> <div> <video id="video" src="./q.mp4" width="350" controls="true"></video> </div> <div> <! - video capture - > < canvas id = "c1" width = "260" height = "190" > < / canvas > <! - deal with green pixels as transparent - > < canvas id = "c2" width = "260" height = "190" > < / canvas > <! - texture to the background above - > < canvas id = "c3" width = "260" height = "190" > < / canvas > < / div > < / body >

2. Add video playback monitor

doLoad: function doLoad() {
  this.video = document.getElementById("video");
  this.c1 = document.getElementById("c1");
  this.ctx1 = this.c1.getContext("2d");
  this.c2 = document.getElementById("c2");
  this.ctx2 = this.c2.getContext("2d");
  this.c3 = document.getElementById("c3");
  this.ctx3 = this.c3.getContext("2d");
  let self = this;
  this.video.addEventListener(
    "play",
    function() {
      self.width = self.video.videoWidth / 5;
      self.height = self.video.videoHeight / 3;
      self.timerCallback();
    },
    false
  );
}

3. Add a timer

After the video is played, the call is made to capture the screenshot of each frame

timerCallback: function timerCallback() {
  if (this.video.paused || this.video.ended) {
    return;
  }
  this.computeFrame();
  let self = this;
  setTimeout(function () {
    self.timerCallback();
  }, 0);
}

4. Video frame operation

Set the green background to transparent and map it to the custom background image

computeFrame: function computeFrame() {
  this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
  let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
  let l = frame.data.length / 4;

  for (let i = 0; i < l; i++) {
    let r = frame.data[i * 4 + 0];
    let g = frame.data[i * 4 + 1];
    let b = frame.data[i * 4 + 2];
    //rgb(8 204 4)
    if (r > 4 && g > 100 && b < 100) {
      frame.data[i * 4 + 3] = 0;
    }
  }
  this.ctx2.putImageData(frame, 0, 0);
  this.ctx3.putImageData(frame, 0, 0);
  return;
}

5. Fine-tuning

// The video in RGB (8 204 4) is not pure in color, it is not always RGB (8 204 4), so this is a simple fine tuning.. if (r > 4 && g > 100 && b < 100) { frame.data[i * 4 + 3] = 0; }

At the end

For more questions, please join the front-end communication group 749539640. Code: Gitee