Use Canvas for real time matting

Canvas getImageData and putImageData apis make it easy to get and modify image data on Canvas. As long as you add some extensions, you can realize the matting of pictures or even videos.

Partial color removal in the picture

The Live Demo code

The data in the ImageData returned by getImageData is an array that arranges the color data for each pixel in the image in RGBA order. So as long as you modify the corresponding RGB value data in the data, and then put the data back through putImageData can be completed matting.

Here, an invisible canvas is used, and two IMG images respectively represent the original image and the processed image.

<canvas id="img-canvas" width="0" height="200" style="display: none;"></canvas>
<div class="wrapper">
  <img id="img-pre" src="xxx.png" height="200"/>
  <div class="arrow-right" style="display: none;" onclick="onArrowClick()"></div>
  <img id="img-cutted" src="" height="200"/>
</div>
Copy the code

After triggering the processing operation, obtain imgData first, then extract the RGB value of each pixel for comparison, and change the alpha value of the qualified pixel to 0 to make the pixel transparent.

function onArrowClick() {
  const colors = [255.255.255.255.255.255]
  canvas.setAttribute('width', imgPre.width)
  const ctx = canvas.getContext('2d')
  ctx.drawImage(imgPre, 0.0, imgPre.width, imgPre.height)

  const imgData = ctx.getImageData(0.0, canvas.width, canvas.height)
  for (let i = 0; i < imgData.data.length / 4; i++) {
    let r = imgData.data[i * 4 + 0]
    let g = imgData.data[i * 4 + 1]
    let b = imgData.data[i * 4 + 2]

    if (r >= colors[0] && r <= colors[1] && g >= colors[2] && g <= colors[3] && b >= colors[4] && b <= colors[5]) {
      imgData.data[i * 4 + 3] = 0 // Make the pixel transparent by setting Alpha to 0
    }
  }
  ctx.putImageData(imgData, 0.0)
  imgOut.src = canvas.toDataURL() // Switch to base64 for display in the IMG tag and download
}
Copy the code

Add some input to set the range of colors you want to clear

<div>
  R: <input class="color-input" type="number"/> to <input class="color-input" type="number"/> 
</div>
<div>
  G: <input class="color-input" type="number"/> to <input class="color-input" type="number"/> 
</div>
<div>
  B: <input class="color-input" type="number"/> to <input class="color-input" type="number"/> 
</div>
Copy the code

const colors = sortColors(Array.prototype.map.call(colorInput, item => (Number(item.value) || 255)))

function sortColors(colors) {
  for (let i = 0; i < 6; i += 2) {
    if (colors[i] > colors[i + 1]) {
      [colors[i], colors[i + 1]] = [colors[i + 1], colors[i]]
    }
  }
  return colors
}
Copy the code

Part of the video color removal

The Live Demo code

Here, a video is used to display the unprocessed video, and the video is looping all the time, using canvas to display the processed video

<div class="wrapper">
  <video id="video-pre" src="./video0.mp4" height="200" autoplay muted loop/></video>
  <canvas id="video-after" height="200"/>
</div>
Copy the code

Basically, there is no difference between the processing logic of the image, except that the transparent color of the video is changed to white, and it is processed through the setTimeout loop, okay

function cutout() {
  const colors = sortColors(Array.prototype.map.call(colorInput, item => (Number(item.value) || 0)))
  canvasVideoAfterCtx.drawImage(videoPre, 0.0, videoWidth, videoHeight)

  const imgData = canvasVideoAfterCtx.getImageData(0.0, videoWidth, videoHeight)
  for (let i = 0; i < imgData.data.length / 4; i++) {
    let r = imgData.data[i * 4 + 0]
    let g = imgData.data[i * 4 + 1]
    let b = imgData.data[i * 4 + 2]

    if (r >= colors[0] && r <= colors[1] && g >= colors[2] && g <= colors[3] && b >= colors[4] && b <= colors[5]) {
      imgData.data[i * 4 + 0] = 255
      imgData.data[i * 4 + 1] = 255
      imgData.data[i * 4 + 2] = 255
    }
  }
  canvasVideoAfterCtx.putImageData(imgData, 0.0)
  setTimeout(cutout) // Loop through the video
}
Copy the code

The resources

Realtime Video Processing with JavaScript (No Library)