Thanks to OpenCV’s cross-language and cross-platform advantages, the client can also provide face detection capabilities with low development costs, and can provide services to the Web container via JsBridge and other means. However, once out of the container, isolated pages will lose this capability. Until one day…

Well, face recognition in the front is still in slash-and-burn ancient times, however, our infrastructure has started, I hope that some of the subsequent introduction can bring certain inspiration for you.

With the gradual improvement of the computing capability of the client hardware, more and more permissions are obtained at the browser level. As image processing requires a large amount of computing resources, in fact, the browser can also undertake some work of image Detection, so the Shape Detection API was developed.

The following simple examples show the basic usage. Before attempting to edit and run the code, make sure that the new feature is enabled on your Chrome version and that the API is restricted by the same origin policy:

chrome://flags/#enable-experimental-web-platform-features

• Barcode Detection (For Chrome 56+)

var barcodeDetector = new BarcodeDetector();
barcodeDetector.detect(image)
  .then(barcodes => {
    barcodes.forEach(barcode => console.log(barcodes.rawValue))
  })
  .catch(err => console.error(err));Copy the code

• Face Detection (For Chrome 56+)

var faceDetector = new FaceDetector();
faceDetector.detect(image)
  .then(faces => faces.forEach(face => console.log(face)))
  .catch(err => console.error(err));Copy the code

• Text Detection (For Chrome 58+)

var textDetector = new TextDetector();
textDetector.detect(image)
  .then(boundingBoxes => {
    for(let box of boundingBoxes) {
      speechSynthesis.speak(new SpeechSynthesisUtterance(box.rawValue));
    }
  })
  .catch(err => console.error(err));Copy the code

The face detection in the video is not much different from the image. Through getUserMedia, the camera can be opened to obtain the information of the video/microphone. Through the detection and display of the video frame, the face detection in the video can be realized.

The core code is as follows:

navigator.mediaDevices.getUserMedia({ video: true, // audio: true }) .then(function (mediaStream) { video.src = window.URL.createObjectURL(mediaStream); video.onloadedmetadata = function (e) { // Do something with the video here. }; }) .catch(function (error) { console.log(error.name); }); setInterval(function () { ctx.clearRect(0, 0, canvas.width, canvas.height); ctx.drawImage(video, 0, 0); image.src = canvas.toDataURL('image/png'); image.onload = function() { detect(); }}, 60);Copy the code

Tracking. Js is a jS-wrapped image processing library that brings rich algorithms and technologies related to computational vision to the browser. It can realize color tracking, face detection and other functions, specific features are as follows:

b. jquery.facedetection

Facedetection is a jquery/zepto facedetection plug-in based on the cross-terminal CCV image classifier and detector.

2.5 the Node. Js & OpenCv

The Node-OpencV module has been around for a few years and is perfectly compatible with OpencV V2.4.x, although it is not yet fully compatible with V3.x and provides limited apis. The arrival of N-API may bring more surprises.

Imagine in a Electron or Node-WebKit container, can we realize real-time face detection by locally enabling websocket service? The implementation of the idea of the code is as follows:

• Back-end processing logic

import cv from 'opencv';
const detectConfigFile = './node_modules/opencv/data/haarcascade_frontalface_alt2.xml';
// camera properties
const camWidth = 320;
const camHeight = 240;
const camFps = 10;
const camInterval = 1000 / camFps;
// face detection properties
const rectColor = [0, 255, 0];
const rectThickness = 2;
// initialize camera
const camera = new cv.VideoCapture(0);
camera.setWidth(camWidth);
camera.setHeight(camHeight);
const frameHandler = (err, im) => {
  return new Promise((resolve, reject) => {
    if (err) {
      return reject(err);
    }
    im.detectObject(detectConfigFile, {}, (error, faces) => {
      if (error) {
        return reject(error);
      }
      let face;
      for (let i = 0; i < faces.length; i++) {
        face = faces[i];
        im.rectangle([face.x, face.y], [face.width, face.height], rectColor, rectThickness);
      }
      return resolve(im);
    });
  });
};
module.exports = function (socket) {
  const frameSocketHanlder = (err, im) => {
    return frameHandler(err, im)
      .then((img) => {
        socket.emit('frame', {
          buffer: img.toBuffer(),
        });
      });
  };
  const handler = () => {
    camera.read(frameSocketHanlder);
  };
  setInterval(handler, camInterval);
};Copy the code

• Front-end call interface

socket.on('frame', function (data) { var unit8Arr = new Uint8Array(data.buffer); var str = String.fromCharCode.apply(null, unit8Arr); var base64String = btoa(str); img.onload = function () { ctx.drawImage(this, 0, 0, canvas.width, canvas.height); } img.src = 'data:image/png; base64,' + base64String; });Copy the code

These cutting-edge technologies will get more extensive application in the front and support is beyond doubt, the future of the image on the front end will also with the traditional image processing – > learning + image processing way, all the credit for leave the infrastructure (hardware, browser, tools, libraries, etc.) gradually strengthen and perfect, including but not limited to:

• GetUserMedia /Canvas => image/video operation
• Shape Detection API => Image Detection
• Web Workers => Parallel computing capability
• ConvNetJS => Deep learning framework

A. the Demo:

Github.com/x-cold/face…
Github.com/x-cold/face…

B. There is no data support for the adaptability of face detection in different scenarios and the detection time. PASCAL VOC and samples provided by AT&T will be introduced to carry out small-scale tests.