This is the 54th original article without water. If you want to get more original articles, please follow us by scanning the QR code above 👆. This article was first published in the blog of the front-end team of the political Cloud: Face recognition authentication based on the Web end

Results show

preface

In recent years, with the gradual maturity of biometrics technology, face recognition technology based on deep learning has made breakthroughs, and its accuracy has significantly improved. At present, as a very important authentication method, face recognition authentication has been widely used in many industries and fields, such as Alipay payment, face-brushing check-in and so on.

Its advantage lies in that the recognition process is more friendly and convenient by taking the face as the object of recognition. Only the recognized person needs to enter the range of the camera, which will not cause the antipathy and vigilance of the recognized person. At present, the application scenarios in the market mainly focus on mobile terminal, but face recognition authentication schemes based on Web browser are few.

This paper will introduce the whole scheme of face recognition authentication based on Web browser, and focus on how to realize automatic face collection in Web browser.

Scenario description and analysis

  • Application scenario: Face recognition real-name authentication. When using the facial recognition authentication function, users only need to point their faces at the camera, and the program automatically detects the face. If only one face is detected in the visible area of the current camera, the current face picture is collected for face comparison, in vivo detection, ID card recognition and other combination capabilities to quickly complete user identity verification.

  • From the above scenario description, several key problems and solutions are analyzed:

    • Question 1: How to obtain the real-time video stream data shot by the camera?

      • Through the navigator. MediaDevices. GetUserMedia API (based on WebRTC) can obtain the real-time video streaming data to the camera

    • Question two: How to detect and collect the unique face in real-time video stream?

      • Use open source face collection JS library. Need to support single and multiple face detection
      • By horizontal comparison with the commonly used open source JS library for Face collection, face-api.js is superior in performance and accuracy
        • Face-api.js: Based on tensorflow. js kernel, three convolution neural network architectures are implemented for face detection, recognition and feature point detection. Face recognition can be carried out in the browser. Its interior implements a very lightweight, fast and accurate 68-point face marker detector. Support a variety of TF models, small model only 80KB. It also supports GPU acceleration and can be run using WebGL
        • Tracking. Js: a standalone JavaScript library that implements color and human (face, facial features, etc.) tracking. A JavaScript event can be triggered by detecting a specific color, or by detecting the presence and movement of a human/face, and then the face can be captured. Tracking. Js uses CPU to perform calculation, which is slower than GPU in terms of image matrix calculation efficiency

    • Q3: How to implement real-name authentication? How to obtain hd photos on the ID card for comparison?

      • High-definition photos on personal ID cards are not directly available. You can use baidu AI’s real-name identity authentication service to upload the face picture to be compared + ID number + name to Baidu AI service, and the matching degree of the picture will be returned (0~100). Its underlying call is the public security real name authentication interface

    • Question 4: How to realize living detection?

      • Considering the performance of The Web end, it is appropriate for the server to handle the liveness detection. Specific can refer to Baidu AI in vivo detection

The overall plan

  • It is mainly divided into the following key steps:
    • Call the camera (authorized by the user) to obtain the video stream data of the camera
    • Face -api.js is used to identify face feature points in the video stream and locate the position of the face
    • In line with face features, pause the video stream, according to the current frame of the video stream, draw an image
    • Transfer the image to Base64 encoding and upload it to the server
    • The server invokes the interface of Baidu AI, compares it with the picture information on id card, and performs in vivo detection

Implementation details

  • In the above schemes, you must be familiar with camera detection, real-time video stream data acquisition and Canvas picture drawing, so I won’t explain them in detail. Some students did not contact it does not matter, the specific implementation is relatively simple, you can directly look at the source code, the source code about these have detailed annotations

  • Now I will elaborate on how to use face-api.js for face detection in real-time video stream

    • Introducing the face – API

      Script tag to obtain the latest script

      <script src="face-api.js"></script>
Copy the code

      Or use NPM

      npm install face-api.js
Copy the code

    • Loading model data

      Loading model data is an asynchronous operation. After model data loading is completed, face detection can be started

      // Load all model data. Models is the directory where model data files are stored
await faceapi.loadModels('/models'); 
// Load a single specified model data
await faceapi.loadTinyFaceDetectorModel('/models');
await faceapi.loadSsdMobilenetv1Model('/models');
await faceapi.loadMtcnnModel('/models');
Copy the code

    • To detect human faces

      When a face is detected in accordance with the configuration parameters of the model, it will be considered as a detected face, and then return a Detection object, including face matching degree, face coordinates and other information. Can be used for subsequent custom drawing face boundary box, as well as logical threshold judgment

      • Faceapi.detectallfaces: Detects all faces in an image
      • Faceapi.detectsingleface: Detects a single face and returns the face with the highest confidence score in the image
      / / data source support: HTMLImageElement | HTMLVideoElement | HTMLCanvasElement type
// Different models have different configuration parameters, which are described below
const detections1 = awaitFaceapi.detectallfaces (Data source to be detected, model configuration parameters);const detections2 = awaitFaceapi.detectsingleface (data source to be detected, model configuration parameters);Copy the code

    • Introduction of common face detection models

      • Tiny Face Detector is a very high performance real-time Face detection model that is faster, smaller and consumes less resources than the SSD Mobilenet V1 Face detection model. The quantization model is only 190 KB in size (tiny_face_DETECtor_model), but it does a little worse job of detecting small faces. The loading time is about 8 ms
      // Model configuration parameters
 new faceapi.TinyFaceDetectorOptions({
 	// The size of the input data source, the smaller the value, the faster the processing. Common values: 128, 160, 224, 320, 416, 512, 608
    inputSize: number,  // default: 416
    // The fractional threshold used to filter the boundary. If the threshold is greater than or equal to this value, the face is considered to be detected and a detection object is returned
    scoreThreshold: number  / / default: 0.5
 });
Copy the code
      • SSD Mobilenet V1 For face detection, the model implements Mobilenet V1 based SSD (single multi-box detector). The neural network will calculate the position of each face in the image and will return the bounding box as well as the probability of each face. The face detector is designed to achieve high accuracy in detecting face boundary frames rather than low reasoning time. The size of the quantized model is about 5.4 MB (SSD_mobilenetv1_Model). The loading time is about 2-3 seconds
      // Model configuration parameters
new faceapi.SsdMobilenetv1Options({
    // Minimum confidence value, greater than or equal to this value is considered to detect a face, and then return a detection object
	minConfidence: number,  / / default: 0.5
    // Returns the maximum number of faces
    maxResults: number  // default: 100
});
Copy the code
      • MTCNN MTCNN (Multi-task Cascaded Convolutional Neural Network) represents an alternative face detection model to SSD Mobilenet V1 and Tiny Yolo V2, which provides more configuration space. MTCNN should be able to detect various face bounding box sizes by adjusting the input parameters. MTCNN is a 3-level cascade CNN, which simultaneously returns 5 facial landmarks as well as boundary boxes and scores for each face. In addition, the model size is only 2 MB. The loading time is about 1-2s
      // Model configuration parameters
new faceapi.MtcnnOptions({
	// The minimum face size. Faces smaller than this size will not be detected
    minFaceSize: number,  // default: 20
    // The score threshold for filtering boundaries. You can set the threshold of 3 stages of boxes respectively.
    scoreThresholds: number[],  // default: [0.6, 0.7, 0.7]
    // Scale factor is used to calculate the scale step of the image
    scaleFactor: number,  / / default: 0.709
    // The maximum number of scaled versions of the input image through CNN. The smaller the number, the shorter the detection time, but the relative accuracy will be worse.
    maxNumScales: number,  // default: 10
    // Manually set the scaling step to scaleStepsscaleSteps? : number[], });Copy the code

    • Special note:

      • The configuration parameters of the model are very important and need to be fine-tuned slowly to optimize the recognition performance and the correctness of the alignment
      • Measured, the Performance of the Tiny Face Detector model is very good and the detection accuracy is also good. Only when the Face is very small, there will be a large deviation. The best scoreThreshold is 0.6

Matters needing attention

  • Face recognition on the Web side is strongly dependent on the local camera. Therefore, the call of the local camera requires detailed error capture and processing, so as to clearly prompt the user how to operate. All possible errors are enumerated below:
const errorMap = {
	'NotAllowedError': 'Camera is disabled, please enable it in system Settings or browser Settings and try again'.'AbortError': 'Hardware problem causing camera access'.'NotFoundError': 'No available camera detected'.'NotReadableError': 'An error occurred at a hardware, browser, or page level on the operating system, causing the camera to be inaccessible'.'OverConstrainedError': 'No available camera detected'.'SecurityError': 'Camera is disabled, please enable it in system Settings or browser Settings and try again'.'TypeError': 'Type error, no available camera detected'
};

await navigator.mediaDevices.getUserMedia({video: true})
    .catch((error) = > {
        if(errorMap[error.name]) { alert(errorMap[error.name]); }});Copy the code
  • In HTTP mode, Chrome cannot invoke the local camera
    • To ensure security, Chrome supports only HTTPS and localhost to invoke the camera. The camera cannot be invoked under HTTP. If you must use the camera in HTTP, you can only modify the Chrome configuration, but this is not recommended

The source code for

  • View the source code

Further reading

  • What can the front end do in the age of ARTIFICIAL intelligence?
  • Tensorflow.js official documentation
  • Deep Learning in the browser: tensorflow.js
  • face-api.js
  • tracking.js

, recruiting

ZooTeam, a young passionate and creative front-end team, belongs to the PRODUCT R&D department of ZooTeam, based in picturesque Hangzhou. The team now has more than 50 front-end partners, with an average age of 27, and nearly 30% of them are full-stack engineers, no problem in the youth storm group. The members consist of “old” soldiers from Alibaba and NetEase, as well as fresh graduates from Zhejiang University, University of Science and Technology of China, Hangzhou Electric And other universities. In addition to daily business docking, the team also carried out technical exploration and practice in material system, engineering platform, building platform, performance experience, cloud application, data analysis and visualization, promoted and implemented a series of internal technical products, and continued to explore the new boundary of front-end technology system.

If you want to change what’s been bothering you, you want to start bothering you. If you want to change, you’ve been told you need more ideas, but you don’t have a solution. If you want change, you have the power to make it happen, but you don’t need it. If you want to change what you want to accomplish, you need a team to support you, but you don’t have the position to lead people. If you want to change the pace, it will be “5 years and 3 years of experience”; If you want to change the original savvy is good, but there is always a layer of fuzzy window… If you believe in the power of believing, believing that ordinary people can achieve extraordinary things, believing that you can meet a better version of yourself. If you want to be a part of the process of growing a front end team with deep business understanding, sound technology systems, technology value creation, and impact spillover as your business takes off, I think we should talk. Any time, waiting for you to write something and send it to [email protected]