preface

Face detection and recognition is a popular technology. At present, there are many algorithms for face detection and recognition. The commonly used algorithms for face detection are Dilb, OpenCV, OpenFace, MTCNN and so on. Commonly used face recognition algorithms include FaceNet, InsightFace model and so on.

This article is originally do face recognition project according to others blog do small procedures, the full text is relatively simple, so I introduce the face recognition project before introducing a simple OpenCV face detection.

First, OpenCV face detection principle

OpenCV uses the Cascade classifier based on Haar.

The Cascade Classifier based on Haar feature was described in Paul Viola and Michael Jone’s 2001 paper “Rapid Object Detection Using a 1951 Cascade of Simple” Features “is an effective method for item detection. It is a machine learning approach that trains the Cascade equation from many positive and negative samples and then applies it to other images.

Like the training of many classifiers, we need to train according to a large number of positive samples (pictures containing faces) and negative samples (pictures without faces), so as to obtain features from these images.

Haar features include three types: edge feature, linear feature, center feature and diagonal feature. Each classifier extracts a corresponding feature from the image.

For the image below, the first feature is selected based on the location of the eyes, which are usually darker than the cheeks and nose, i.e. the horizontal black line extracts the darker eyes from the face; The second feature was chosen on the basis that the eyes were darker than the bridge of the nose, that is, the vertical white path extracted the brighter bridge of the nose from the face.

In terms of how to choose the best feature in a large number of features, Adaboost method is adopted to find the best face classification effect and the least error rate of features through machine learning. At the beginning of the training, all the pictures in the training set have the same weight. For the wrongly classified pictures, the weight is increased and the new error rate and new weight are recalculated. Until the error rate or number of iterations is reached.

The final classifier is the weighted sum of these weak classifiers. The reason why it is called weak classifier is that each classifier cannot classify images individually, but they can be grouped together to form a strong classifier. The paper shows that the accuracy of the classifier, which only needs 200 features, reaches 95% in detection. The final classifier has about 6,000 features. (Down from over 160,000 features to 6,000, which is a big improvement)

In fact, the vast majority of an image is not a face. Extracting 6,000 features from every corner of an image would waste an enormous amount of computing resources. If you can find a simple way to detect whether a window is a face area, if the window is not a face area, then just look at it and skip it, and there is no further processing, so that you can focus on identifying areas that might be faces.

To this end, someone introduced the Cascade classifier. Rather than having all 6,000 features in a single window, it divides features into phases and applies them phase by phase (usually the first few phases have only a small number of features). If the window fails detection in the first phase, then it is discarded, regardless of the remaining features. If the detection passes, the characteristics of the second stage are considered and processing continues. If all the stages pass, then this window is the face area.

The author’s detector divides 6000+ features into 38 stages, with the first five stages having 1, 10, 25, 25 and 50 features respectively (the two features mentioned in the previous figure for identifying eyes and nose bridge are actually the two best features obtained in Adaboost). According to the authors, on average only about 10 out of 6,000 + features are needed for each child window.

Two, OpenCV face detection

In fact, OpenCV Harr cascade classifier is a mature classifier that does not need to be trained again and requires very little code.

OpenCV contains many trained classifiers, such as face, eye, smile, and so on. These XML files are stored in opencv/data/haarcascades/folder, can be directly obtained when using.

import cv2

path = '[Picture]' # image address
img = cv2.imread(path) # fetch image

face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades+'haarcascade_frontalface_default.xml') # Get face cascade classifier, '.xml' file contains trained face featuresFaces = face_cascade.detectMultiScale(img,scaleFactor= minNeighbors= times)Face recognition with classifier, return face coordinate list

for (x,y,w,h) in faces: # Operate on every face
    img = cv2.rectangle(img,(x,y),(x+w,y+h),(255.0.0),2) # Draw the face frame, the reciprocal two parameters are the color and line width respectively, where the color adopts BGR format

cv2.imshow('img',img) # Show face box
Copy the code

How about, isn’t it easy ~

Take the poster of “Celebration of Life” as an example, the face area is circled with a blue box.

In addition to face classifiers, there are eye classifiers

eye_cascade = cv2.CascadeClassifier(cv2.data.haarcascades+'haarcascade_eye.xml')
Copy the code

And the smile classifier

smile_cascade = cv2.CascadeClassifier(cv2.data.haarcascades+'haarcascade_smile.xml')
Copy the code

The code is similar to the example code, loading the corresponding model, but in order to save computing resources, need to be detected in the recognized face frame.

Sample code:

for (x,y,w,h) in faces:
    img = cv2.rectangle(img,(x,y),(x+w,y+h),(255.0.0),2)
    # box selects the face region, and performs eye detection in the face region instead of the full image, saving computing resources
    face_area = img[y:y+h, x:x+w]
    eyes = eye_cascade.detectMultiScale(face_area)
    # Human eye cascade classifier engine is used for human eye recognition in face region. The returned eyes is the list of eye coordinates
    for (ex,ey,ew,eh) in eyes:
        # Draw the human eye frame, green, brush width 1
        cv2.rectangle(face_area,(ex,ey),(ex+ew,ey+eh),(0.255.0),1)
Copy the code

Three, call the camera real-time face detection

The principle is also relatively simple, OpenCV has called the computer camera function, and then use the real-time face frame display cycle.

import cv2

face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades+'haarcascade_frontalface_default.xml')

# Call camera camera
cap = cv2.VideoCapture(0) #opencv function, parameter 0 represents to call the computer built-in camera, if changed to 1 to call peripheral camera

while(True) :# Get the footage from the camera
    ret, frame = cap.read()
    faces = face_cascade.detectMultiScale(frame, 1.3.5)
    img = frame
    for (x,y,w,h) in faces:
        # Draw the face frame in blue with a small brush width
        img = cv2.rectangle(img,(x,y),(x+w,y+h),(255.0.0),2)
        
    # Real-time display effect screen
    cv2.imshow('frame2',img)
    Listen for keyboard actions every 5 milliseconds
    if cv2.waitKey(5) & 0xFF == ord('q') :break

Finally, close all Windows
cap.release()
cv2.destroyAllWindows()
Copy the code

Dynamic monitoring is as follows:

(ORIGINALLY I wanted to use my girlfriend’s photo, since I have a lot on my phone, but she didn’t agree, so I chose another photo to test whether I could recognize an unrealistic face.)

The next phase adopts deep learning to realize face recognition (identify corresponding information of people based on detection)

After: MTCNN+FaceNet construction of face recognition in detail

defects

  1. No face region can be recognized when the face is tilted
  2. 2D recognition, the image can be recognized, not according to the face of the 3D modeling

The resources

1. Use Haar Cascade for face recognition

2. Ten lines of Python code for face recognition