This is the 16th day of my participation in the November Gwen Challenge. Check out the event details: The last Gwen Challenge 2021

Detect facial markers using Dlib, OpenCV, and Python

Today, we will use Dlib and OpenCV to detect facial markers in images. Dlib installation tutorial:

Wanghao.blog.csdn.net/article/det…

Face detector model:

Download.csdn.net/download/hh…

Face marker has been successfully applied to face alignment, head posture estimation, face exchange, blink detection and other fields.

In today’s blog post, we’ll focus on the basics of facial markers, including:

1. What exactly are facial markers and how they work.

How to use Dlib, OpenCV and Python to detect and extract facial markers from images.

This post is divided into three main parts:

1. Facial markers and why they are used in computer vision applications will be discussed.

2. Demonstrate how to detect and extract facial markers using Dlib, OpenCV and Python.

3. We’ll look at some of the results of applying face marker detection to images.

What are facial markers?

Face marker detection is a subset of shape prediction. Given an input image (usually an ROI that specifies the object of interest), the shape predictor attempts to locate the key points of interest along the shape.

In the context of face tagging, our goal is to use shape prediction methods to detect important face structures on human faces.

Therefore, detecting facial markers is a two-step process:

Step 1: Locate the face in the image.

Step 2: Detect key facial structures on ROI.

Face detection (Step 1) can be implemented in a number of ways.

We can use The Haar cascade built into OpenCV.

We can use the pre-trained HOG+ linear SVM object detector specifically for face detection tasks.

Or we could even use deep learning-based algorithms to locate faces.

In either case, the actual algorithm used to detect faces in the image is irrelevant. Instead, the important thing is that somehow we get the face bounding box (that is, the (x, y) -coordinates of the face in the image).

Given a face area, we can apply Step 2: Detect key facial structures in the face area.

There are a variety of facial marker detectors, but all methods essentially attempt to locate and tag the following facial areas:

• mouth

• The right eyebrow

• The left eyebrow

• In the right eye

• The left eye

• The nose

• The jaw

The face marker detector included in the DLIB library is an implementation of the one-millisecond face calibration of a series of regression trees proposed by Kazemi and Sullivan (2014).

This method starts with:

• Training set of facial markers marked on images. The images were manually tagged to specify specific (x, y) coordinates for the area around each facial structure.

• Prior, more specifically, is the probability of the distance between pairs of input pixels.

Given this training data, the regression tree set is trained to estimate the face marker position directly from the pixel intensity itself (i.e., no “feature extraction”).

The end result is a facial marker detector that can be used for real-time detection of facial markers with high quality predictions.

Dlib’s face marker detector

The pre-trained face marker detector in the DLIB library is used to estimate the position of the 68 (x, y) coordinates mapped to the face structure.

The index of 68 coordinates can be shown in the following figure:

These annotations are part of the 68-point iBUG 300-W dataset on which the DLIB facial marker predictor was trained.

Of note, there are other styles of facial marker detectors, including a 194-point model that can be trained on the HELEN dataset.

Regardless of which data set is used, the same Dlib framework can be used to train shape predictors on input training data – useful if you want to train facial marker detectors or custom shape predictors.

In the rest of this blog post, I’ll demonstrate how to detect these facial markers in images.

Detect facial markers using Dlib, OpenCV, and Python

This post uses two functions from the imutils library face_utils.py.

The first utility function is rect_to_bb, which stands for “rectangle to border” :

def rect_to_bb(rect): # Take the bounds predicted by dlib and convert them # in the format we normally use (x, y, w, X = rect.left() y = rect.top() w = rect.right() -x h = rect.bottom() -y # return a tuple (x, y, w, h) y, w, h)Copy the code

This function takes a rect argument, which is assumed to be the bounding rectangle generated by the Dlib detector (that is, the face detector).

The RECT object contains the detected (x, y) – coordinates.

However, in OpenCV, we usually think of bounding boxes as “(x, y, width, height)”, so rect_to_bb converts this RECt object to 4-tuple coordinates for convenience.

Second, the shape_to_NP function:

def shape_to_np(shape, dtype="int"): Coords = Np. zeros((68, 2), dType =dtype) # Loop over the 68 face markers and convert them # to the 2-tuple of (x, y) -coordinates for I in range(0, 68): Coords [I] = (shape.part(I).x, shape.part(I).y) # return the list of coordinates (x, yCopy the code

The DLIB face marker detector will return a shape object containing the coordinates of 68 (x, y) of the face marker area.

Using the shape_to_NP function, we can convert this object to a NumPy array.

With these two helper functions, we can now detect facial markers in the image.

Open a new file, name it facial_landmarks. Py, and insert the following code:

# import the necessary packages from imutils import face_utils import numpy as np import argparse import imutils import Ap = argparse.argumentParser () ap.add_argument("-p", "--shape-predictor", required=True, help="path to facial landmark predictor") ap.add_argument("-i", "--image", required=True, help="path to input image") args = vars(ap.parse_args())Copy the code

Import the required Python packages.

The helper functions detailed above are accessed using the face_utils submodule of imUtils.

The dlib will then be imported.

Parse our command-line arguments:

Shape-predictor: This is the path to dLIB’s pre-trained face marker detector. You can download the detector model here, or use the Download section of this article to get code + sample images + pre-trained detectors.

Image: We need to detect the path of the input image of the face marker.

Now that our import and command-line arguments are processed, let’s initialize dlib’s face detector and face marker predictor:

# Initialize dlib's face detector(based on HOG) and then create the # face marker predictor detector = dlib.get_frontal_face_detector() predictor = dlib.shape_predictor(args["shape_predictor"])Copy the code

Initialize dLIB’s pre-trained face detector based on a modification of the directional gradient standard histogram + linear SVM method for object detection.

The facial marker predictor is then loaded using the path provided for shape_Predictor.

However, before we can actually detect facial markers, we first need to detect faces in the input image:

# Load the input image, resize it, Image = imread(args["image"]) image = imutils. Resize (image, width=500) gray = cv2.cvtcolor (image, width=500) Cv2.color_bgr2gray) # Rects = Detector (Gray, 1)Copy the code

Load our input image from disk via OpenCV, then preprocess the image by resizing it to a width of 500 pixels and converting it to grayscale.

Process the boundary box of the face in the detection image.

The first parameter of the detector is our grayscale image (although this method can also be used for color images).

The second parameter is the number of pyramid layers of the image to be applied when amplifying the image before applying the detector (this is equivalent to calculating Cv2.N times on the image).

The advantage of improving the resolution of the input image before face detection is that it may allow us to detect more faces in the image – the disadvantage is that the larger the input image, the more computationally expensive the detection process is.

Given the (x, y) – coordinates of the face in the image, we can now apply face marker detection to each face region:

For (I, rect) in enumerate(rects): # Identify the facial markers of the facial area, Then # convert the face marker (x, y) coordinates to the NumPy array shape = Predictor (Gray, Rect) shape = face_utils. Shape_to_np (shape) # Convert dlib rectangle to OpencV-style bounding box # [that is (x, y, w, h)], Rect_to_bb (rect) cv2. Rectangle (image, (x, y), (x + w, y + h), (255, 255, 0) PutText (image, "Face #{}". Format (I + 1), (x-10, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255,) 0), 2) # Loop the (x, y) coordinates of the face mark # and draw them on the image for (x, y) in shape: Cv2.imshow ("Output", image) cv2.waitKey(0) cv2.imshow("Output", image) cv2.waitKey(0) cv2.imshow("Output", image)Copy the code

Loop each face detection.

For each face detection, 68 (x, y) – coordinates are given, which map to a specific face feature in the image.

The dlib shape object is then converted into a NumPy array with shapes (68,2).

Draw the boundary box around the face detected on the image, draw the index of the face.

Finally, loop over the detected facial markers and draw them separately.

Displays the output image on the screen.

The test results

Open the terminal and enter:

python facial_landmarks.py --shape-predictor shape_predictor_68_face_landmarks.dat --image 11.jpg
Copy the code

conclusion

In today’s blog post, we learned what face tags are and how to detect them using Dlib, OpenCV, and Python.

Detecting facial markers in images involves two steps:

First, we have to locate the face in the image. This can be done using a number of different techniques, but usually involves Haar cascade or HOG+ linear SVM detector (but any method of generating bounding boxes around the face will suffice).

Shape predictors, especially facial marker detectors, are applied to obtain (x, y) – coordinates of the facial region in facial ROI.

Given these facial markers, we can apply a variety of computer vision techniques, including:

Facial part extraction (i.e. nose, eyes, mouth, jaw line, etc.)

Facial alignment

Head pose estimation

Surface exchange

Blink of an eye test

… .

Hope this article will help you!!

For complete code and documentation see:

Download.csdn.net/download/hh…