This is my sixth day of the August Challenge

Software and Hardware Environment

  • Ubuntu 18.04 64 – bit
  • GTX 1070Ti
  • Anaconda with python 3.6
  • Face_recognition 1.2.3
  • dlib
  • opencv

Face_recognition profile

Labeled as the world’s simplest Python-based face recognition library, Face_Recognition is an integration of the well-known deep learning framework DLIB (Labeled Faces in the Wild), with 99.38 accuracy on LFW(Labeled Faces in the Wild). In addition, Face_Recognition provides the corresponding command line tool, you can use the command line to image folder face recognition, very cool.

Install face_recognition

You can install it using PIP

pip install face_recognition
Copy the code

Or install from the source code

git clone https://github.com/ageitgey/face_recognition.git
cd face_recognition
python setup.py install
Copy the code

Face_recognition workflow

Find out the face

The first step is to locate all the faces in a given picture

import face_recognition

image = face_recognition.load_image_file("your_file.jpg")
face_locations = face_recognition.face_locations(image)
Copy the code

Operating feature point

Find out each person’s eyes, nose, mouth and chin

import face_recognition

image = face_recognition.load_image_file("your_file.jpg")
face_landmarks_list = face_recognition.face_landmarks(image)
Copy the code

For identification

The final and most crucial step is to identify who the face belongs to. Face_recognition used Euclidean distance (you can refer to my another article xugaoxiang.com/2019/11/30/…). To make sure it’s the same face.

import face_recognition

known_image = face_recognition.load_image_file("biden.jpg")
unknown_image = face_recognition.load_image_file("unknown.jpg")

biden_encoding = face_recognition.face_encodings(known_image)[0]
unknown_encoding = face_recognition.face_encodings(unknown_image)[0]

results = face_recognition.compare_faces([biden_encoding], unknown_encoding)
Copy the code

Command line tools

Face_recognition provides two command-line tools

  • face_detection– Find the face in the given image or image folder
  • face_recognition– Run face recognition

You can use –help to see the parameters. Face_detection is easy, so I won’t go into that here.

The command line tool face_recognition is used

To do face recognition, you must first prepare the information of the known person, which is stored in a separate folder, known, and then each person’s picture is ordered by his name, such as Joe Biden.jpg or Kobe.jpg.

Then place the image to be identified in another folder, such as Unknown, and run the following command to identify it

face_recognition known unknown
Copy the code

By default, without a threshold, the correct recognition rate is very, very low. In the actual situation, you need to adjust the threshold appropriately according to your own situation. In the same test environment, set the similarity threshold to 0.38 and the identification result is correct.

Face_recognition known unknown --tolerance 0.38Copy the code

“Unknown_person” in the print indicates a stranger

Another useful parameter is –cpus, which you can use to improve recognition speed if you are using a multi-core CPU. –cpus=-1 indicates that all CPU cores are used.

KNN classifier

KNN(K-nearest Neighbor) is classified by measuring the distance between different feature values. The idea is that a sample belongs to a category if most of the k most similar (i.e. closest neighbors) samples in the feature space belong to that category, where K is usually an integer not greater than 20. In the KNN algorithm, the selected neighbors are all objects that have been correctly classified. This method only determines the category of the samples to be divided according to the category of the nearest one or several samples.

In actual projects, a classifier is generally used to store the data of known faces for convenient management. Face_recognition uses a classifier based on KNN algorithm.

Making a classifier

The code is from examples/face_recognition_knn.py, but the comments are too detailed to explain. When tested, the more images of the same person, the more accurate they were.

def train(train_dir, model_save_path=None, n_neighbors=None, knn_algo='ball_tree', verbose=False): """ Trains a k-nearest neighbors classifier for face recognition. :param train_dir: directory that contains a sub-directory for each known person, with its name. (View in source code to see train_dir example tree structure) Structure: < train_dir > / ├ ─ ─ < person1 > / │ ├ ─ ─ < somename1 >. Jpeg │ ├ ─ ─ < somename2 >. Jpeg │ ├ ─ ─... ├ ─ ─ < person2 > / │ ├ ─ ─ < somename1 >. Jpeg │ └ ─ ─ < somename2 >. Jpeg └ ─ ─... :param model_save_path: (optional) path to save model on disk :param n_neighbors: (optional) number of neighbors to weigh in classification. Chosen automatically if not specified :param knn_algo: (optional) underlying data structure to support knn.default is ball_tree :param verbose: verbosity of training :return: returns knn classifier that was trained on the given data. """ X = [] y = [] # Loop through each person in the training set for class_dir in os.listdir(train_dir): if not os.path.isdir(os.path.join(train_dir, class_dir)): continue # Loop through each training image for the current person for img_path in image_files_in_folder(os.path.join(train_dir, class_dir)): image = face_recognition.load_image_file(img_path) face_bounding_boxes = face_recognition.face_locations(image) if len(face_bounding_boxes) ! = 1: # If there are no people (or too many people) in a training image, skip the image. if verbose: print("Image {} not suitable for training: {}".format(img_path, "Didn't find a face" if len(face_bounding_boxes) < 1 else "Found more than one face")) else: # Add face encoding for current image to the training set X.append(face_recognition.face_encodings(image, known_face_locations=face_bounding_boxes)[0]) y.append(class_dir) # Determine how many neighbors to use for weighting in  the KNN classifier if n_neighbors is None: n_neighbors = int(round(math.sqrt(len(X)))) if verbose: print("Chose n_neighbors automatically:", n_neighbors) # Create and train the KNN classifier knn_clf = neighbors.KNeighborsClassifier(n_neighbors=n_neighbors, algorithm=knn_algo, weights='distance') knn_clf.fit(X, y) # Save the trained KNN classifier if model_save_path is not None: with open(model_save_path, 'wb') as f: pickle.dump(knn_clf, f) return knn_clfCopy the code

Using a classifier

Def predict(X_img_path, knn_clf=None, model_path=None, distance_threshold=0.6): """ Recognizes faces in given image using a trained KNN classifier :param X_img_path: path to image to be recognized :param knn_clf: (optional) a knn classifier object. if not specified, model_save_path must be specified. :param model_path: (optional) path to a pickled knn classifier. if not specified, model_save_path must be knn_clf. :param distance_threshold: (optional) distance threshold for face classification. the larger it is, the more chance of mis-classifying an unknown person as a known one. :return: a list of names and face locations for the recognized faces in the image: [(name, bounding box), ...] . For faces of unrecognized persons, the name 'unknown' will be returned. """ if not os.path.isfile(X_img_path) or os.path.splitext(X_img_path)[1][1:] not in  ALLOWED_EXTENSIONS: raise Exception("Invalid image path: {}".format(X_img_path)) if knn_clf is None and model_path is None: raise Exception("Must supply knn classifier either thourgh knn_clf or model_path") # Load a trained KNN model (if one was passed in) if knn_clf is None: with open(model_path, 'rb') as f: knn_clf = pickle.load(f) # Load image file and find face locations X_img = face_recognition.load_image_file(X_img_path) X_face_locations = face_recognition.face_locations(X_img) # If no faces are found in the image, return an empty result. if len(X_face_locations) == 0: return [] # Find encodings for faces in the test iamge faces_encodings = face_recognition.face_encodings(X_img, known_face_locations=X_face_locations) # Use the KNN model to find the best matches for the test face closest_distances = knn_clf.kneighbors(faces_encodings, n_neighbors=1) are_matches = [closest_distances[0][i][0] <= distance_threshold for i in range(len(X_face_locations))] # Predict classes and remove classifications that aren't within the threshold return [(pred, loc) if rec else ("unknown", loc) for pred, loc, rec in zip(knn_clf.predict(faces_encodings), X_face_locations, are_matches)]Copy the code

CUDA acceleration

If want to lift the performance, it must be on the graphics card, because face_recognition dependence and dlib, so need to install support CUDA dlib, can refer to another article xugaoxiang.com/2019/12/13/…

import face_recognition

image = face_recognition.load_image_file("my_picture.jpg")
face_locations = face_recognition.face_locations(image, model="cnn")

# face_locations is now an array listing the co-ordinates of each face!
Copy the code

Linkage with the Camera

Opencv is used to achieve this. Opencv reads the data of each frame of the camera, and then converts the resize and color space (converted from BGR used by OpECV to RGB used by Face_recognition), and finally performs face detection and recognition.

import face_recognition
import cv2

video_capture = cv2.VideoCapture(0)

# Load a sample picture and learn how to recognize it.
obama_image = face_recognition.load_image_file("obama.jpg")
obama_face_encoding = face_recognition.face_encodings(obama_image)[0]

# Load a second sample picture and learn how to recognize it.
biden_image = face_recognition.load_image_file("biden.jpg")
biden_face_encoding = face_recognition.face_encodings(biden_image)[0]

# Create arrays of known face encodings and their names
known_face_encodings = [
    obama_face_encoding,
    biden_face_encoding
]
known_face_names = [
    "Barack Obama",
    "Joe Biden"
]

# Initialize some variables
face_locations = []
face_encodings = []
face_names = []
process_this_frame = True

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()

    # Resize frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:
        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
            name = "Unknown"

            # If a match was found in known_face_encodings, just use the first one.
            if True in matches:
                first_match_index = matches.index(True)
                name = known_face_names[first_match_index]

            face_names.append(name)

    process_this_frame = not process_this_frame


    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()

Copy the code

In addition, the local video file, webcam (RTSP based video stream) recognition scene is very similar to the local camera processing, so I will not continue to post code here.

The resources

  • Github.com/ageitgey/fa…
  • medium.com/@ageitgey/m…