In this article, we will use Python to detect face and hand markers. We will use a module

Solution for detecting all face and hand markers. In addition, we will look at how to obtain different facial and hand markers that can be used in different computer vision applications such as sign language detection, drowsiness detection, etc

The required modules

  • Mediapipe is a cross-platform library, developed by Google, that provides amazing off-the-shelf ML solutions for computer vision tasks.

  • OpenCVPython library is a computer vision library widely used in image analysis, image processing, detection, recognition and other fields.

Install the required libraries

pip install opencv-python mediapipe msvc-runtime
Copy the code

Here is a step-by-step approach to face and hand landmark detection.

Step 1: Import all the necessary libraries, in this case only two.

Python 3

# Import Libraries

import cv2

import time

import mediapipe as mp

Step 2: Initialize the overall model and drawing function to detect and draw landmarks on the image.

Python 3

# Grabbing the Holistic Model from Mediapipe and

# Initializing the Model

mp_holistic = mp.solutions.holistic

holistic_model = mp_holistic.Holistic(

Min_detection_confidence = 0.5,

Min_tracking_confidence = 0.5

)

# Initializing the drawng utils for drawing the facial landmarks on image

mp_drawing = mp.solutions.drawing_utils

Let’s examine the parameters of the overall model:

Holistic(static_image_mode=False, model_complexity=1, smooth_landmarks=True, min_DETECtion_confidence =0.5, Min_tracking_confidence = 0.5)Copy the code
  • ** Static image mode: ** This is used to specify whether the input image must be treated as a static image or a video stream. The default value is false.
  • ** model complexity: ** it describes the complexity of the pose landmark model: 0,1, or 2. As model complexity increases, landmark accuracy and delay increase. The default value is 1.
  • ** Smooth landmarks: ** This parameter reduces the jitter in the prediction by filtering the pose markers of different input images. The default value is True.
  • ** Minimum detection confidence: ** This is used to specify the minimum confidence value for detection success from the human-detection model. You can specify a value in [0.01.0]. The default value is 0.5.
  • ** Minimum trace confidence: ** This is used to specify the minimum confidence value for detection success from the landmark trace model. You can specify a value in [0.01.0]. The default value is 0.5.

Step 3: Detect landmarks of the face and hand from the image. The holistic model processes the images to generate landmarks for the face, left hand and right hand, and detects them

  1. Use OpenCV to capture successive frames from the camera.
  2. The BGR image is converted to RGB image and the prediction is made using the initialized global model.
  3. The predictions made by the overall model are stored in the result variables, from which we can use sul.Faces_landmark, sul. Right_Hand_landmark, and Resul respectively. Left _Hand_landmark to access landmarks.
  4. Use the drawing function to draw detected landmarks on the image.
  5. Display the resulting image.

Python 3

# (0) in VideoCapture is used to connect to your compyter's default camera

capture = cv2.VideoCapture( 0 )

# Initializing current time and precious time for calculating the FPS

previousTime = 0

currentTime = 0

while capture.isOpened():

# capture frame by frame

ret, frame = capture.read()

# resizing the frame for better view

frame = cv2.resize(frame, ( 800 , 600 ))

# Converting the from from BGR to RGB

image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

# Making predictions using holistic model

# To improve performance, optionally mark the image as not writeable to

# pass by reference.

image.flags.writeable = False

results = holistic_model.process(image)

image.flags.writeable = True

# Converting back the RGB image to BGR

image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)

# Drawing the Facial Landmarks

mp_drawing.draw_landmarks(

image,

results.face_landmarks,

mp_holistic.FACE_CONNECTIONS,

mp_drawing.DrawingSpec(

color = ( 255 , 0 , 255 ),

thickness = 1 ,

circle_radius = 1

),

mp_drawing.DrawingSpec(

color = ( 0 , 255 , 255 ),

thickness = 1 ,

circle_radius = 1

)

)

# Drawing Right hand Land Marks

mp_drawing.draw_landmarks(

image,

results.right_hand_landmarks,

mp_holistic.HAND_CONNECTIONS

)

# Drawing Left hand Land Marks

mp_drawing.draw_landmarks(

image,

results.left_hand_landmarks,

mp_holistic.HAND_CONNECTIONS

)

# Calculating the FPS

currentTime = time.time()

fps = 1 / (currentTime - previousTime)

previousTime = currentTime

# Displaying FPS on the image

cv2.putText(image, str ( int (fps)) + " FPS" , ( 10 , 70 ), cv2.FONT_HERSHEY_COMPLEX, 1 , ( 0 , 255 , 0 ), 2 )

# Display the resulting image

cv2.imshow( "Facial and Hand Landmarks" , image)

# Enter key 'q' to break the loop

if cv2.waitKey( 5 ) & 0xFF = = ord ( 'q' ):

break

# When all the process is done

# Release the capture and destroy all windows

capture.release()

cv2.destroyAllWindows()

The overall model produced 468 front landmarks, 21 left landmarks and 21 right landmarks. Individual landmarks can be accessed by specifying the index of the desired landmark. Example: result. left _HAND_landmark. Landmark [0]. You can get the index of all individual landmarks using the following code:

Python 3

# Code to access landmarks

for landmark in mp_holistic.HandLandmark:

print (landmark, landmark.value)

print (mp_holistic.HandLandmark.WRIST.value)

HandLandmark.WRIST 0
HandLandmark.THUMB_CMC 1
HandLandmark.THUMB_MCP 2
HandLandmark.THUMB_IP 3
HandLandmark.THUMB_TIP 4
HandLandmark.INDEX_FINGER_MCP 5
HandLandmark.INDEX_FINGER_PIP 6
HandLandmark.INDEX_FINGER_DIP 7
HandLandmark.INDEX_FINGER_TIP 8
HandLandmark.MIDDLE_FINGER_MCP 9
HandLandmark.MIDDLE_FINGER_PIP 10
HandLandmark.MIDDLE_FINGER_DIP 11
HandLandmark.MIDDLE_FINGER_TIP 12
HandLandmark.RING_FINGER_MCP 13
HandLandmark.RING_FINGER_PIP 14
HandLandmark.RING_FINGER_DIP 15
HandLandmark.RING_FINGER_TIP 16
HandLandmark.PINKY_MCP 17
HandLandmark.PINKY_PIP 18
HandLandmark.PINKY_DIP 19
HandLandmark.PINKY_TIP 20
0
Copy the code

​

​

​

Output:

​​​​​

​

​​​

Watch the reader! Don’t stop learning now. Master all important machine learning concepts

That’s the end of it. Did you learn?

Full source point: here to pick up