Google has come up with an open source, cross-platform, customizable machine learning solution toolkit for online streaming (as well as general video, images, etc.). For those of you who are interested, go to mediapipe.dev/
It provides gesture, body posture, face, object recognition and tracking functions, and provides C++, Python, JavaScript and other programming languages toolkit, as well as iOS, Android platform solutions. Today we will take a look at how to use MediaPipe’s body posture recognition function. Complete a “pull-up check” program in Python.
Python3 needs to be installed on the PC. Python3.8.x is recommended. In addition to installing Opencv-Python, MediaPipe, and Numpy toolkits, you can use PIP to install:
pip install mediapipe numpy opencv-python
My computer is Python3.8.3, each toolkit version is:
Mediapipe = = 0.8.3.1 numpy = = 1.20.2 opencv - python = = 4.5.1.48Copy the codeWrite a poseutil.py module that has a PoseDetector class that provides methods for detecting body posture, obtaining body posture data, and obtaining the angles of human joints. The code is as follows, and the detailed solution is to see the code comment:
import cv2
import mediapipe as mp
import math
class PoseDetector() :
Human posture detection
def __init__(self,
static_image_mode=False,
upper_body_only=False,
smooth_landmarks=True,
min_detection_confidence=0.5,
min_tracking_confidence=0.5) :
Initializing :param static_image_mode: static image (default: no) :param upper_body_only: upper body (default: no) :param smooth_landmarks: Set to True to reduce jitter :param min_detection_CONFIDENCE: minimum confidence value of the staff detection model, default is 0.5 :param min_tracking_confidence: minimum confidence value of the pose trust flag, default is 0.5 ""
self.static_image_mode = static_image_mode
self.upper_body_only = upper_body_only
self.smooth_landmarks = smooth_landmarks
self.min_detection_confidence = min_detection_confidence
self.min_tracking_confidence = min_tracking_confidence
Create a Pose object to detect human body posture
self.pose = mp.solutions.pose.Pose(self.static_image_mode, self.upper_body_only, self.smooth_landmarks,
self.min_detection_confidence, self.min_tracking_confidence)
def find_pose(self, img, draw=True) :
"" Detection posture method: Param IMG: a frame image: Param draw: whether to draw the human body posture node and connection diagram :return: processed image" "
imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# pose. Process (imgRGB) will recognize the human pose data in this frame and save it to self.Results
self.results = self.pose.process(imgRGB)
if self.results.pose_landmarks:
if draw:
mp.solutions.drawing_utils.draw_landmarks(img, self.results.pose_landmarks,
mp.solutions.pose.POSE_CONNECTIONS)
return img
def find_positions(self, img) :
"Get human pose data :param img: a frame image: Param Draw: whether to draw human pose nodes and link diagrams :return: list of human pose data"
# Body posture data list, each member consists of 3 numbers: ID, X, Y
# ID represents a node of the human body, x and y represent coordinate position data
self.lmslist = []
if self.results.pose_landmarks:
for id, lm in enumerate(self.results.pose_landmarks.landmark):
h, w, c = img.shape
cx, cy = int(lm.x * w), int(lm.y * h)
self.lmslist.append([id, cx, cy])
return self.lmslist
def find_angle(self, img, p1, p2, p3, draw=True) :
Param img: a frame image: Param P1: first point: Param P2: second point: Param P3: third point: Param draw: param img: a frame image: Param P1: first point: Param P2: second point: Param P3: third point: Param draw: Whether to draw a connection diagram of 3 points:
x1, y1 = self.lmslist[p1][1], self.lmslist[p1][2]
x2, y2 = self.lmslist[p2][1], self.lmslist[p2][2]
x3, y3 = self.lmslist[p3][1], self.lmslist[p3][2]
# Use the trigonometric formula to obtain the Angle values of 3 points p1-P2-p3, with Angle P2 as the value, 0-180 degrees
angle = int(math.degrees(math.atan2(y1-y2, x1-x2) - math.atan2(y3-y2, x3-x2)))
if angle < 0:
angle = angle + 360
if angle > 180:
angle = 360 - angle
if draw:
cv2.circle(img, (x1, y1), 8, (0.255.255), cv2.FILLED)
cv2.circle(img, (x2, y2), 15, (255.0.255), cv2.FILLED)
cv2.circle(img, (x3, y3), 8, (0.255.255), cv2.FILLED)
cv2.line(img, (x1, y1), (x2, y2), (255.255.255.3))
cv2.line(img, (x2, y2), (x3, y3), (255.255.255.3))
#cv2.putText(img, str(angle), (x2-50, y2+50),cv2.FONT_HERSHEY_SIMPLEX, 2, (0, 255, 255), 2)
return angle
Copy the codeWrite another posetracking.py code that calls methods provided by the PoseDetector class of PoseUtil. Py to get the bending angles of the left and right elbows of the human body from a pull-up video (or live camera video), Suppose we identify a standard pull-up at an elbow Angle from 170 degrees (when fully relaxed) to 20 degrees (when pull-up), then we can measure the number of standard pull-ups completed.
From the figure above, we can see that 11, 13 and 15 are the nodes of the left shoulder, elbow and wrist, while 12, 14 and 16 are the right hand. The complete code is as follows:
import cv2
import numpy as np
from poseutil import PoseDetector
# OpencV opens a video
cap = cv2.VideoCapture('mp4/1.mp4')
Create a PoseDetector object
detector = PoseDetector()
# Direction and completion variables
dir = 0
count = 0
while True:
# Read the video picture frame
success, img = cap.read()
if success:
# Detect human body posture in video picture frames
img = detector.find_pose(img, draw=True)
Get body posture list data
lmslist = detector.find_positions(img)
# Right elbow Angle
right_angle = detector.find_angle(img, 12.14.16)
Measure right elbow bend at 170 to 20 degrees
right_per = np.interp(right_angle, (20.170), (100.0))
Progress bar height data
right_bar = np.interp(right_angle, (20.170), (200.400))
# Use OpencV to draw the progress bar and write the degree of bending of the right elbow
cv2.rectangle(img, (1200.200), (1220.400), (0.255.0), 3)
cv2.rectangle(img, (1200.int(right_bar)), (1220.400), (0.255.0), cv2.FILLED)
cv2.putText(img, str(int(right_per)) + The '%', (1190.450), cv2.FONT_HERSHEY_SIMPLEX, 1, (0.255.255), 2)
# Angle of left elbow
left_angle = detector.find_angle(img, 11.13.15)
left_per = np.interp(left_angle, (20.170), (100.0))
left_bar = np.interp(left_angle, (20.170), (200.400))
cv2.rectangle(img, (500.200), (520.400), (0.255.0), 3)
cv2.rectangle(img, (500.int(left_bar)), (520.400), (0.255.0), cv2.FILLED)
cv2.putText(img, str(int(left_per)) + The '%', (490.450), cv2.FONT_HERSHEY_SIMPLEX, 1, (0.255.255), 2)
# Number of detection, I set here from 20% to 80%, it is considered one
if (left_per >= 80 and right_per >= 80) :if dir= =0:
count = count + 0.5
dir = 1
if (left_per <= 20 and right_per <= 20) :if dir= =1:
count = count + 0.5
dir = 0
# Display the number of completions on the video
cv2.putText(img, str(int(count)), (1000.100), cv2.FONT_HERSHEY_SIMPLEX, 3, (0.255.255), 4)
cv2.imshow('Image', img)
else:
break
k = cv2.waitKey(1)
if k == ord('q') :break
cap.release()
cv2.destroyAllWindows()
Copy the codeRunning the code, we will see the two elbow angles as well as the two progress bars and the number of completions, which can be used to detect the number of completions in a standard pull-up.
