[Image recognition] Gait cycle detection and gait Angle feature based on MATLAB

The framework of gait feature extraction in this experiment is shown in Figure 2-1:

Figure 2-1 Frame of gait feature extraction

1. Gait data

CASIA gait database of Chinese Academy of Sciences is specially used for human gait research, and it has collected a large number of human gait pictures. This time, the human gait pictures in CASIA gait database are used. Using CASIA gait database of Chinese Academy of Sciences, a total of 50~60 gait image sets were selected from 10 different people.

2. Image preprocessing

In the stage of image preprocessing, the main code to achieve five functional modules: gray conversion, binary conversion, image morphology processing, portrait contour extraction, and portrait thinning processing.

3. Gait cycle detection

A complete gait cycle is shown in the gait portrait as the time interval between the legs being opened to the maximum width (or minimum width) during walking and the next time the legs are extended to the maximum width (or minimum width). In this paper, portrait aspect ratio is used for periodic detection.

4. Gait feature extraction

Gait feature extraction is mainly divided into the following steps: extracting the centroid of the portrait, establishing the coordinate system with the centroid as the center, partitioning the portrait, and calculating the feature quantity of each partition.

The commonly used color picture is RGB format, with 24 bit depth, the picture contains too much information, long processing time. The purpose of graying and binarization is to convert RGB images into binary images, although the operation will lose the original image color, gray value size information; But for the portrait image, it retains the contour texture information of the portrait, and greatly reduces the size of the image data, which improves the efficiency of subsequent processing.

Morphology was originally the study of the form and structure of living things. In images, digital morphology is mainly used to represent the content of digital morphology, and mathematical morphology is used as a tool to extract the useful components of image expression and description of regional shape, such as boundary, skeleton and convex shell, etc. Its basic operations include corrosion and expansion, open and close operation, hit and miss transformation, etc. [3].

1. The process of corrosion followed by expansion; Used to eliminate small objects, separate objects at slender points, and smooth the boundaries of larger objects without significantly changing their area. Open operation is usually used to remove small particle noise and break the adhesion between objects. Its main function is similar to that of corrosion. Compared with corrosion operation, it has the advantage of basically keeping the original size of the target unchanged (such as disconnecting the connection between white areas in Figure 4).

1. The process of expansion followed by corrosion; It is used to fill small voids in the body of the object, connect adjacent objects and smooth its boundary without significantly changing its area (as shown in Figure 3-1, black box in white area is filled).

Figure 3-1 Operation of open and close

After obtaining the gray scale image, there are defects of human body in the figure image and the connection parts that should not appear. In this case, all images need to be opened and closed operation. This time, the disk structural element with length 4 (see Figure 3-2 for structural elements) was selected to perform the operation of closing and then opening on the image, so as to fill the defective part and corrod the area that should not be connected. Part of the processing effect is as follows. The main purpose of this open and close operation is to remove the defect and incomplete difference in the gait image and the image misconnection area.

FIG. 3-2 A disk structure element in image processing

The portrait pair after morphological operation is shown in figure 3-3 below. On the left is the unprocessed gait portrait, which has defects and misconnected areas. After morphological processing, the internal defects of the portrait contour have been filled, and the misconnected pixels have been filtered out.

Figure 3-3 Portrait gait contour is processed by open and close operation

The purpose of image contour extraction is to extract human body contour. On the premise of retaining the shape and size of human figures, the calculation amount of a large number of pixels in the contour is reduced. Thus further improving the speed of operation.

In the gray scale image, the edge is caused by the discontinuity or mutation of the gray value of adjacent regions. The first and second derivatives are usually used to detect edges. At the edge position, the amplitude value of the first derivative will appear at the local extreme value, while the amplitude value of the second derivative will appear at the zero crossing point. Therefore, the edge position can be determined by calculating the grayscale derivative and detecting the local extreme point or the zero crossing point. Commonly used edge detection operators include Sobel operator, Laplacian operator and Canny operator.

In this contour extraction, edge detection of Sobel operator (structure as shown in Figure 3-4) is used to detect the edges of characters and obtain contour information. After processing, partial contours are shown in Figure 3-5 below:

Figure 3-4 Sobel operator

Figure 3-5 Portrait contour extraction

ZS thinning algorithm is used to obtain human skeleton. ZS thinning algorithm is an iterative algorithm, which determines whether to delete a pixel point by analyzing the pixel values of each pixel point in the eight neighborhood of a pixel point P1 in the image. The neighborhood position relationship is shown in Figure 3-6.

Figure 3-6 Neighborhood location relation of ZS refinement algorithm

The iteration process of ZS thinning algorithm is divided into two steps:

1) Loop all pixels with pixel value 1 and delete the pixels that meet the following four conditions simultaneously:

In the formula, N(P1) represents the number of pixels with pixel value 1 in the 8 neighborhood pixels of P1, and S(P1) represents the number of pixels with pixel value changing from 0 to 1 in the order of P2, P3, P4, P5, P6, P7, P8, P9 and P2 in the neighborhood of P1.

P4.p8 =0; p4.p8 =0; p6.p8 =0

The loop iterates over the above two steps until no pixels are removed; Figure 3-7-a shows the refined skeleton in the gait diagram. After the human skeleton is obtained, the contour diagram based on skeleton is generated by combining portrait contour and skeleton, as shown in 3-7:

Figure 3-7 Extracting the skeleton and superimposing it with the contour to generate the contour with the skeleton

Gait is the periodic performance of human walking process. As shown in Figure 4-1, the height of the human body is relatively fixed, so when walking, the height of the portrait will not change. In the process of walking, the human leg is in a periodic swing, so that the width of the portrait will change periodically with the swing of the leg; When the leg stride is the largest, the portrait width is the largest, and when the leg stride is the smallest, the portrait width is the smallest.

Figure 4-1 Changes in width and height during walking

In this case, the image width to height ratio is used as the feature of periodic detection. The gait cycle is calculated according to the ratio of width to height of human body area in the gait diagram. The change of width to height ratio of a gait sequence is shown in Figure 4-2.

Figure 4-2 Variation curve of aspect ratio during human walking

The characteristic of width to height ratio of gait can approximate the periodic change of gait. Therefore, on the premise of obtaining the characteristics of the width to height ratio, the minimum value of the first appearance of the width to height ratio is the starting point of the gait cycle, and the minimum value of the third appearance is the end point of the gait cycle. By this method, the figure photo corresponding to a gait period is extracted from the image set, so as to complete the gait period extraction operation, which is denoted as T.

Figure 4-3 Intercepts the aspect ratio of a gait cycle

Based on the aspect ratio feature, gait cycle detection and extraction of different pedestrian gait images can not only get different pedestrian gait cycle portrait pictures; It can also make comparative analysis according to the aspect ratio of different pedestrians in the cycle.

Figure 5-1 and Table 5-1 can be used to understand the features of portrait aspect ratio, which can not only separate the gait cycle, but also find differences in portrait aspect ratio curves of different pedestrians, such as different cycle time, peak and trough occurrence time and amplitude. Therefore, width to height ratio can be used as a preliminary classification gait feature.

Figure 5-1 Intercepted variation curves of aspect ratio characteristics of different people in a gait cycle

Considering that the same person in different gait, leg swing amplitude is different, the distance and Angle between the leg and the center of mass is different; So the distance Angle is another feature here.

Firstly, the centroid of the portrait contour was obtained, and then a new coordinate system was established with the centroid as the center. The lower body contour of the human body was divided into left and right regions, and the Angle characteristics between pixel and centroid in each region were obtained respectively.

5.2.1 pretreatment

1. Centroid acquisition

Firstly, input the gait contour image, scan the contour image successively from top to bottom and left to right, accumulate the horizontal and vertical coordinate values of the target pixels with non-zero pixel value, and add up the number of pixels. Finally, calculate the average value of the sum of horizontal and vertical coordinate. Figure 5-2 shows the following figure. The specific formula is as follows:

Where, N is the number of gait contour pixels, xi and yi are the horizontal and vertical coordinates of the ith pixel.

Figure 5-2 Center of mass of a portrait

2. Establish a new coordinate system with the center of mass as the center

On the premise of obtaining the centroid of portrait contour; The coordinate system was established with the center of mass as the center, the horizontal axis as the X axis and the vertical axis as the Y axis. And the position of contour pixel is converted to the position in the coordinate system, as shown in Figure 5-3 below.

Figure 5-3 Creating a new coordinate system

5.2.2 Angle characteristics

Taking the newly established coordinate system as the base coordinate system and the center of mass as the center; Divide the portrait into the upper and lower parts, and divide the lower part of the portrait into left and right sections. Then calculate the included Angle between the contour pixels in the sections and the X axis, as shown in Figure 5-4.

Figure 5-4 Partition the lower part of the portrait

The included Angle θ I of the ith pixel is calculated by the following formula:

Where, θ I is the included Angle of the ith pixel, xi and yi are the horizontal and vertical coordinates of the ith pixel.

1. Angle characteristics of each zone of a single person

After calculating the included Angle of each pixel, sum and average are taken as the Angle value of the partition. Thus, the corresponding Angle features are obtained for each frame of the image. The change curve of Angle features obtained from the gait frame image in the process of solo walking is shown in Figure 5-5 below.

Figure 5-5 Angle characteristic curves of the two zones in the process of the same person walking

According to the Angle characteristic curve of the gait, it can be found that the Angle characteristic of the left and right zones of the pedestrian has a quasi-periodic change. In the figure, the whole gait image has 50 frames; The period of the Angle feature is 14 to 27 frames as an example, and a period is 13 frames.

As shown in Figure 5-6 below, its Angle features also show a quasi-symmetric distribution along the number axis of 0 (i.e. the vertical axis here). And the amplitude of Angle variation between the two zones is between 20° and 40°.

Figure 5-6 Angle characteristic curves of the two zones in the process of the same person walking

2. Angle characteristics of different pedestrian zones

As shown in Figure 5-7, the Angle characteristic curves of 5 people in their respective left and right zones are intercepted. By intuitively comparing the gait Angle characteristics of different people, it can be found that the Angle characteristic curves of different people reflect different peaks, but the distribution of curves in each zone is relatively concentrated, and the trend is basically the same, and its amplitude varies within 20°.

Figure 5-7 Angle characteristic curves of the two zones in the walking process of different people are intercepted

Figure 5-8 and Table 5-2 show the Angle characteristic change curve of gait frame sequence of a gait cycle extracted from the above five people, as well as their respective period, maximum stride Angle, corresponding time and other information. It can be seen from the curve that a gait cycle contains two angular characteristic cycles. In the actual walking process, a gait cycle can be divided into the alternation of left and right legs, that is, the minimum leg span is the starting point, and the minimum third span is the end point of the cycle. During this period, the alternation of left and right legs occurs once each, and the Angle characteristics of left and right legs will appear a transition from the minimum to the maximum and then to the minimum.

It can be seen from the figure that a gait cycle contains two angular feature cycles. Moreover, different people have different periods and different peaks and troughs. The Angle feature can reflect the periodicity of gait and the difference of different people’s gait.

Figure 5-8 Angle characteristic curves of 2 sections in a gait cycle of 5 people were intercepted

Complete code downloadwww.cnblogs.com/ttmatlab/p/…