Face recognition algorithm principle
The mainstream face recognition technology can be basically divided into three categories, namely: geometric feature based method, template based method and model based method.
1. Geometric feature-based method is the earliest and most traditional method, which usually needs to be combined with other algorithms to achieve better results; 2. Template-based methods can be divided into correlation matching method, eigenface method, linear discriminant analysis method, singular value decomposition method, neural network method, dynamic connection matching method and so on. 3. Model-based methods include hidden Markov model, active shape model and active appearance model.
1. Method based on geometric features
Face is composed of eyes, nose, mouth, chin and other parts, because of the shape, size and structure of these parts of the various differences in the world each face is very different, so the geometric description of the shape and structure of these parts, can be an important feature of face recognition. Geometric features were first used for the description and recognition of face profile. Firstly, a number of salient points were determined according to the profile curve, and a set of feature measures for recognition such as distance and Angle were derived from these salient points. It is a novel method for Jia et al. to simulate side profile by integral projection near the center line of front gray-scale image.
Frontal face recognition using geometric features is generally performed by extracting the positions of important feature points such as eyes, mouth and nose and geometric shapes of important organs such as eyes as classification features. However, Roder conducted experimental research on the accuracy of geometric feature extraction, and the results are not optimistic.
The deformable template method can be regarded as an improvement of geometric feature method. Its basic idea is to design an organ model with adjustable parameters (deformable template), define an energy function, minimize the energy function by adjusting model parameters, and then the model parameters are regarded as the geometric features of the organ. This method has a good idea, but there are two problems: one is that the weighted coefficients of various costs in the energy function can only be determined by experience, which is difficult to be generalized; the other is that the optimization process of the energy function is time-consuming and difficult to be applied in practice. Parameter-based face representation can achieve an efficient description of salient features, but it requires a lot of pre-processing and fine parameter selection. At the same time, the use of general geometric features only describes the basic shape and structure of the components, ignoring the local subtle features, resulting in the loss of part of the information, more suitable for rough classification, and the existing feature point detection technology is far from meeting the requirements of accuracy, and the amount of calculation is large.
2. Local Face Analysis
Main atoms space is compact, greatly reduced the characteristic dimension of, but it is a localized, the kernel function of support to expand in the whole coordinate space, at the same time, it is a topology, neighboring points after a certain axial projection and the approach of the middle point of the original image space has nothing to do, and local and topology analysis model and segmentation is one of the characteristics of the ideal, It seems that this is more consistent with the mechanism of neural information processing, so it is important to look for expressions with this property. Based on this consideration, Atick proposed a facial feature extraction and recognition method based on local features. This approach has achieved good results in practical applications and forms the basis of FaceIt facial recognition software.
3. Eigenface method (Eigenface or PCA)
The feature face method is one of the most popular algorithms proposed by Turk and Pentland in the early 1990s. It has the characteristics of simple and effective, and is also known as the face recognition method based on Principal Component Analysis (PCA). The basic idea of feature sub-face technology is: from a statistical point of view, to find the basic elements of face image distribution, that is, the feature vector of face image sample set covariance matrix, so as to approximately characterize the face image. These eigenvectors are called eigenfaces.
In fact, the characteristic face reflects the information hidden in the face sample set and the structural relationship of the face. The feature vectors of the sample set covariance matrix of eyes, cheeks and jaws are called feature eyes, feature jaws and feature lips, which are collectively called feature sub-faces. Feature subfaces generate subspaces in corresponding image Spaces, which are called subface Spaces. The projection distance of the test image window in the sub-face space is calculated. If the window image meets the threshold comparison condition, it is judged to be a face. The method based on feature analysis, that is, the relative ratio of face reference points and other shape parameters or category parameters describing face features together constitute the recognition feature vector, this recognition based on the whole face not only retains the topological relationship between face parts, but also retains the information of each part itself. The component-based recognition is designed by extracting local contour information and gray level information. Now Eigenface(PCA) algorithm has become a benchmark algorithm to test the performance of face recognition systems together with the classical template matching algorithm. Since the birth of feature face technology in 1991, researchers have carried out a variety of experiments and theoretical analysis on it. FERET’96 test results also show that the improved feature face algorithm is the mainstream face recognition technology and one of the best performance recognition methods. The method is to determine the size, position, distance and other attributes of the facial contour of iris, nose and mouth, and then calculate their geometric feature quantities, which form a feature vector describing the facial image. The core of its technology is actually “local body feature analysis” and “graphic/neural recognition algorithm.” This algorithm is a method that uses facial organs and characteristic parts of human body. For example, the identification parameters of the corresponding geometric relationship multi-data formation are compared, judged and confirmed with all the original parameters in the database. Turk and Pentland proposed the feature face method, which constructs the principal element subspace according to a group of face training images. Since the principal element has the shape of face, also known as the feature face, the test image is projected onto the principal element subspace during recognition, and a set of projection coefficients are obtained, which are compared with the face images of each known person for recognition. Pentland et al. reported fairly good results, 95% correct recognition rate in 3000 images of 200 people, and only one false recognition rate in 150 frontal face images in FERET database. However, a lot of pre-processing such as normalization needs to be done before the eigenface method is implemented. On the basis of traditional feature faces, researchers have noticed that feature vectors with large eigenvalues (i.e. feature faces) do not necessarily lead to good classification performance. Therefore, a variety of feature (subspace) selection methods have been developed, such as Peng’s Gemini space method, Weng’s linear ambiguity analysis method, Belhumeur’s FisherFace method, etc. In fact, the eigenface method is an explicit principal component analysis method for face modeling, and some linear self-association and linear compression BP networks are implicit principal component analysis methods. They represent the face as a weighted sum of vectors that are principal eigenvectors of the cross-product matrix of the training set, which Valentin discusses in detail. In conclusion, the feature face method is a simple, fast and practical algorithm based on transformation coefficient features. However, it has great limitations because it essentially depends on the gray correlation between the images of the training set and the test set, and requires the test images to be more similar to the training set.
Basic principle of feature face recognition method based on KL transform: KL transform is a kind of optimal orthogonal transformation of image compression, people use it for statistical feature extraction, thus formed the basis of the subspace method of pattern recognition, if the KL transform used in face recognition, will be expected to assume that faces in a low dimensional linear space, and different human faces have separability, due to the high-dimensional image space KL transform can be obtained after a new set of orthogonal basis, So can keep some orthogonal basis, to generate a lower dimensional space face, and the base is based on the analysis of low dimensional space to face the statistical features of the training sample set for, the generation of KL transform matrix can be overall scattering matrix of the training sample set, it can also be between the scattering matrix of the training sample set, can use the same images of people on average for training, In this way, the interference of light and so on can be eliminated to a certain extent, and the calculation amount can be reduced, and the recognition rate will not decrease.
4. Elastic model-based approach
Lades and others for object recognition of distortion invariance dynamic link (DLA) model was proposed, the sparse graph is used to describe objects (see below), the vertex with local energy spectrum multi-scale description tag, edge indicates the topological connection relationship with the geometric distance to tag, then apply the plastic pattern matching technique to find the nearest known graphics. Wiscott et al. improved on this, using the FERET image library to compare 300 face images with another 300 images, achieving 97.3% accuracy. The disadvantage of this method is the huge amount of computation. Nastar models the face image (ⅰ) (x, Y) as a deformable 3D mesh surface (x, Y, I(x, Y)) (as shown in the figure below), thus transforming the face matching problem into an elastic matching problem of deformable surfaces. Finite element analysis is used to deform the surface and judge whether the two images are the same person according to the deformation. The feature of this method is that the space (x,y) and gray scale I(x,y) are both considered in a 3D space, and the experiment shows that the recognition result is obviously better than that of the feature face method. Lanitis et al. proposed a flexible representation model method, which encoded the face into 83 model parameters by automatically locating the salient feature points of the face, and used the discrimination analysis method for face recognition based on shape information. Elastic graph matching technology is a kind of based on the geometric features and analysis of gray distribution information for small ripple bedding the recognition algorithm of combining with the algorithm using the structure of the human face and gray level distribution information, but also has the function of automatic precise positioning facial feature points and has good recognition effect, strong adaptability recognition rate is higher, In FERET test, this technology ranks among the top in several indexes, but its disadvantages are high time complexity, slow speed and complicated implementation.
5. Neural Networks
Artificial neural network is a kind of nonlinear dynamic system, which has good self-organization and adaptive ability. At present, the research of neural network in face recognition is in the ascendant. Valentin proposed a method that firstly extracted 50 principal elements of human face, then mapped them into 5-dimensional space with autocorrelation neural network, and then discriminated them with a common multilayer perceptron. It is good for some simple test images. Intrator et al. proposed a hybrid neural network for face recognition, in which the unsupervised neural network is used for feature extraction and the supervised neural network for classification. Lee et al. described the features of a face with six rules, and then positioned the features of the five features according to the six rules. The geometric distance between the features was input into the fuzzy neural network for recognition, which was much better than the general method based on Euclidean distance. Laurence et al. used the convolutional neural network method for face recognition. Because the convolutional neural network integrates the correlation knowledge between adjacent pixels, the invariance of image translation, rotation and local deformation is obtained to a certain extent, so the recognition results are very ideal. Lin et al. proposed a neural network method based on probabilistic decision making (PDBNN), whose main idea is to use virtual (positive and negative examples) samples for reinforcement and anti-reinforcement learning, so as to obtain ideal probability estimation results, and adopt modular network structure (OCON) to speed up network learning. This method has been well applied in various steps of face detection, face location and face recognition. Other studies include :Dai et al. proposed Hopfield network for low-resolution face association and recognition, Gutta et al. proposed a hybrid classifier model combining RBF and tree classifier for face recognition. Phillips et al. used MatchingPursuit filter for face recognition, while support vector machine in statistical learning theory was used for face classification in China. Neural network method in the application of face recognition method than the foregoing categories to have certain advantages, for many of the laws or rules of face recognition on explicit description is quite difficult, and the neural network method can be gained by studying the process of implicit expression of these laws and rules, it is more flexible, general is easy to implement. Therefore, the recognition speed of artificial neural network is fast, but the recognition rate is low. But the neural network method usually needs to take the face as a one-dimensional vector input, so the input node is huge, and an important goal of its recognition is dimensionality reduction. The algorithm description of PCA: Recognition by principal Component Analysis (Principle Component Analysis, PCA) was proposed by Anderson and Kohonen. Since PCA maximizes the variance of each component of the low-dimensional vector when transforming the high-dimensional vector to the low-dimensional vector, and each component is unrelated to each other, the optimal feature extraction can be achieved.
6. Other methods:
In addition to the above methods, face recognition and other ideas and methods, including some of the following: 1) Hidden Markov Model (Hidden Markov Model) 2) Gabor wavelet transform + graph matching (1) Accurate extraction of facial feature points and matching algorithm based on Gabor engine, with good accuracy. Able to eliminate changes due to facial posture, expression, hairstyle, glasses, lighting environment, etc. (2) Gabor filter limits the Gaussian network function to the shape of a plane wave, and selects the preferred orientation and frequency in filter design, which is sensitive to line edges. (3) However, the recognition speed of this algorithm is very slow, and it is only suitable for the playback recognition of video data, with poor adaptability to the scene.
3) face isodensity line analysis matching method (1) Multiple template matching method this method is to store a number of standard face image templates or face image organ templates in the library, in comparison, all pixels of the sample face image and all templates in the library using normalized correlation measurement for matching. (2) Linear Discriminant Analysis, LDA) (3) the method of eigen face Method of eigen face image as a matrix, the calculation of eigenvalue and corresponding eigen vector as algebraic features for identification, has no advantages of geometric features such as extraction eye mouth nose, but in a single sample recognition rate is not high, and in the face model number when large amount of calculation is big (4) the speaker-dependent LianZi space (FSS) algorithm This technology is derived from but differs in essence from the traditional “feature face” facial recognition method. In the “feature face” method, everyone shares a personal face space, and this method establishes a personal face space for each individual face, which can not only better describe the differences between different individual faces, but also eliminate the intra-class differences and noise that are harmful to recognition as much as possible. Therefore, it has better discrimination ability than traditional “feature face algorithm”. In addition, aiming at the face recognition problem where each individual to be recognized only has a single training sample, a technique of generating multiple training samples based on a single sample is proposed, so that the individual human face space method which requires multiple training samples can be applied to the face recognition problem with a single training sample. (5) Singular Value Decomposition (SVD) is an effective algebraic feature extraction method. Because singular value features are stable in describing images and have important properties such as transpose invariance, rotation invariance, displacement invariance and mirror transform invariance, singular value features can be used as an effective algebraic feature description of images. Singular value decomposition (SVD) has been widely used in image data compression, signal processing and pattern analysis.
7. Major commercial systems for face image recognition
Since the late 1990s, some commercial face recognition systems began to enter the market. Major commercial systems include: ● Visionics’ FaceIt image recognition system, based on local Feature Analysis (LFA) algorithms developed at Rockefeller University; ● Face image recognition/confirmation system, using MIT technology; ● Miros Trueface and eTrue authentication systems, whose core technology is neural network; ● C-VIS face recognition/confirmation system; ● Banque-TEC. Authentication system; Visage Gallery’s identity system, based on MIT Media Lab’s Eigenface technology; ● Plettac Electronic’s FaceVACS Access control System; ● BioID system in Taiwan, which is based on Biometrics system that integrates human face, lip movement and speech.
Among them, FaceIt system is the most representative commercial product, which has been applied in many places. Last year it was used in Britain in an anti-criminal system called Mandrake, which searches video sequences from 144 surveillance cameras for known criminals or suspects, and notifying officers in central control rooms of possible offenders. The author has used FaceIt system and evaluated its various indicators. The results show that the system has good recognition performance under the condition of controlled illumination, quasi-frontal (rotation on three axes is no more than 15 degrees) and no decorations. But were also found in the practical process, only the training set of face image acquisition conditions and test set of face image acquisition conditions are basically identical to has a good recognition performance, otherwise, its performance will drop sharply, especially the illumination changes, posture, black-rimmed glasses, hats, exaggerated expression, beard and hair, a greater influence on its performance.
Testing of face image recognition system
Based on the opposite side as the importance of identification technology in the fields such as military security, the United States department of defense’s ARPA fund established a review of existing surface like recognition technology application, and in August 1994, in March 1995 and September 1996 (March 1997) organized three times like recognition and face to confirm performance evaluation, Its purpose is to show the latest progress and the highest academic level of face recognition research, and to find the main problems facing the existing face recognition technology, and provide direction guidance for future research. Although the test is only available to research institutions in the United States, it has become the de facto accepted standard in the field, and its results are considered to reflect the highest academic level of face-recognition research. According to the FERET ’97 test report published in 2000, the face recognition technology of the University of Southern California (USC), The University of Maryland (UMD), and the Massachusetts Institute of Technology (MIT) has the best recognition performance. In the recognition test of 200 people with similar camera conditions in training set and test set, the recognition rate of nearly 100% was produced by several systems. It is worth mentioning that even the simplest correlation matching algorithm has high recognition performance. In FERET tests with a larger set of subjects (1166 or more), the highest preferred recognition rate is 95% for frontal images collected under the same camera condition. For the test images collected with different cameras and different lighting conditions, the highest preferred recognition rate plummets to 82%. For images taken a year later, the maximum accuracy was only close to 51 percent.
The test results show that the current face image recognition algorithm has poor adaptability to different cameras, different lighting conditions and age changes, which deserves the researchers’ sufficient attention. Moreover, it is worth noting that the face images used in this test are relatively standard frontal face images, with very little gesture change, no exaggerated expressions and ornaments, and no mention of facial hair changes. Therefore, we believe that in addition to the above problems revealed by FERET test, it is also necessary to consider the influence of variable factors such as posture, accessories (glasses, hats, etc.), facial expression, facial hair and so on on face image recognition performance. These factors are also the most critical technical problems inevitably encountered when developing practical face recognition products. In order to further test the performance of commercial image recognition systems and reveal the latest advances in rear image recognition technology before 2000, the U.S. Department of Defense’s Anti-Drug Technology Development Program Office conducted a review of major commercial image recognition systems in the United States in May and June of last year. It’s called FRVT ‘2000 (Face Recognition Vender Test). A total of 24 face recognition system manufacturers in the United States were invited to participate in the program, but only eight responded. Finally, five companies participated in the evaluation, and only three systems completed all comparison tests within the specified time. It can be considered that the products of these three companies are the most competitive business identification systems, they are FaceIt system and Lau Tech. Company’s system and C-VIS’s system. FRVT ‘2000 evaluated the recognition performance of these systems for influencing factors such as image compression, user-camera distance, expression, lighting, recording equipment, posture, resolution and time interval. The results show that the performance of the surface image recognition system has made some progress compared with the test in 1997, but its recognition performance is still far from people’s expectation under various conditions such as illumination, aging, distance, attitude, etc.
Domestic:
Chinese Academy of Sciences – Shanghai Yinchen
In recent years, on the basis of careful research on feature face technology, domestic scholars have tried to combine feature extraction method based on feature face with various back-end classifiers, and put forward a variety of improved versions or extended algorithms. The main research contents include linear/nonlinear discriminant analysis (LDA/KDA), Bayesian probability model, Support vector machine (SVM), artificial neural network (NN) and Inter /intra-class dual subspace analysis methods, etc.
In recent years, on the basis of careful research on the feature face technology, the Institute of Computing Science of the Chinese Academy of Sciences has tried the combination of feature extraction method based on feature face and various back end classifiers, and put forward a variety of improved versions or extended algorithms. The main research contents include linear/nonlinear discriminant analysis (LDA/KDA), Bayesian probability model, Support vector machine (SVM), artificial neural network (NN) and Inter /intra-class dual subspace analysis methods, etc. According to the shortcomings of Eigenface algorithm, a Eigenface subspace (FSS) algorithm is proposed. The technology is derived from, but substantially different from, traditional “feature face” facial recognition methods: “Face” everyone has a personal LianZi space, and the Chinese Academy of Sciences calculation method is established for each individual faces a private person LianZi space by the individual object, which not only can better describe the differences between different individuals face, and the biggest possible devoid of adverse to the identification of class difference and noise, Therefore, it has better discrimination ability than traditional “feature face algorithm”. In addition, aiming at the face recognition problem that each individual to be recognized only has a single training sample, the Institute of Computer Science of the Chinese Academy of Sciences proposed a technique to generate multiple training samples based on a single sample, so that the individual human face space method which requires multiple training samples can be applied to the face recognition problem with a single training sample. Comparison experiments on Yale face database and 350 people image database of our laboratory also show that the proposed method has better robustness and recognition performance than the traditional eigenface method and template matching method for facial expression, illumination, and pose changes within a certain range. Elastic graph matching technology is a kind of based on the geometric features and analysis of gray distribution information for small ripple bedding the recognition algorithm of combining with the algorithm using the structure of the human face and gray level distribution information, but also has the function of automatic precise positioning facial feature points and has good recognition effect, the technology on FERET certain indicators are among the best in the test, Its disadvantages are high time complexity and complex implementation. Institute of Computing Science of Chinese Academy of Sciences has studied this algorithm and put forward some heuristic strategies. 4, face recognition key problem research a) face recognition illumination problem illumination change is the most critical factor affecting the performance of face recognition, the degree of the problem is related to the practical process of face recognition success or failure. On the basis of systematic analysis, the institute of Computer Science of the Chinese Academy of Sciences will consider the possibility of quantitative research, including the quantification of light intensity and direction, the quantification of face reflection attributes, face shadow and illuminance analysis, etc. On this basis, a mathematical model describing these factors is considered, in order to use these lighting models to compensate or eliminate the influence of face image on recognition performance as much as possible in the stage of face image preprocessing or normalization. This paper focuses on how to separate the inherent face attributes (reflectivity attributes, 3D surface shape attributes) from the non-inherent face attributes such as light source, occlusion and highlight. Albedo attribute estimation, 3D surface shape estimation, illumination mode estimation and arbitrary illumination image generation algorithm based on statistical vision model are the main research content of INSTITUTE of Computing Science of Chinese Academy of Sciences. Two different solutions are considered: 1. Estimate the illumination mode using the illumination mode parameter space, and then carry out targeted illumination compensation, so as to eliminate the shadow and highlight caused by uneven frontal illumination; 2. 2. Arbitrary illumination image generation algorithm based on illumination subspace model is used to generate a number of training samples under different illumination conditions, and then use face recognition algorithms with good learning ability, such as subspace method, SVM and other methods for recognition.
B) Research on the pose problem in face recognition The pose problem involves the facial changes caused by the rotation of the head around three axes in the three-dimensional vertical coordinate system, in which the depth rotation in two directions perpendicular to the image plane will cause partial loss of facial information. The pose problem becomes a technical problem of face recognition. There are three ways to solve the attitude problem: The first idea is to learn and memorize many characteristics, for the more attitude face the situation of the data can be easy to get practical, and the unity of frontal face recognition, and its advantage is algorithm does not need additional technical support, its defect is large in the storage requirements and generalization ability can’t be sure, cannot be used for face recognition algorithm based on single photo is medium. The second idea is to generate a multi-angle view based on a single view, which can synthesize multiple learning samples of the user under the condition of only obtaining a single photo of the user, which can solve the multi-pose face recognition problem with less training samples, so as to improve the recognition performance. The third approach is based on attitude invariant features, which seek features that do not change with attitude changes. The idea of the Institute of Computer Science of the Chinese Academy of Sciences is to use a visual model based on statistics to correct the input attitude image into a positive image, so that feature extraction and matching can be done in a unified attitude space.
So much posture view generation algorithm based on single posture view will be calculate the core algorithm to research of Chinese Academy of Sciences, Chinese Academy of Sciences calculate the basic idea is to use the machine learning algorithm learning attitude change of 2 d model, and the face of the 3 d model as a general prior knowledge, compensation of 2 d profile not visible part of the transformation, and its application to the new input image.