FaceRecognition Interface description

/ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * the function name: faceRecognition function description: * * * * parameters: facial recognition [int] recognitionPic: recognized photo, [int] targetFaceIndex: target matching photo index value ** Return: -1 on failure, 0 on success ** Note: ********************************************************************/ int faceRecognition(char *recognitionPic, int targetFaceIndex) { Mat srcImg, targetImg, compImage; IplImage* faceImage; int predict = 0; char cmd[100] = {0}; /* Load face detector */ CascadeClassifier Cascade;if(! cascade.load(cascadeName)) {printf("[cwr] load CascadeClassifier fail\n");
		return- 1; } //model = createEigenFaceRecognizer(0, SIMILAR_VALUE); model = createEigenFaceRecognizer();if (0 != access(TRAIN_XML, F_OK)) {
		trainPicture();
	}
	model->load(TRAIN_XML);
	srcImg = imread(recognitionPic);
	faceImage = detectFace(srcImg, cascade);
	if (faceImage == NULL) return- 1; Mat cutImage(faceImage, 1); resize(cutImage, compImage, Size(DEFAULT_CUT_SIZE_W, DEFAULT_CUT_SIZE_H), ZOOM_SCALE, ZOOM_SCALE); // Scale the face and set it to a specific resolution imwrite(savePicture, compImage); /* Start face comparison */#if 0
	imshow("compare", compImage);
	waitKey(0);
#endif
	int predictLabel = -1;
	double predictConfidence = 0.0;
	model->predict(compImage, predictLabel, predictConfidence);
	printf("[cwr] predictLabel = %d\n", predictLabel);
	printf("[cwr] predictConfidence = ");
	cout<<predictConfidence<<endl;
		
	//predict = model->predict(compImage);
	//printf("[cwr] faceRecognition return predict: %d\n", predict);
#if 1
	if (predictLabel == targetFaceIndex && predictConfidence <= SIMILAR_VALUE) {
		printf("[cwr] Face recognition successful\n");
		s_faceRecognitionSuc++;
	} else {
		printf("[cwr] Face recognition fail\n");
		s_faceRecognitionFail++;
	}
#endif
	printf("[cwr] Face recognition success total cnt = %d, Face recognition fail total cnt = %d, Detect face fail total cnt = %d\n", s_faceRecognitionSuc, s_faceRecognitionFail, s_delectFaceFail);
    return 0;
}
Copy the code

1.1 Mat

(1) What is Mat

The English definition of matrix matrix, hence the extension of the abbreviation Mat, Mat class is simply understood as the container storing picture data. In computer memory, digital images are stored and manipulated in the form of matrices. For example, in MatLab, images are read by a matrix, and in OpenCV, the same is true. In digital image storage, we store the value of each pixel of the image, corresponding to a digital matrix. [Photos are stored as matrices]

The pixel data of a grayscale image is a matrix. The rows of the matrix correspond to the height of the image (in pixels), the columns of the matrix correspond to the width of the image (in pixels), the elements of the matrix correspond to the pixels of the image, and the values of the matrix elements are the gray values of the pixels. Grayscale images are black and white photos, so OpencV operations on images need to be converted to grayscale photos to improve speed. Gray value, as I understand it, is the difference between black and white, light or dark, the relationship between the two.

Image grey value of grey value and the relationship between the pixel values – bingqingsuimeng column – CSDN blog blog.csdn.net/bingqingsui…

The composition of Mat class is composed of two parts of data: the matrix header (including the matrix size, storage method, storage address, etc.) and a pointer to the matrix that stores all pixel values (the matrix can be different dimensions according to the selected storage method).

1.2 IplImage

In OpenCV IplImage is a structure that represents an image, which is also the most important structure from OpenCV1.0 to now. In the previous image representation with IplImage, and the previous OpenCV is written in C language, the interface provided is C language interface. Mat is a C++ class packaged by OpenCV later, used to represent an image, which is basically the same as IplImage, but Mat also adds some image functions;

Turn IplImage Mat

IplImage* faceImage;

Mat cutImage(faceImage, 1);

1.3 OpencV commonly used image processing functions

(1) Resize function description

void resize(InputArray src, OutputArray dst, Size dsize, double fx=0, double fy=0, int interpolation=INTER_LINEAR );

Parameter Description:

SRC: input, the original image, that is, the image to be resized; DST: output an image that has the same content as the original image but is of a different size. Dsize: indicates the size of the output image. If this parameter is not 0, it means that the original image is scaled to the Size specified by the Size(width, height); If this parameter is 0, then the size of the original image after scaling is calculated using the following formula: Dsize = Size(round(fx*src.cols), round(fy*src.rows)) Fx: width scale, if it is 0, then it is calculated as (double) dsie.width /src.cols; Fy: scale in the height direction. If it is 0, then it is calculated as (double) dsie.height /src.rows; Interpolation: This is an interpolation method. After the image is zoomed, the pixel must be recalculated. We use this parameter to specify the pixel recalculation method. INTER_NEAREST - nearest interpolation INTER_LINEAR - bilinear interpolation, if you do not specify the last parameter, The default method is inter_area-resampling using Pixel area relation. It may be a preferred methodforImage decimation, as it gives moire '-free results. But when the image is zoomed, It is similar to the INTER_NEAREST method. INTER_CUBIC - 4x4 pixel neighborhood Lanczos4-8x8 pixel neighborhood Lanczos interpolationCopy the code

[Feel the last parameter need not be set]

Resize (img, DST, Size(300, 200), 0.3, 0.3); // Scale the image by 0.3 and set the resolution to 300, 200

To scale is to scale to the resolution of the original image. If Size already sets resolution, there is no need to set scaling. Resolution Settings are preferred.

But if you want to scale an image down, the easiest way is to adjust the scale. Resize (img, DST, Size(), 0.3, 0.3) Size()

Save the specified resolution resize(img, DST, Size(300, 200), 1, 1) set the desired parameters inside Size().

Precautions for use:

  1. Dsize and fx/fy cannot both be 0, or you can specify dsize and leave fx and fy empty and use the default values, like resize(img, imgDst, Size(30,30)); Alternatively, you can set dsize to 0 and specify fx and fy values, such as fx=fy=0.5, which is equivalent to doubling the size of the original image!
  2. As for the final interpolation method, the default bilinear interpolation is normally sufficient. The efficiency of several commonly used methods is: nearest neighbor interpolation > bilinear interpolation > bicubic interpolation >Lanczos interpolation; But efficiency is inversely proportional to effect, so use it according to your own situation.
  3. Normally, the size and type of the DST image are not known until it is used. The type is inherited from the SRC image and the size is calculated from the original image according to the parameters. Resize (SRC, DST, dst.size(), 0, 0, interpolation); resize(SRC, DST, dst.size(), 0, 0, interpolation);

X,y in OpenCV image; width,height; cols,rows

X: abscissa y: ordinate width: height: height

Cols is for how many columns, which is just x, and rows is for how many rows, which is just y

Imwrite (2)

1. Filename: specifies the name of the file to be written. It will be created by itself (like imwrite(“1.jpeg”, SRC). (img: image to save params: parameter encoding for a particular format Note: whichever format you want to save the image in, you take the suffix.

Imshow (3)
Displays an image inThe specified window. The image was displayed on the specified window. C++: void imshow(const string& winname, InputArray image) Parameters: winname -- Name of the window. Window name. Image - image to be shown.Copy the code
Imread (4)

API is a:

Mat imread(const string& filename, int flags = 1);

Parameter 1: need to load the picture path name, such as “C: / daima practice/opencv/mat3 mat3 / image4. JPG”

Parameter 2: The color type of the loaded image. The default value is 1. If it is 0, the gray level is returned; if it is 1, the original image is returned.

Flags = -1: Imread reads the image as decoded

Flags = 0: imread reads the image in a single-channel manner, i.e., the gray image

Flags = 1: imread reads images in three-channel mode, i.e., color images

(5) waitKey

waitKey(x); First argument: wait for x ms. If a key was pressed during this time, it ends immediately and returns the ASCII code for the pressed key, otherwise -1 is returned

If x=0, then wait indefinitely until a key is pressed and the image will not display properly after imshow without a waitKey statement.

1.4 Face detection

(1) CascadeClassifier is called CascadeClassifier. For OpencV face detection. Cascade.load (cascadeName) initializes the cascade classifier.

You only need to initialize the CascadeClassifier CascadeClassifier. By loading an XML file (face detector), the CascadeClassifier can support different recognition effects based on XML.

Some members of the commonly used specifications, specific can view the source/opencv – 2.4.10 / modules/objdetect/doc/cascade_classification RST

Class CV_EXPORTS_W CascadeClassifier {public: CV_WRAP CascadeClassifier(); CV_WRAP CascadeClassifier(const String& filename); ~CascadeClassifier(); // check whether the cascade classifier is loaded CV_WRAP bool empty() const; CV_WRAP bool load(const String& filename); CV_WRAP bool reads the classifier from the FileStorage noderead( const FileNode& node ); /** Detect objects of different sizes in the input image. The detected objects are returned as a rectangular list. Parameters: image: CV_8U matrix of type objects: vector of rectangles, each of which contains the object being detected. The rectangles can be partially located outside the original image. MinNeighbors: Specifies how many adjacent rectangles should be reserved for each candidate rectangle flags: has the same meaning as the old cascade in cvHaarDetectObjects. It is not used for the new cascading minSize: the minimum size of an object, and objects smaller than this value are ignored. MaxSize: Maximum possible object size, CV_WRAP void detectMultiScale(InputArray image, CV_OUT STD ::vector<Rect>& objects, Double scaleFactor = 1.1, int minNeighbors = 3, int flags = 0, Size minSize = Size(), Size maxSize = Size()); /** detectMultiScale overloaded parameters: image: CV_8U matrix of type objects: vector of rectangles, where each rectangle contains the object being detected. Rectangles can be partially located outside the original image. Detection number vector of the corresponding object. The number of times an object is detected is the number of adjacent actively classified rectangles that are joined together to form object scaleFactor: MinNeighbors: Specifies how many adjacent rectangles should be reserved for each candidate rectangle flags: has the same meaning as the old cascade in cvHaarDetectObjects. It is not used for the new cascading minSize: the minimum size of an object, and objects smaller than this value are ignored. MaxSize: Maximum possible object size, Objects larger than this value are ignored */ CV_WRAP_AS(detectMultiScale2) void detectMultiScale(InputArray image, CV_OUT STD ::vector<int>& objects, CV_OUT STD ::vector<int>& numDetections, double scaleFactor=1.1, int minNeighbors=3, int flags=0, Size minSize=Size(), Size maxSize=Size() ); /** detectMultiScale overloaded function, which allows you to retrieve the final stage decision determinism of a classification that needs to be resolved' outputRejectLevels 'Set totrueAnd to provide' rejectLevels 'and' levelWeights 'Parameters. For each result check, 'levelWeights' will contain classification determinism at the end. This value can be used to distinguish between strong and weak classifications. Specific use of sample code Mat img; vector<double> weights; vector<int> levels; vector<Rect> detections; CascadeClassifier model("/path/to/your/model.xml"); Model.detectmultiscale (img, DETECtions, levels, weights, 1.1, 3, 0, Size(), Size(),true);
    cerr << "Detection " << detections[0] << " with weight "<< weights[0] << endl; */ CV_WRAP_AS(detectMultiScale3) void detectMultiScale( InputArray image, CV_OUT std::vector<Rect>& objects, CV_OUT STD ::vector<int>& rejectLevels, CV_OUT STD ::vector<double>& levelWeights, double scaleFactor = 1.1, int minNeighbors = 3, int flags = 0, Size minSize = Size(), Size maxSize = Size(), bool outputRejectLevels =false );

    CV_WRAP bool isOldFormatCascade() const;
    CV_WRAP Size getOriginalWindowSize() const;
    CV_WRAP int getFeatureType() const;
    void* getOldCascade();

    CV_WRAP static bool convert(const String& oldcascade, const String& newcascade);

    void setMaskGenerator(const Ptr<BaseCascadeClassifier::MaskGenerator>& maskGenerator);
    Ptr<BaseCascadeClassifier::MaskGenerator> getMaskGenerator();

    Ptr<BaseCascadeClassifier> cc;
};
Copy the code

(2) detectMultiScale

Call the member function within the classifier, detectMultiScale parameter 1: image-- images to be detected, generally grayscale images to speed up detection; Parameter 2: objects-- a rectangular box vector set of detected objects; Parameter 3: scaleFactor-- represents the scaling factor of the search window in two successive scans. The default is 1.1, which means that each search window is expanded by 10%. Parameter 4: minNeighbors-- Indicates the minimum number of adjacent rectangles (default: 3) that make up the detection target, and the minimum number of times each face must be detected before it is considered a real face. If the number of small rectangles that make up the detection target is smaller than min_neighbors - 1, both are excluded. If min_neighbors is 0, it returns all candidate rectangles without doing anything. This setting is used in user-defined combinators for detection results. Parameter 5: Flags -- either use the default value or CV_HAAR_DO_CANNY_PRUNING. If CV_HAAR_DO_CANNY_PRUNING is set to CV_HAAR_DO_CANNY_PRUNING, then the function will use Canny edge detection to exclude areas with too many or too few edges. So these areas are not usually where the face is; Determines whether to scale the classifier for detection or to scale the image. Parameters 6, 7: minSize and maxSize are used to limit the range of the resulting target region. Represents the maximum and minimum size of the human face. Parameter 3 setting test situation, refer to the online, visual test does not have any egg. : The recognition rate of 1.1:400 test pictures was 85.3%. The recognition rate of 1.2:400 test images was 78.7%. The recognition rate of 400 test images was 94.0%.#define CV_HAAR_DO_CANNY_PRUNING 1 // This value tells the classifier to skip the smooth (no edge) area
#define CV_HAAR_SCALE_IMAGE 2 // This value tells the classifier not to scale the classifier, but to scale the image
#define CV_HAAR_FIND_BIGGEST_OBJECT 4 tells the classifier to return only the largest target
#define CV_HAAR_DO_ROUGH_SEARCH 8 // It can only be used with the above parameter to tell the classifier to end the search in any window as soon as the first candidate is found.
Copy the code

You can see it in the data folder of sources in the OpenCV installation directory, The folder names “Haarcascades,” “Hogcascades,” and “LBPcascades” refer to the classifier trained by three different characteristics: “Haar,” “HOG,” and “LBP,” respectively: the files in each folder. Haar “feature is mainly used for face detection,” HOG “feature is mainly used for pedestrian detection, and” LBP “feature is mainly used for face recognition.

(3) of the rect

If you create a Rect object Rect (100, 50, 50, 100), then Rect has the following functions:

rect.area(); // Return the area of rect 5000 rect.size(); // Return the size of rect [50 × 100] rect.tl(); // Return the coordinates of the upper left vertex of rect [100, 50] rect.br(); // Return the coordinates of the lower right vertex of rect [150, 150] rect.width(); // Return the width of rect 50 rect.height(); Rect. contains(Point(x, y)); // Return a Boolean variable to determine whether rect contains a Point(x, y)Copy the code

Faces.size() is used to determine whether a face has been detected. Returning 0 means no face has been detected, and returning 1 means a face has been detected. By analogy, this parameter returns the number of detected faces.

1.5 Face Recognition

(1) FaceRecognizer

Face recognition class FaceRecognizer

class FaceRecognizer : public Algorithm
{
public:
    //! virtual destructor
    virtual ~FaceRecognizer() {}

    // Trains a FaceRecognizer.
    virtual void train(InputArray src, InputArray labels) = 0;

    // Updates a FaceRecognizer.
    virtual void update(InputArrayOfArrays src, InputArray labels);

    // Gets a prediction from a FaceRecognizer.
    virtual int predict(InputArray src) const = 0;

    // Predicts the label and confidence for a given sample.
    virtual void predict(InputArray src, int &label, double &confidence) const = 0;

    // Serializes this object to a given filename.
    virtual void save(const string& filename) const;

    // Deserializes this object from a given filename.
    virtual void load(const string& filename);

    // Serializes this object to a given cv::FileStorage.
    virtual void save(FileStorage& fs) const = 0;

    // Deserializes this object from a given cv::FileStorage.
    virtual void load(const FileStorage& fs) = 0;

    // Sets additional information as pairs label - info.
    void setLabelsInfo(const std::map<int, string>& labelsInfo);

    // Gets string information by label
    string getLabelInfo(const int &label);

    // Gets labels by string
    vector<int> getLabelsByString(const string& str);
};
Copy the code
(2) OpencV Ptr

A smart pointer used in OpenCV. Similar to STD ::smart_ptr, but in OpenCV you can easily manage various types of Pointers using Ptr. // We can use Ptr instead of MyObjectType* Ptr. MyObjectType can be a C struct or a C++ class.

OpenCV notes (Ptr) – fireae – blog garden www.cnblogs.com/fireae/p/36… The principle, use of smart pointer in c + + 11, implementation – wxquare – blog www.cnblogs.com/wxquare/p/4…

(3) Currently supported three algorithms

Eigenfaces face createEigenFaceRecognizer () Fisherfaces createFisherFaceRecognizer () LocalBinary Patterns Histograms local binary histogram createLBPHFaceRecognizer()

(4) predict

Method 1:

int predict = 0;
predict = model->predict(compImage);
Copy the code

Passing the photo into the Predict interface returns the closest photo label value from the training photo. train(images,labels); The labels container contains the labels of all training photos. By returning the tag value and their preset driver’s tag comparison, if not the same will show face recognition failure.

Method 2:

Through the threshold setting to control the accuracy of face recognition. In creating the process of face recognition algorithm can also transfer the threshold in the model = createEigenFaceRecognizer (0, SIMILAR_VALUE); As a result, the Predict interface does not return valid data.

int predictLabel = -1;
double predictConfidence = 0.0;
model->predict(compImage, predictLabel, predictConfidence);
	if (predictLabel == targetFaceIndex && predictConfidence <= SIMILAR_VALUE) {
		cout<<"[cwr] Face recognition successful"<<endl;
		s_faceRecognitionSuc++;
	} else {
		cout<<"[cwr] Face recognition fail"<<endl;
		s_faceRecognitionFail++;
	}
Copy the code

Website shows

FaceRecognizer – OpenCV 2.4.13.7 documentation docs.opencv.org/2.4/modules…

FaceRecognizer::predict
C++: int FaceRecognizer::predict(InputArray src) const = 0
C++: void FaceRecognizer::predict(InputArray src, int& label, double& confidence) const = 0
Predicts a label and associated confidence (e.g. distance) for a given input image.
The suffix const means that prediction does not affect the internal model state, so the method can be safely called from within different threads.
The following example shows how to get a prediction from a trained model:
using namespace cv;
// Do your initialization here (create the cv::FaceRecognizer model) ...
// ...
// Read in a sample image:
Mat img = imread("person1/3.jpg", CV_LOAD_IMAGE_GRAYSCALE);
// And get a prediction from the cv::FaceRecognizer:
int predicted = model->predict(img);

Or to get a prediction and the associated confidence (e.g. distance):
using namespace cv;
// Do your initialization here (create the cv::FaceRecognizer model) ...
// ...
Mat img = imread("person1/3.jpg", CV_LOAD_IMAGE_GRAYSCALE);
// Some variables for the predicted label and associated confidence (e.g. distance):
int predicted_label = -1;
double predicted_confidence = 0.0;
// Get the prediction and associated confidence from the model
model->predict(img, predicted_label, predicted_confidence);
Copy the code