In general, license plate recognition requires Sobel location, color location and SVM to evaluate and score the candidate license plate. Finally, it extracts HOG features to enter ANN recognition according to the training model.
This chapter introduces the logical codes related to positioning, in which positioning uses Sobel positioning (edge detection positioning) and color positioning: CarSobelPlateLocation and CarColorPlateLocation in corresponding codes; After both positioning, some candidate images are obtained, and these images are sent to SVM for evaluation. SVM extracts edge information features based on HOG, HOG is similar to LBP which deals with texture features before. The project code is developed on Clion, and the source address is used for license plate positioning
Sobel positioning
CarSobelPlateLocation, noise reduction through the following steps:
- Gaussian blur
- Gray,
- marginalized
- binarization
- Close your operations
Gaussian blur
// Preprocessing: denoising makes the license plate area more prominent
Mat blur;
//1, gaussian blur (smooth) (1, for subsequent operation 2, noise reduction)
GaussianBlur(src, blur, Size(5.5), 0);
//imshow(" blur ",blur);
Copy the codeGray,
Mat gray;
// Grayscale removes color because it does not use noise reduction here
cvtColor(blur, gray, COLOR_BGR2GRAY);
imshow("Gray", gray);
Copy the codemarginalized
Mat sobel_16;
//3, edge detection to make the license plate more prominent in the call to 16 bits to save the data in the subsequent operation and display need to switch back to 8 bits
Sobel(gray, sobel_16, CV_16S, 1.0);
// Convert to 8 bits
Mat sobel;
convertScaleAbs(sobel_16, sobel);
imshow("Sobel", sobel);
Copy the codebinarization
//4. Binary black and white
Mat shold;
// Large law maximum class algorithm
threshold(sobel, shold, 0.255, THRESH_OTSU + THRESH_BINARY);
imshow("Binary", shold);
Copy the codeClose your operations
//5
// Expand the adjacent white areas to form a whole
Mat close;
Mat element = getStructuringElement(MORPH_RECT, Size(17.3));
morphologyEx(shold, close, MORPH_CLOSE, element);
imshow("Closed operation", close);
Copy the codeThe above operation is in the process of noise reduction, the sixth step of the preliminary selection.
The sixth step: maximum area, minimum area. Wide high force.
//6
/ / get a preliminary screening plate profile = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
// Outline detection
vector< vector<Point>> contours;
// Extract the outermost contour and put the result into a sequence of points into the collection
findContours(close, contours, RETR_EXTERNAL, CHAIN_APPROX_NONE);
/ / traverse
vector<RotatedRect> vec_sobel_roi;
for(vector<Point> point:contours){
RotatedRect rotatedRect= minAreaRect(point);
//rectangle(src, rotatedRect.boundingRect(), Scalar(255, 0, 255));
// Perform a preliminary filter to exclude completely incompatible contours (e.g., 1x1, 5x1000)
if(verifySizes(rotatedRect)) { vec_sobel_roi.push_back(rotatedRect); }}Copy the codeFloat aspec (1 * 1, 5* 1000) float aspec
int CarPlateLocation::verifySizes(RotatedRect rotated_rect) {
/ / fault-tolerant rate
float error = 0.75 f;
// The width and height of the model are 136 * 32
// Get the aspect ratio
float aspect = float(136) / float(32);
// Discard the minimum and maximum areas that do not meet the requirements
// Give a rough idea
// Make it as big as possible. This is still preliminary screening.
int min = 20 * aspect * 20;
int max = 180 * aspect * 180;
// Scale float error is also satisfied
// Minimum width and height ratio
float rmin = aspect - aspect * error;
// Maximum aspect ratio
float rmax = aspect + aspect * error;
// The area of the rectangle
float area = rotated_rect.size.height * rotated_rect.size.width;
// The ratio of the rectangle
float r = (float) rotated_rect.size.width / (float) rotated_rect.size.height;
if ((area < min || area > max) || (r < rmin || r > rmax))
return 0;
return 1;
}
Copy the codeTurning oblique pictures straight: affine transformation
Size of rectangle 4. Coordinates of the center point of rectangle 5
void CarPlateLocation::rotation(Mat src, Mat &dst, Size rect_size,
Point2f center, double angle) {
// Get the rotation matrix
Mat rot_mat = getRotationMatrix2D(center, angle, 1);
// Apply affine transformation
Mat mat_rotated;
// After correction, the size will be different, but the diagonal will definitely accommodate
int max = sqrt(pow(src.rows, 2) + pow(src.cols, 2));
// Affine transformation
warpAffine(src, mat_rotated, rot_mat, Size(max, max),
CV_INTER_CUBIC);
imshow("Before rotation", src);
imshow("Rotation", mat_rotated);
// Cut off as much of the license plate as possible
getRectSubPix(mat_rotated, Size(rect_size.width, rect_size.height), center, dst);
imshow("Capture", dst);
mat_rotated.release();
rot_mat.release();
}
Copy the codeColor orientation
HSV color model
Hue (H), saturation (S), brightness (V);
BGR 转成 HSV
cvtColor(src,hsv,COLOR_BGR2HSV);
Copy the codeTonal H
In terms of Angle, the value ranges from 0° to 360°. Starting from red, the value is 0°, green is 120°, and blue is 240°. Their complementary colors are: yellow for 60 degrees, cyan for 180 degrees, magenta for 300 degrees;
Saturation S
Saturation S indicates the degree to which the color approximates the spectral color. A color that can be seen as the result of a certain spectral color mixed with white. The larger the proportion of spectral color, the higher the degree of color close to spectral color, the higher the saturation of color. High saturation, deep and bright color. The white component of spectral color is 0 and the saturation is the highest. The value ranges from 0% to 100%. A larger value indicates a more saturated color.
Lightness V
Brightness indicates the brightness of the color. For the color of the light source, the brightness value is related to the brightness of the illuminator. For object color, this value is related to the transmittance or reflectance of the object. Usually the value ranges from 0% (black) to 100% (white)
In OpenCV, if the HSV data is 8UC, the values are 0-180, 0-255, 0-255, that is, the blue value should be 120
Follow the table above to find the blue area (100 ~ 124), then change the H and S in HSV to 0, and the V to 255. The HSV value for the other areas is 0.
/ / 3
int chanles = hsv.channels();
/ / high
int h = hsv.rows;
// Wide data length
int w = hsv.cols * 3;
// Determine whether the data is stored in a row
// If there is enough memory, MAT's data is stored in a contiguous block of memory
if (hsv.isContinuous()) {
w *= h;
h = 1;
}
for (size_t i = 0; i < h; ++i) {
// HSV uchar = Java byte
uchar *p = hsv.ptr<uchar>(i);
for (size_t j = 0; j < w; j += 3) {
int h = int(p[j]);
int s = int(p[j + 1]);
int v = int(p[j + 2]);
bool blue = false;
/ / blue
if (h >= 100 && h <= 124 && s >= 43 && s <= 255 && v >= 46 && v <= 255) {
blue = true;
}
if (blue){
p[j] = 0;
p[j + 1] =0;
p[j + 2] =255;
}else {
// HSV model H: 0 red is 0 in brightness and saturation, so it becomes black
p[j] = 0;
p[j + 1] = 0;
p[j + 2] = 0; }}}Copy the codeThe following graph is obtained:
Next, extract brightness:
// Pull out the brightness data
// Separate h, s, and v
vector<Mat> hsv_split;
split(hsv, hsv_split);
Copy the codeAnd then, like Sobel, through binarization, grand law, etc
// The whole image + the license plate selected in the preliminary competition + the candidate license plate obtained
tortuosity(src, vec_sobel_roi, dst);
for (Mat s: dst) {
imshow("Candidate", s);
waitKey();
}
Copy the codeFilter out a set:
The two combinations are combined and then evaluated by SVM, because unlike face detection, there is no ready-made model.
vector< Mat > sobel_plates;
/ / sobel positioning
plateLocation->location(src, sobel_plates);
// Color localization
vector< Mat > color_plates;
plateColorLocation->location(src, color_plates);
vector<Mat> plates;
// Add all the contents of sobel_plates to the plates vector
plates.insert(plates.end(),sobel_plates.begin(), sobel_plates.end());
plates.insert(plates.end(), color_plates.begin(), color_plates.end());
Copy the codeSVM
Blog.csdn.net/liukun321/a…
Simply put, SVM is used to distinguish between different types (license plate and non-license plate). SVM training data has both features and labels. Through training, the machine can find the relationship between features and labels by itself, and judge labels when faced with data with only features and no labels. Supervised learning in machine learning. Linear separable, linear indivisible, indivisible with the kernel function to distinguish:
Kernel function: used to extract different types
LBP is used for face detection to extract features, and HOG is adopted here to extract features.
SVM load model, the model is also XML file
svm = SVM::load(svm_model);
CarPlateRecgnize p("/Users/xiuchengyin/Documents/Tina-NDK/OpencvCarRecgnize/resource/HOG_SVM_DATA2.xml");
Copy the codeHOG feature
Locally normalized histogram of gradient direction is an intensive descriptor for local overlapping regions of images. It constructs features by calculating the histogram of gradient direction of local regions.
The width of parameter 1(detection window) – the width of parameter 2(block size) and the remainder of parameter 3(block sliding increment) must be 0 high
Parameter 4 is cell size, and parameter 5 is gradient direction
HOGDescriptor hog(Size(128, 64), Size(16, 16), Size(8, 8), Size(8, 8), 3);
Initialize the HOG variable
// Width of parameter 1 - width of parameter 2 turns out to be the same as height of parameter 3 with remainder 0
svmHog = new HOGDescriptor(Size(128.64),Size(16.16),Size(8.8),Size(8.8),3);
Copy the code
Inspection window is divided into: ((128-16) / 8 + 1) * ((64-16) / 8 + 1) = 105 pieces (Block);
A Block has four cells;
The length of Hog descriptor vector of a Cell is 9;
The feature of statistical gradient histogram is to divide the gradient direction (0-360) into X intervals and convert the image into several Windows of 16×16. Each window is divided into X blocks, and each block is converted into 4 cells (8×8). For each cell, calculate the gradient direction of each pixel point, increase the corresponding bin value according to the gradient direction, and finally synthesize the gradient histogram of N cells to form features.
To put it simply, the edge of the license plate is a group of information composed of the internal text (the gradient value at the edge and corner is very large, and the edge and corner contain a lot of object shape information). HOG is to extract these information to form a histogram.
HOG: The gradient direction weakens the influence of light and is suitable for capturing contour.
LBP: The LBP value of the center pixel reflects the texture information of the surrounding area of the pixel.
SVM scores the candidate images according to the features extracted from HOG and selects the optimal one:
string CarPlateRecgnize::plateRecgnize(Mat src) {
vector< Mat > sobel_plates;
/ / sobel positioning
sobelPlateLocation->location(src, sobel_plates);
// Color localization
vector< Mat > color_plates;
colorPlateLocation->location(src, color_plates);
vector< Mat > plates;
// Add all the contents of sobel_plates to the plates vector
plates.insert(plates.end(),sobel_plates.begin(), sobel_plates.end());
plates.insert(plates.end(), color_plates.begin(), color_plates.end());
int index = - 1;
float minScore = FLT_MAX; // The maximum value of float
// Use SVM for profiling
for (int i = 0; i< plates.size(); ++i) { Mat plate = plates[i];// Grayscale first, then binarization, grayscale only one channel
Mat gray;
cvtColor(plate, gray,COLOR_BGR2GRAY);
// Binarization must be a single channel
Mat shold;
threshold(gray, shold, 0.255, THRESH_OTSU + THRESH_BINARY);
// Extract the feature
Mat features;
getHogFeatures(svmHog, shold, features);
// save the data into a single line
Mat samples = features.reshape(1.1);
// Convert the data store format
samples.convertTo(samples, CV_32FC1 );
// Original mode
// SVM: tells you directly what type of data this data belongs to.
// RAW_OUTPUT: let SVM give a score
// char name[100];
// sprintf(name, "%d", I);
// imshow(name, plate);
float score = svm->predict(samples, noArray(), StatModel::Flags::RAW_OUTPUT);
printf("Grade: %f\n",score);
if (score < minScore) {
minScore = score;
index = i;
}
gray.release();
shold.release();
features.release();
samples.release();
}
Mat dst;
if (index >= 0) {
dst = plates[index].clone();
}
// imshow(" license ", DST);
// waitKey();
/ / release
for (Mat p : plates) {
p.release();
}
return string("123");
}
Copy the codeSVM scores are as follows:
/ Users/xiuchengyin/Documents/Tina - the NDK/OpencvCarRecgnize/cmake - build - the debug/OpencvCarRecgnize score: 1.224322 score: 1.255759 score: 1.831937 Score: -0.070820 Score: 1.525869 score: 1.117042Copy the codeWhat came out of the test was our license plate selection.
Reference: github.com/liuruoze/Ea…
www.cnblogs.com/subconsciou…