First, SVM support vector machine

1. Principle of SVM

In machine learning, the supervised learning model of support vector machines and associated learning algorithms can be analyzed for data classification and regression analysis. Given a set of training sample, each training sample is marked as belonging to one or another of the two categories, the SVM training algorithm to establish a model of new examples will be assigned to a category or another category, making it a nonprobability binary linear classifier (although methods such as Platt zoom exist in setting of using the SVM classification probability). The SVM model is a representation of these examples as a mapping of points in space in order to divide the examples of each category by as wide a distinct gap as possible. The new examples are then mapped into the same space and predicted to fall into categories based on which side they fall on.

For example, we have two colored balls on the table and we want to separate them.

We got a stick and we put it on the table, didn’t we do a nice job?

Some bitch came and put more balls on the table, which worked well, but one of the balls was on the wrong side and now there might be a better place to put the stick.

SVM tries to place the bar in the best position by leaving as much space as possible on either side of the bar.

Now, when the bitch comes back, the stick is still a good place to be.

The SVM toolkit has another, more important technique. Bitch has seen how good you are with a stick, so he’s given you a new challenge.

There’s no stick in the world that you can split the ball with, so what do you do? Of course you did! Throw the ball in the air. Then, using your professional ninja skills, you grab a piece of paper and slide it between the balls.

Now, bitches stand and watch, and their balls will look like they’re separated by curved lines.

After that, bored grown-ups call the balls “data,” the stick “classifier,” the maximum gap trick “optimization,” the table thumping “kernelling,” and the paper “Hyperplane.”

2, OPencV SVM parameters

The Support Vector Machines class CV :: ML ::SVM class is declared in include/ Opencv2 /ml.hpp. Realize in modules/ml/ SRC /svm. CPP file, it supports both classification, also support multiple classification, also support regression, etc., SVM in OpenCV from the libSVM library. Among them:

(1), CV ::ml::SVM class: inherited from CV ::ml::StateModel, and CV ::ml::StateModel inherited from CV ::Algorithm;

(2) Create function: static, new SVMImpl to create a SVM object;

(3) setType/getType function: set/obtain SVM formula types, including C_SVC, NU_SVC, ONE_CLASS, EPS_SVR, NU_SVR, used to specify classification, regression, etc., the default is C_SVC;

(4) setGamma/getGamma function: set /get the kernel function γ parameter, the default value is 1;

(5) setCoef0/getCoef0 function: set /getCoef0 parameter of the kernel function, default value is 0;

(6) setDegree/getDegree: set/obtain the degreee parameter of the core function. The default degreee is 0.

(7) setC/getC function: set/obtain THE C parameter of SVM optimization problem, the default value is 0;

(8) setNu/getNu function: set/obtain the υ parameter of SVM optimization problem, the default value is 0;

(9) setP/getP function: set/obtain the ε parameter of SVM optimization problem, the default value is 0;

(10), setClassWeights/getClassWeights function: application in SVM: : C_SVC, set up/get weights, the default value is an empty CV: : Mat;

(11), setTermCriteria/getTermCriteria function: Set/obtain SVM training iteration termination conditions, the default value is CV ::TermCriteria(CV ::TermCriteria::MAX_ITER + TermCriteria::EPS,1000, FLT_EPSILON);

(12) setKernel/getKernelType function: set/obtain the SVM kernel function types, including CUSTOM, LINEAR, POLY, RBF, SIGMOID, CHI2, INTER, the default value is RBF;

(13) setCustomKernel function: initialize the CUSTOM kernel function;

(14) trainAuto function: Train SVM with optimal parameters;

(15), getSupportVectors/getUncompressedSupportVectors functions: for all the support vectors;

(16), getDecisionFunction;

(17), getDefaultGrid/getDefaultGridPtr function: to generate the SVM parameters of grid;

(18) Save /load function: save/load the trained model, supporting XML, YAML, JSON format;

(19) Train /predict function: used for training/prediction, both in the base class StatModel.

The original link: http://bytesizebio.net/2014/02/05/support-vector-machines-explained-well/ formula link: https://en.wikipedia.org/wiki/Support_vector_machine link the SVM parameters: https://docs.opencv.org/master/d1/d2d/classcv_1_1ml_1_1SVM.html

The instance

Svm_type () — Specifies the type of the SVM:

Parameters:

• C_SVC: C represents the penalty factor. A higher C indicates a higher penalty for misclassification.
• NU_SVC: the same as C_SVC.
• ONE_CLASS: no class label is required, used for density estimation and clustering of support vectors.
• EPSILON_SVR: -insensitive loss function. For the sample point, there exists a region that does not provide any loss value for the objective function, i.e. the -band.
• NU_SVR: Because EPSILON_SVR requires parameters to be determined in advance, however, choosing the appropriate parameters is not an easy task in some cases. NU_SVR calculates parameters automatically.

Kernel_type () — KERNEL type of SVM:

Parameters:

• LINEAR kernel:
• POLY: Ploynomial kernel
• RBF: Radical Basis Function
• SIGMOID: Nonlinear action Function kernel of neurons (SIGMOID TANh)
• PRECOMPUTED: User-defined kernel functions

train()

Parameters:

• InputArray samples
• Int layout, layout, parameters: ROW_SAMPLE, each training sample is a row of samples, COL_SAMPLE, each training sample occupies a row of samples
• InputArray Responses, response vectors associated with training samples

getDecisionFunction()

Parameters:

• Int I, index of the decision function. If the problem to be solved is regression, class 1 or class 2 classification, then there will be only one decision function and the index should always be 0. Otherwise, there will be N (n-1) / 2 decision capability in the case of N class classification.
• OutputArray alpha, alpha weights are optional output vectors corresponding to different support vectors. In the case of linear SVM, all alpha is 1.
• OutputArray svidx, an optional output vector of the support vector index within the support vector matrix. In the case of linear SVM, each decision function consists of a single “compressed” support vector.

import cv2
import numpy as np
import matplotlib.pyplot as plt
# 1 to prepare the data
rand1 = np.array([[155.48], [159.50], [164.53], [168.56], [172.60]])
# Height and weight data for girls
rand2 = np.array([[152.53], [156.55], [160.56], [172.64], [176.65]])
# Height and weight data for boys

# 2 Create group labels, 0 for girls, 1 for boys
label = np.array([[0], [0], [0], [0], [0], [1], [1], [1], [1], [1]])

# 3 Merge data
data = np.vstack((rand1,rand2))
data = np.array(data,dtype='float32')

# 4 training
# ml Machine learning module SVM_create() was created
svm = cv2.ml.SVM_create() 

# Attribute Settings
svm.setType(cv2.ml.SVM_C_SVC) # svm type
svm.setKernel(cv2.ml.SVM_LINEAR) # line
svm.setC(0.01)

# training
result = svm.train(data,cv2.ml.ROW_SAMPLE,label)

# prediction
pt_data = np.vstack([[167.55], [162.57]]) #0 girl 1 boy
pt_data = np.array(pt_data,dtype='float32')
print(pt_data)
(par1,par2) = svm.predict(pt_data)
print(par2)
Copy the code

Results:

[[167.55.] [162.57.] [[0.] [1.]Copy the code

2. Hog characteristics

Histogram of Oriented Gradient (HOG) feature is a feature descriptor used for object detection in computer vision and image processing. It constructs features by calculating and counting the gradient direction histogram of the local area of the image. Hog feature combined with SVM classifier has been widely used in image recognition, especially achieved great success in pedestrian detection. It should be reminded that the HOG+SVM method for pedestrian detection was proposed by French researcher Dalal in 2005 CVPR. However, although many pedestrian detection algorithms are constantly proposed, they are basically based on HOG+SVM.

The realization process of HOG feature extraction algorithm:

HOG feature extraction method is to put an image (the target or scan window you want to detect) :

1) Grayscale (the image is regarded as a THREE-DIMENSIONAL image of X, Y and Z (grayscale));

2) The color space of the input image is normalized (normalized) by Gamma correction method; The purpose is to adjust the contrast of the image, reduce the influence caused by the local shadow and illumination changes of the image, and suppress the interference of noise;

3) Calculate the gradient (including size and direction) of each pixel of the image; The main purpose is to capture contour information and further weaken the interference of light.

4) Divide the image into small cells (e.g. 6*6 pixels /cell);

5) Calculate the gradient histogram (number of different gradients) of each cell to form the descriptor of each cell;

6) Each cell forms a block (for example, 3*3 cells /block), and the feature descriptor of all cells in a block is connected in series to obtain the HOG feature descriptor of the block.

7) Concatenate the HOG feature Descriptor of all blocks in the image image to get the HOG feature descriptor of the image (the target you want to detect). So this is the final eigenvector for classification.

The original link: https://blog.csdn.net/liulina603/article/details/8291093

Three, the instance,

HOGDescriptor()

Parameters:

• Size win_size=Size(64, 128),
• Size block_size=Size(16, 16), block Size, currently only Size(16, 16) is supported.
• Size block_stride=Size(8, 8), the slider step of the block. The Size can only be a multiple of cell_size.
• Size cell_size=Size(8, 8), the Size of the cell, currently only Size(8, 8) is supported.
• Int nbins=9, the number of histogram bin (number of ballot boxes), currently each Cell only supports 9.
• Double win_sigma=DEFAULT_WIN_SIGMA, the parameter of gaussian filtering window.
• Double threshold_L2hys=0.2, normalized shrinkage of the normalized type L2-HYs of the intra-block histogram
• Bool GAMMA_correction =true, whether gamma correction is correct
• Nlevels =DEFAULT_NLEVELS, the maximum number of detection Windows

Samples are:

Negative samples:

import cv2
import numpy as np
import matplotlib.pyplot as plt
# Step 1: Determine the criteria
PosNum = 820
# Number of positive samples
NegNum = 1931
# negative sample individual
winSize = (64.128)
The size of a window (WIN) is 64x128
blockSize = (16.16) 
# Each block size is 16x16
blockStride = (8.8) 
The step size of each block is 8x8
# a window has a ((64-16) / 8 + 1) * ((128-16) / 8 + 1) = 7 * 15 = 105 pieces (block)
cellSize = (8.8)
# Each cell is 8x8 in size
A block has (16/8)*(16/8) = 4 cells.
nBin = 9 
# A cell has 9 bin, which means that the vector corresponding to each cell is 9 dimensions
# The dimension of the corresponding one-dimensional feature vector n = 105 * 4 * 9 = 3780
# HOG + SVM pedestrian detection, the final detection method is the most basic linear discriminant function, wx + B = 0, the 3780 dimension vector just obtained is actually W, and the addition of one-dimensional B forms the default 3781 dimension detection operator of OpencV

# Step 2: Create a HOG descriptor and detector
hog = cv2.HOGDescriptor(winSize,blockSize,blockStride,cellSize,nBin)
# Step 3: Start the SVM separator
svm = cv2.ml.SVM_create()

Step 4: Calculate the Hog
featureNum = int(((128- 16) /8+1) * ((64- 16) /8+1) *4*9)
# The dimension of the one-dimensional feature vector corresponding to the window n
featureArray = np.zeros(((PosNum+NegNum),featureNum),np.float32)
Create the Hog feature matrix
labelArray = np.zeros(((PosNum+NegNum),1),np.int32)
Create a tag matrix

# Tag the image sample with positive sample 1 and negative sample -1
for i in range(0,PosNum):
    fileName = 'pos/'+str(i+1) +'.jpg'
    # Import sample image
    img = cv2.imread(fileName)
    hist = hog.compute(img,(8.8))
    The dimension of each HOG feature is 3780
    for j in range(0,featureNum):
        featureArray[i,j] = hist[j]
    # featureArray Load hog feature, [1,:] represents HOG feature 1, [2,:] represents HOG feature 2
    labelArray[i,0] = 1
    The label of the positive sample is 1
    
for i in range(0,NegNum):
    fileName = 'neg/'+str(i+1) +'.jpg'
    img = cv2.imread(fileName)
    hist = hog.compute(img,(8.8))
    for j in range(0,featureNum):
        featureArray[i+PosNum,j] = hist[j]
    labelArray[i+PosNum,0] = - 1
    The label of the negative sample is -1
    
# SVM attribute Settings
svm.setType(cv2.ml.SVM_C_SVC)
# SVM model type: C_SVC represents SVM classifier, and C_SVR represents SVM regression
svm.setKernel(cv2.ml.SVM_LINEAR)
LINEAR: LINEAR kernel, POLY: ploynomial kernel, RBF: radical basis function, SIGMOID: Neural nonlinear function kernel (Sigmoid TANh), PRECOMPUTED: User-defined kernel
svm.setC(0.01)
# SVM type (C_SVC/ EPS_SVR/ NU_SVR) parameter C,C represents the penalty factor, the higher C means the greater the penalty for misclassification

# step 6: Train the function
ret = svm.train(featureArray,cv2.ml.ROW_SAMPLE,labelArray)

Create myHOG with myDeteect
alpha = np.zeros((1),np.float32)
# Optional output vector of weight
rho = svm.getDecisionFunction(0,alpha)
Retrieve the decision function.
print(rho)
print(alpha)

alphaArray = np.zeros((1.1),np.float32)
supportVArray = np.zeros((1,featureNum),np.float32)
resultArray = np.zeros((1,featureNum),np.float32)
alphaArray[0.0] = alpha
resultArray = - 1*alphaArray*supportVArray

myDetect = np.zeros((3781),np.float32)
for i in range(0.3780):
    myDetect[i] = resultArray[0,i]
myDetect[3780] = rho[0]
# myDetect is 3781 dimensions, of which 3780 dimensions are from resultArray and the last dimension is from rHO

myHog = cv2.HOGDescriptor()
myHog.setSVMDetector(myDetect)

# step 8: Import the load of images to be detected
imageSrc = cv2.imread('Test2.jpg'.1)
# Load images to be detected
objs = myHog.detectMultiScale(imageSrc,0, (8.8), (32.32),1.05.2)
# Test images
x = int(objs[0] [0] [0])
y = int(objs[0] [0] [1])
w = int(objs[0] [0] [2])
h = int(objs[0] [0] [3])
# Original coordinates (x,y), width W, height H

# 9: Draw and display
cv2.rectangle(imageSrc,(x,y),(x+w,y+h),(255.0.0),2)
cv2.imshow('dst',imageSrc)
cv2.waitKey(0)
Copy the code

Images to be tested:

Results:

(0.19296025963377925, array([[1.]]), array([[0]], dtype=int32))
[0.]
Copy the code