Support vector machine

Support Vector Machine (SVM) is a binary classification model. The goal is to find a standard (called hyperplane) to segment sample data. The principle of segmentation is to ensure the classification optimization (maximum spacing between categories).

When data sets are small, classification using support vector machines is very effective.

Support vector machines are one of the best off-the-shelf classifiers. “off-the-shelf” means that the classifier can be used directly without modification.

In the process of classifying raw data, it may not be possible to achieve segmentation using linear methods. In the classification process, support vector machine maps the data that cannot be linearly segmented to the high-dimensional space, and then finds the optimal linear classifier in the high-dimensional space.

Python support vector machine libraries: SK-learn, LIBSVM, etc

OpenCV also provides support for support vector machines

Theoretical basis

The lines used to divide categories are called classifiers.

When constructing a classifier, it is very important to find the optimal classifier.

Find the support vector machine: In the existing data, find the points closest to the classifier and make sure they are as far away from the classifier as possible.

The distance from the nearest point to the classifier to the classifier is called the margin. You want the interval to be as large as possible so that the classifier can be more accurate in processing the data.

The points closest to the classifier are called support vectors. Determines the location of the classifier.

Make indivisible into divisible

Support vector machines make data that is not so easily classified classifiable by function mapping.

In the process of data processing, if the classification cannot be completed in the low-dimensional space, the support vector machine will automatically map the data to the high-dimensional space and make it (linearly) separable. Simply put, it is a function mapping operation to the current data.

For example, in the classification process, the data in the left figure that cannot be classified by linear classifier can be changed into linearly separable data in the right figure through the mapping of function F.

At the same time, support vector machines can effectively reduce computational complexity through kernel functions.

Support vector machines can process virtually any dimension of data. In different dimensions, support vector machines will try to find linear classifiers similar to lines in two-dimensional space.

For example, in two dimensions, a support vector looks for a line that divides the current data. In three-dimensional space, the support vector machine looks for a plane that can divide the current data. In higher dimensions, the support vector machine tries to find a hyperplane that divides the current data.

In general, the data that can be divided by a straight line (more generally, a hyperplane) is called linearly separable data, so the hyperplane is a linear classifier.

“Support vector machine” is composed of “support vector” and “machine”.

  • The “support vectors” are the points closest to the classifier on the maximum “interval”. In general, classification depends only on these points and has nothing to do with other points.
  • “Machine” means classifier.

Support vector machine (SVM) is a classification algorithm based on key points.

Introduction to SVM

When using support vector machine modules, the function cv2.ml.svm_create () should be used first to generate the empty classifier model for subsequent training.

Syntax format:

svm = cv2.ml.SVM_create()
Copy the code

After acquiring the empty classifier SVM, the svm.train() function is used to train the training data for the model

Syntax format

Training results = SVm. train(training data, training data arrangement format, training data label)Copy the code
  • Training data: represents raw data used to train the classifier.
  • Training data arrangement format: The original data can be arranged in two forms: row arrangement (Cv2.ml. ROW_SAMPLE, where each training data occupies one row) and column arrangement (Cv2.ml. COL_SAMPLE, where each training data occupies one column)
  • Labels for training data: Labels for raw data.
  • Training result: The return value of the training result.

For example, the data used for training is data, and its corresponding label is label. Each data is arranged in line. The statement used for training classifier model SVM is as follows:

Return value = svm.train(data, cv2.ml.row_SAMPLE, label)Copy the code

After the classifier training is completed, svm.predict() function can be used to classify test data using the trained classifier model, and its syntax format is as follows:

Result = SVM. Predict (test data)Copy the code

OpenCV supports customization of multiple parameters. For example, setType() can be used to set categories, setKernel() can be used to set core types, and setC() can be used to set the support vector machine parameter C(penalty coefficient, that is, tolerance for errors, the default value is 0).

Example is introduced

Title: Given the written test scores, interview scores and corresponding grade performance of the employee, predict the possible performance of the new employee according to the written test scores and interview scores.

Firstly, a set of random numbers are constructed and divided into two classes. Then, OpenCV’s support vector machine modules are used to complete training and classification. Finally, the calculation results are displayed.

Specific steps:

  1. Generate simulated data

    Generate the written test and interview scores of the employees with grade A performance one year after entry.

    Construct 20 groups of data pairs with written and interview scores distributed within [95, 100] :

    a = np.random.randint(95.100, (20.2)).astype(np.float32) 
Copy the code

    One year later, the corresponding work performance will be grade A.

    The written test and interview scores of employees with grade B performance one year after entry were simulated.

    Construct 20 groups of data pairs with written and interview scores distributed within [90, 95] :

    b = np.random.randint(90.95, (20.2)).astype(np.float32) 
Copy the code

    One year later, the corresponding job performance will be B grade.

    Finally, merge the two sets of data and convert them using numpy.array:

    data = np.vstack((a, b)) 
data = np.array(data, dtype='float32')     
Copy the code

  2. Constructing grouping labels

    To correspond to the a-level data distributed in the interval [95, 100], construct the label “0” :

    aLabel=np.zeros((20.1)) 
Copy the code

    To correspond to the b-level data distributed in the interval [90, 95], construct label “1” :

    bLabel=np.ones((20.1)) 
Copy the code

    Combine the above tags and convert them using numpy.array:

    label = np.vstack((aLabel, bLabel)) 
label = np.array(label, dtype='int32') 
Copy the code

  3. training

    Support vector machine modules are used to train the known data and its corresponding labels:

    svm = cv2.ml.SVM_create() 
result = svm.train(data, cv2.ml.ROW_SAMPLE, label) 
Copy the code

  4. classification

    Generate two random data pairs (written test scores, interview scores) for the test.

    test = np.vstack([[98.90], [90.99]]) 
test = np.array(test, dtype='float32') 
Copy the code

    SVM. Predict () was used to classify random scores:

    (p1, p2) = svm.predict(test) 
Copy the code

  5. Display classification results

    Display the basic data (training data), the data used for testing (test data) on the image:

    plt.scatter(a[:,0], a[:,1].80.'g'.'o') 
plt.scatter(b[:,0], b[:,1].80.'b'.'s') 
plt.scatter(test[:,0], test[:,1].80.'r'.The '*') 
plt.show() 
Copy the code

    Display test data and predicted classification results:

    print(test) 
print(p2)
Copy the code

Complete the program

import cv2 
import numpy as np 
importMatplotlib.pyplot as PLT # prepare data a = np.random.randint(95.100, (20.2)).astype(np.float32) 
b = np.random.randint(90.95, (20.2)).astype(np.float32) 
data = np.vstack((a, b)) 
data = np.array(data, dtype='float32'# create group tag,0Stands for GRADE A,1ALabel =np.zeros((20.1)) 
bLabel=np.ones((20.1)) 
label = np.vstack((aLabel, bLabel)) 
label = np.array(label, dtype='int32'SVM_create() = cv2.ml.svm_create (#svm.setType(cv2.ml.SVM_C_SVC) # svm type
#svm.setKernel(cv2.ml.SVM_LINEAR) # line 
# SVM. The setC (0.01)Result = svm.train(data, cv2.ml.ROW_SAMPLE, label) # forecast test = np.vstack([[98.90], [90.99]]) 
test = np.array(test, dtype='float32') 
(p1, p2) = svm.predict(test)   # test is a [[data 1],[data 2]] structurePrint (test) print("res1",p2[0])
print("res2",p2[1]) 
plt.scatter(a[:,0], a[:,1].80.'g'.'o') 
plt.scatter(b[:,0], b[:,1].80.'b'.'s') 
plt.scatter(test[:,0], test[:,1].80.'r'.The '*') 
plt.show() 
Copy the code