ML SVM: An introduction to SVM algorithms, applications, and a detailed walkthrough of classic cases
directory
The introduction of SVM algorithm
1. Classification of SVM models — linearly separable, linear and nonlinear
2. Decision boundary of SVM
3. Kernel function in SVM
4. SVM algorithm derivation
1.1. The solution of SVM to dual problem — SMO algorithm thought
5. Flexibility of “loss + punishment” framework in SVM
6. Loss function analysis of SVM
Application of SVM – Type of problem solved
1. SVM is used for classification
1.1. SVM and dichotomies
1.2. SVM and Multiple Classification
2. SVM is used as regression
The classic case of SVM
The introduction of SVM algorithm
Support Vector Machine (SVM), SVM (supervised learning) is a kind of generalized Linear classifier that conducts binary classification on data. The decision boundary is the maximum-margin hyperplane solved for the learning sample. The basic idea of SVM is to find some data on the edge of the set (called Support Vector), and use these points to find a plane (called decision plane), so as to maximize the distance between the Support Vector and the plane.
1. Classification of SVM models — linearly separable, linear and nonlinear
SVM models from simple to complex include:
- When the training sample is linearly separable, a linearly separable support vector machine is learned by hard interval maximization.
- When the training sample is approximately linearly separable, a linear support vector machine is learned by soft interval maximization.
- When the training sample is linearly untime-sharing, a nonlinear support vector machine is learned by kernel technique and soft interval maximization.
| Linearly fractional data |            |
|---|---|
| Linear non-fractional data |   |
| Nonlinear data |  |
2. Decision boundary of SVM
In classification problems, many times there are more than one solution, as shown on the left in the figure below. In the case of ideal linear divisibility, there will be more than one decision plane. The basic model of SVM is to find the optimal separation hyperplane in the feature space to maximize the interval between positive and negative samples on the training set, and the boundary calculated by the SVM algorithm will retain the maximum distance between pairs of categories, that is, there is enough margin, as shown on the right side of the figure below.
3. Kernel function in SVM
When solving the problem of linear inseparability, it can solve the inner product operation in high dimensional space skillfully by introducing kernel function, and thus solve the nonlinear classification problem well. As shown in the figure below, the introduction of kernel function maps the linearly indivisible data into a high-latitude feature space, making the data separable in the feature space. As shown below:
The Gaussian kernel is infinitely dimensional
4. SVM algorithm derivation
1.1. The solution of SVM to dual problem — SMO algorithm thought
5. Flexibility of “loss + punishment” framework in SVM
6. Loss function analysis of SVM
Application of SVM – Type of problem solved
1. SVM is used for classification
1.1. SVM and dichotomies
Binary Classification, solve the problem of output 0 or 1. SVM solves dichotomies.
1.2. SVM and Multiple Classification
T1, Linear Kernal: the strategy is “one versus one”. T2, Linear support vector classifier LinearSVC, adopts the strategy of “One versus All”
2. SVM is used as regression
Support Vector Regression (SVR) can be obtained by extending SVM from classification problem to Regression problem. The standard SVM algorithm is also called Support Vector Classification (SVC).
The method of support vector classification can be extended to solve regression problems, which is called support vector regression. The model generated by support vector classification only relies on a subset of the training data because the cost function of creating the model does not consider training points beyond the boundary. Similarly, models generated by support vector regression rely on only a subset of the training data, because the cost function for creating the model ignores any training data that is close to the model’s prediction. Support vector regression has three different implementations: rbfSVR, polySVR, linearSVR. LinearSVR executes faster than SVR, but only provides linear cores. The fit method takes X and y as input arguments, where y takes a floating-point value rather than an integer value of the classification.
from sklearn import svm
X = [[0.0], [2.2]]
y = [0.5.2.5]
clf = svm.SVR()
clf.fit(X, y)
Copy the code
The classic case of SVM
NN/Average_Darkness/SVM: Handwritten digital image recognition (local dataset 50000 training set + dataset + 4 times) Based on a large number of cases with existing data sets, the SVM/BP: SVR(BETTER) and BP methods for high accuracy prediction of MAT for breast cancer diagnosis are compared and the SVM for prediction of building reinforced concrete compressive strength ML is realized by inputting feature vectors of existing cases. 100 points were randomly generated, a SVM model was established, and the SVM of hyperplane equation ML was found: SVM algorithm based on SkLearn was implemented to annotate the data of support vector