SVM – Code case 4 – Comparison of different effects of DIFFERENT SVM penalty parameter C values

Mr Bai moss

  • test
  • kernel

Abstract: The chapter of SVM has been finished, the specific content please refer to: “01 SVM-outline” “14 SVM-Code Case I – SVM classification of Iris data” “15 SVM-Code Case II – Effect comparison of different classifiers of Iris data” “16 SVM-Code Case III – Effect comparison of different SVM kernel functions” routine operations: 1, header file to introduce svM-related package 2,…

The chapter of SVM has been finished. For details, please refer to 01 SVM – Outline.

“14 SVM-Code Case I – SVM classification of Iris data” “15 SVM-Code Case II – Effect comparison of different classifiers of Iris data” “16 SVM-Code Case III – Effect comparison of different SVM kernel Functions”

General operation:

1. Introduction of SVM-related packages into header files 2. Prevention of Chinese garble 3

import time import numpy as np import pandas as pd import matplotlib as mpl import matplotlib.pyplot as plt from sklearn.svm import SVC from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score RcParams ['font. Sans-serif '] = [u'SimHei'] mpl.rcParams['axes. Unicode_minus '] = False ## Read data # 'sepal , 'petal length', 'petal length', 'petal length', 'petal width', 'petal length', Data = pd.read_csv(path, header=None) x, y = data[list(4)], Data [4] y = pd. Categorical (y) codes x = x [[0, 1]] # # x_train data segmentation, x_test, y_train, y_test = train_test_split (x, y, Random_state = 28, train_size = 0.6)Copy the code

Data SVM classifier construction:

Svm1 = SVC(C=0.1, kernel=' RBF ') SVM2 = SVC(C=1, kernel=' RBF ') SVM3 = SVC(C=10, kernel=' RBF ') svM4 = SVC(C=100, kernel=' RBF ') kernel='rbf') svm5 = SVC(C=500, kernel='rbf') svm6 = SVC(C=100000, kernel='rbf')Copy the code

The larger C is, the worse the generalization ability is, and the problem of over-fitting will occur. The smaller C is, the better the generalization ability is, but the problem of under-fitting is easy to occur

Model training:
t0=time.time()
svm1.fit(x_train, y_train)
t1=time.time()
svm2.fit(x_train, y_train)
t2=time.time()
svm3.fit(x_train, y_train)
t3=time.time()
svm4.fit(x_train, y_train)
t4=time.time()
svm5.fit(x_train, y_train)
t5=time.time()
svm6.fit(x_train, y_train)
t6=time.time()Copy the code

Effect evaluation:

svm1_score1 = accuracy_score(y_train, svm1.predict(x_train))
svm1_score2 = accuracy_score(y_test, svm1.predict(x_test))

svm2_score1 = accuracy_score(y_train, svm2.predict(x_train))
svm2_score2 = accuracy_score(y_test, svm2.predict(x_test))

svm3_score1 = accuracy_score(y_train, svm3.predict(x_train))
svm3_score2 = accuracy_score(y_test, svm3.predict(x_test))

svm4_score1 = accuracy_score(y_train, svm4.predict(x_train))
svm4_score2 = accuracy_score(y_test, svm4.predict(x_test))

svm5_score1 = accuracy_score(y_train, svm5.predict(x_train))
svm5_score2 = accuracy_score(y_test, svm5.predict(x_test))

svm6_score1 = accuracy_score(y_train, svm6.predict(x_train))
svm6_score2 = accuracy_score(y_test, svm6.predict(x_test))Copy the code

Figure-iris data COMPARISON of different effects of SVM classifier C value:

X_tmp = [0,1,2,3, 4, 5] t_score = [t1-t0, t2-t1, t3-t2, t4-t3, t5-t4, t6-t5] y_score1 = [svm1_score1, svm2_score1, svm3_score1, svm4_score1, svm5_score1, svm6_score1] y_score2 = [svm1_score2, svm2_score2, svm3_score2, svm4_score2, Svm5_score2 svm6_score2] PLT. Figure (facecolor = 'w', figsize = (12, 6))Copy the code
1. Prediction accuracy of the model:
Plt.plot (x_tmp, y_score1, 'r-', lw=2, label=u' Training set accuracy ') plt.plot(x_tmp, y_score2, 'g-', lw=2, Label = u 'test set accuracy) PLT. Xlim (0.3, 3.3) PLT. Ylim (np) min ((np) min (y_score1), np. Min (y_score2))) * 0.9, Max ((np.max(y_score1), np.max(y_score2)))*1.1) plt.legend(loc = 'lower left') plt.title(u' Fontsize = 13) PLT. Xticks (x_tmp, [u 'C = 0.1, u' C = 1, u 'C = 10, u' C = 100 'u' C = 500 'u' C = 10000], rotation = 0) PLT, the grid (b = True)Copy the code
2. Model training time:
Plt.plot (x_tmp, t_score, 'b-', lw=2, label=u' model training time ') plt.title(u' model training time ', Fontsize = 13) PLT. Xticks (x_tmp, [u 'C = 0.1, u' C = 1, u 'C = 10, u' C = 100 'u' C = 500 'u' C = 10000], rotation = 0) PLT, the grid (b = True)Copy the code
Plt.suptitle (comparison of different effects of C value of SVM classifier for U 'Iris data ', fontsize=16) plt.show()Copy the code

Drawing of prediction results:

Comparison of drawing – Comparison of different C parameters of SVM classifier for Iris data
N = 500 x1_min, x2_min = x.min() x1_max, x2_max = x.max() t1 = np.linspace(x1_min, x1_max, N) t2 = np.linspace(x2_min, Meshgrid (T1, T2) # Generate grid_show = Np.dstack ((x1.flat, x2.flat))[0] # test pointCopy the code
Get the test values of different algorithms:
Svm1_grid_hat = svm1.predict(grid_show) Svm1_grid_hat = svm1_grid_hat. shape(0) # Make it the same shape as the input svm2_grid_hat = Svm2.predict (grid_show) Svm2_grid_hat = svm2_grid_hat. shape(0) # Make it the same shape as the input svm3_grid_hat = Svm3.predict (grid_show) Svm3_grid_hat = svm3_grid_hat. shape(0) # Make it the same shape as the input svm4_grid_hat = Svm4. predict(grid_show) SvM4_grid_hat = svm4_grid_hat. shape(0) # Make it the same shape as the input svM5_grid_hat = Svm5. Predict (grid_show) SvM5_grid_hat = svM5_grid_hat. 0 (x1.shape) # Make it the same shape as the input svM6_grid_hat = Svm6. predict(grid_show) Svm6_grid_hat = svm6_grid_hat. 0 (x1.shape) # Make it the same shape as the inputCopy the code

Drawing:

cm_light = mpl.colors.ListedColormap(['#A0FFA0', '#FFA0A0', '#A0A0FF']) cm_dark = mpl.colors.ListedColormap(['g', 'r', 'b']) PLT. Figure (facecolor = 'w', figsize = (14, 7))Copy the code
1, C = 0.1
Pcolormesh (x1, x2, svm1_grid_hat, cmap=cm_light) # X_test [0], x_test[1], s=120, facecolors='none', Xlabel (iris_feature[0], fontsize=13) plt.ylabel(iris_feature[1], Fontsize =13) plt.xlim(x1_min, x1_max) plt.ylim(x2_min, x2_max) plt.title(u'C=0.1', fontsize=15) plt.grid(b=True, Ls = ':') PLT. Tight_layout (pad = 1.5)Copy the code
2, C = 1
Pcolormesh (x1, x2, svm2_grid_hat, cmap=cm_light) X_test [0], x_test[1], s=120, facecolors='none', Xlabel (iris_feature[0], fontsize=13) plt.ylabel(iris_feature[1], fontsize=13) plt.xlim(x1_min, x1_max) plt.ylim(x2_min, x2_max) plt.title(u'C=1', fontsize=15) plt.grid(b=True, Ls = ':') PLT. Tight_layout (pad = 1.5)Copy the code
3, C = 10
Pcolormesh (x1, x2, svm3_grid_hat, cmap=cm_light) X_test [0], x_test[1], s=120, facecolors='none', Xlabel (iris_feature[0], fontsize=13) plt.ylabel(iris_feature[1], fontsize=13) plt.xlim(x1_min, x1_max) plt.ylim(x2_min, x2_max) plt.title(u'C=10', fontsize=15) plt.grid(b=True, Ls = ':') PLT. Tight_layout (pad = 1.5)Copy the code
4, C = 100
(x1, x2, svm4_grid_hat, cmap=cm_light) # X_test [0], x_test[1], s=120, facecolors='none', Xlabel (iris_feature[0], fontsize=13) plt.ylabel(iris_feature[1], fontsize=13) plt.xlim(x1_min, x1_max) plt.ylim(x2_min, x2_max) plt.title(u'C=100', fontsize=15) plt.grid(b=True, Ls = ':') PLT. Tight_layout (pad = 1.5)Copy the code
5, C = 500
Pcolormesh (x1, x2, svm5_grid_hat, cmap=cm_light) X_test [0], x_test[1], s=120, facecolors='none', Xlabel (iris_feature[0], fontsize=13) plt.ylabel(iris_feature[1], fontsize=13) plt.xlim(x1_min, x1_max) plt.ylim(x2_min, x2_max) plt.title(u'C=500', fontsize=15) plt.grid(b=True, Ls = ':') PLT. Tight_layout (pad = 1.5)Copy the code
6, C = 10000
Pcolormesh (x1, x2, svm6_grid_hat, cmap=cm_light) X_test [0], x_test[1], s=120, facecolors='none', Xlabel (iris_feature[0], fontsize=13) plt.ylabel(iris_feature[1], fontsize=13) plt.xlim(x1_min, x1_max) plt.ylim(x2_min, x2_max) plt.title(u'C=10000', fontsize=15) plt.grid(b=True, Ls = ':') PLT. Tight_layout (pad = 1.5)Copy the code
Plt.suptitle (comparison of effects of different C parameters of SVM classifier for U 'Iris data ', fontsize=16) plt.show()Copy the code

Conclusion: The larger C is, the worse the generalization ability is, and the problem of over-fitting will occur. The smaller C is, the better the generalization ability is, but the problem of under-fitting is easy to occur

For details, please click
Comments (0)

To:


Related articles

  • On December 7, the cloud selected night | ali database extreme flexibility
  • Different kernel functions of SVM…
  • Summary of Gaussian kernel tuning for support vector machines
  • Python implementation SVM
  • Svm-soft interval model
  • Scikit-learn support vector machine algorithm library uses summary
  • Scikit-learn SVM algorithm for learning
  • Support vector machines (SVM) support vector machines (SVM)
  • 14 SVM – Code case one – SVM classification of iris data
  • Text classification and SVM

The net friend comment on