“This is the 23rd day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021”

preface

Support Vector Machine (SVM) is a supervised learning technique, which constructs a hyperplane or a group of hyperplanes in a high-dimensional space through optimal separation of training data according to a specified class. In OpenCV, SVM algorithms can be implemented and trained. In this paper, we will learn how to use SVM classifier to perform handwritten number recognition, and also explore the effects of different parameters on model performance to obtain the best performance of SVM classifier.

Handwritten number recognition using SVM

We have already introduced the MNIST handwritten number dataset and how to recognize handwritten numbers using KNN algorithm in “Handwriting Number Recognition with KNN Algorithm”. The optimal classification accuracy is obtained by preprocessing digital images (desew() function) and using high-level descriptors (HOG descriptors) as feature vectors used to describe each number. Therefore, there is no need to repeat the same content. Next, preprocessing and HOG features will be directly introduced in recognition of Handwritten Numbers using KNN Algorithm, and SVM algorithm will be used to classify digital images.

First load the data and divide it into training sets and test sets:

# load data
(train_dataset, train_labels), (test_dataset, test_labels) = keras.datasets.mnist.load_data()
SIZE_IMAGE = train_dataset.shape[1]
train_labels = np.array(train_labels, dtype=np.int32)
# preprocessor function
def deskew(img) :
    m = cv2.moments(img)
    if abs(m['mu02']) < 1e-2:
        return img.copy()
    skew = m['mu11'] / m['mu02']
    M = np.float32([[1, skew, -0.5 * SIZE_IMAGE * skew], [0.1.0]])
    img = cv2.warpAffine(img, M, (SIZE_IMAGE, SIZE_IMAGE), flags=cv2.WARP_INVERSE_MAP | cv2.INTER_LINEAR)

    return img
# HOG high-level descriptor
def get_hog() :
    hog = cv2.HOGDescriptor((SIZE_IMAGE, SIZE_IMAGE), (8.8), (4.4), (8.8), 9.1, -1.0.0.2.1.64.True)

    print("hog descriptor size: {}".format(hog.getDescriptorSize()))

    return hog
# Data shredding
shuffle = np.random.permutation(len(train_dataset))
train_dataset, train_labels = train_dataset[shuffle], train_labels[shuffle]

hog = get_hog()

hog_descriptors = []
for img in train_dataset:
    hog_descriptors.append(hog.compute(deskew(img)))
hog_descriptors = np.squeeze(hog_descriptors)

results = defaultdict(list)
# Data partitioning
split_values = np.arange(0.1.1.0.1)
Copy the code

Next, initialize SVM and train:

# model initialization function
def svm_init(C=12.5, gamma=0.50625) :
    model = cv2.ml.SVM_create()
    model.setGamma(gamma)
    model.setC(C)
    model.setKernel(cv2.ml.SVM_RBF)
    model.setType(cv2.ml.SVM_C_SVC)
    model.setTermCriteria((cv2.TERM_CRITERIA_MAX_ITER, 100.1e-6))

    return model
# Model training function
def svm_train(model, samples, responses) :
    model.train(samples, cv2.ml.ROW_SAMPLE, responses)
    return model
# Model prediction function
def svm_predict(model, samples) :
    return model.predict(samples)[1].ravel()
# Model evaluation function
def svm_evaluate(model, samples, labels) :
    predictions = svm_predict(model, samples)
    acc = (labels == predictions).mean()
    print('Percentage Accuracy: %.2f %%' % (acc * 100))
    return acc *100
# Use different training sets and test set partitioning methods for training and testing
for split_value in split_values:
    partition = int(split_value * len(hog_descriptors))
    hog_descriptors_train, hog_descriptors_test = np.split(hog_descriptors, [partition])
    labels_train, labels_test = np.split(train_labels, [partition])

    print('Training SVM model ... ')
    model = svm_init(C=12.5, gamma=0.50625)
    svm_train(model, hog_descriptors_train, labels_train)

    print('Evaluating model ... ')
    acc = svm_evaluate(model, hog_descriptors_test, labels_test)
    results['svm'].append(acc)
Copy the code

As shown in the figure above, the accuracy of the SVM model with default parameters can reach 98.60% when using 70% digital image training algorithm. Next, we modified the PARAMETERS C and γ of the SVM model to test whether the model has room for improvement.

Effects of parameters C and γ on the accuracy of handwritten digit recognition

When RBF kernel is used in SVM model, there are two important parameters — C and γ. In the above example, C=12.5 and γ=0.50625 are used as parameter values. The setting of C and γ depends on specific data sets. Therefore, some method must be used to search for parameters, in this case using a grid to search for suitable parameters C and γ.

for C in [1.10.100.1000] :for gamma in [0.1.0.15.0.25.0.3.0.35.0.45.0.5.0.65] : model = svm_init(C, gamma) svm_train(model, hog_descriptors_train, labels_train) acc = svm_evaluate(model, hog_descriptors_test, labels_test)print("{}".format("%.2f" % acc))
        results[C].append(acc)
Copy the code

Finally, visualization results:

fig = plt.figure(figsize=(10.6))
plt.suptitle("SVM handwritten digits recognition", fontsize=14, fontweight='bold')
ax = plt.subplot(1.1.1)
ax.set_xlim(0.0.65)
dim = [0.1.0.15.0.25.0.3.0.35.0.45.0.5.0.65]

for key in results:
    ax.plot(dim, results[key], linestyle=The '-', marker='o', label=str(key))

plt.legend(loc='upper left', title="C")
plt.title('Accuracy of the SVM model varying both C and gamma')
plt.xlabel("gamma")
plt.ylabel("accuracy")
plt.show()
Copy the code

The running result of the program is as follows:

As shown in the figure, by using different parameters, the accuracy can reach about 99.25%. By comparing the performance of KNN classifier and SVM classifier in handwritten digit recognition task, we can conclude that SVM is superior to KNN classifier in handwritten digit recognition task.

A link to the

KNN algorithm is used to recognize handwritten digits