Support Vector Machine (SVM) is a powerful machine learning algorithm. Compared with neural network, it can provide a clearer and more powerful way to learn complex nonlinear equations. To learn about support vector machines, we can start with logistic regression and see how we can get support vector machines with small changes.
In logistic regression, the function h_θ(x) is assumed to be:
Image as follows:
For a sample, the cost function is:
When y =1, the second term is 0, so we just have to think about the first term and graph the first term:
We modify this image slightly, the trend is similar to logistic regression, and it becomes the image cost_1(z) as part of the cost function of support vector machine, as shown in the pink line below:
Same for the second term, cost_0(z) :
Regularization cost function in logistic regression is:
Remove the coefficient 1/m and replace the red part with the function shown in pink above:
Then switch the weight of the light blue and pink parts:
Thus, the cost function of the support vector machine can be obtained:
You just need to minimize this function to get the argument. This is the whole process of deriving support vector machines from logistic regression.
Ps. This article is based on the study notes of Ng’s machine learning course. If you want to learn machine learning together, you can follow the wechat public account “SuperFeng”, looking forward to meeting you.
