This article has participated in the activity of "New person creation Ceremony", and started the road of digging gold creation together

In our previous blog, we shared the concept of Regularization **, which preserves all the features but decreases the Magnitude of the parameter.

This time, I’ll talk to you about normalized linear regression models and normalized logistic regression models.

First of all, just to be clear, why do some machine learning models need to use normalization? There are two answers:

  • Normalization accelerates the pace of gradient descent, that is, the speed at which the optimal solution is obtained
  • Normalization can submit the accuracy of the model

We can discuss the specific analysis later, so we don’t need to go over it here.

Normalized linear regression model(Regularized Linear Regression)

We talked about the normalization cost function before. The cost function of linear regression is the same as the expression of normalized cost function:

​

If we use the gradient descent algorithm to minimize this cost function, the gradient descent algorithm we get will be in the following form :(we have not normalized ø0)

​

For the above algorithm, j=1,2,3… ,n, the updated expression can be adjusted to obtain:

​

It can be seen that the gradient descent algorithm of normalized linear regression differs from the previous one in that each time the value of ø is reduced by an additional value based on the updating rules of the original algorithm.

Similarly, if the ** Normal Equation ** is used to solve the normalized linear regression model, the expression is as follows:

​

In this expression, the size of the matrix is n+1*n+1

Normalized logistic regression model(Regularized Logistic Regression)

Similarly, for the logistic regression model, we also add a normalized expression to the cost function and get the following expression:

​

To obtain the minimum value of this cost function, the gradient descent algorithm expression is obtained by taking the derivative as follows:

​

Note: it just looks the same as linear regression, but it’s an assumed function, so it is different from linear regression.

That is the content of the normalization of the two regression models. So far, we have learned the regression model in machine learning related content is basically involved. An important algorithm in machine learning, Neural Network, will be discussed below.