What is underfitting?

Generally speaking, if the fitting is not accurate enough and the effect is not good in the training set, the effect will certainly not be good in the test set. An important topic in machine learning is the generalization ability of models, and a model with strong generalization ability is a good model. For the well-trained model, if it performs poorly in the training set, it will also perform poorly in the test set, which may be caused by under-fitting, that is, the model has not been well learned and has not captured data features well. Underfitting means that the model fitting degree is not high and the actual data is far from the fitting curve, or the model does not capture the data characteristics well and cannot fit the data well. The basic problem of machine learning is to use models to fit data. The purpose of learning is not to correctly predict the limited training set, but to accurately predict the samples that have not appeared in the training set, and to accurately predict the new data. The error of the model to the data of the training set is called the empirical error and the error to the data of the test set is called the generalization error. The prediction ability of a model to samples outside the training set is called the generalization ability of the model, and the pursuit of this generalization ability is always the goal of machine learning. Underfitting and overfitting are two common causes of low model generalization ability, both of which are the result of mismatch between model learning ability and data complexity.Copy the code

What causes the underfitting?

Owed "fitting" often in the model learning ability is weak, and the data of high complexity Model at this time due to lack of ability to learn can't learn the data set "general rule", not very good capture data characteristics "Fitting", in contrast, often in the condition of strong ability to learn a model, the model of learning ability is too strong, So that the characteristics of a single sample in the training set can be captured and regarded as a "general rule", which will also lead to the decline of model generalization ability. The difference between overfitting and underfitting lies in that underfitting has poor performance in both training set and test set, while overfitting can learn the properties of data in training set better, and has poor performance and generalization ability in test set. In the process of neural network training, underfitting is mainly manifested as high deviation of output results, while overfitting is mainly manifested as high variance of output resultsCopy the code

The solution?

Common solutions include: add new features (continue to extract some features with distinguishing degree), consider adding feature combination and higher-order features to increase the hypothesis space; Add polynomial features, which are commonly used in machine learning algorithms, such as linear models by adding quadratic or cubic terms to make the model generalization ability stronger; Reduce regularization parameters, the purpose of regularization is to prevent overfitting, but the model underfitting, need to reduce regularization parameters; Use nonlinear models, such as kernel SVM, decision tree, deep learning and other models; Adjust the capacity of the model. Colloquially, the capacity of the model refers to its ability to fit various functions. Low capacity model may be difficult to fit the training set; Using integrated learning methods such as Bagging, Bagging multiple weak learners.Copy the code