A sequence of

Continue the previous greedy NLP course. The previous section 60 starts with the introduction of Python’s high and powerful features. Try to make up for it during the Spring Festival holiday.

Evaluation of the model

After the modeling of machine learning, data mining and recommendation system is completed, the effect of the model needs to be evaluated

This paper deals with binary classifiers.

Accuracy Accuracy

 

acc:  

Accuracy is the correct number/total

When our sample is imbalanced (the proportion of positive and negative samples is very different), accuracy is not suitable for evaluating the model.

The teacher gave an example, lung cancer suppose 1000 people come to check, suppose 5 are diagnosed, 995 are healthy.

So even if I don’t do any processing, for the new data, directly based on the probability of calculating healthy, accuracy is 99.5%. This is highly accurate, but not of any value.

In order to make up for the lack of accuracy, study recall rate.

So here, let’s say 10 of the 1,000 people get cancer, 10 of them are recalled, and 8 of them actually get cancer.

P(Positive) and N(Negative) indicate the judgment result of the model. T(True) and F(False) indicate whether the judgment result of the model is correct

FP: false positive example FN: false negative example TP: true negative example TN: true negative example.

Accuracy Precision =

The Recall rate (Recall) = 

Recall rate refers to all positive cases in the data set, while precision rate refers to all positive cases judged by the model.

Recall rate and accuracy rate are mutually exclusive, and a balance point needs to be found. In general, high accuracy and low recall rate mean low recall rate and high accuracy.

P accuracy rate and R recall rate are sometimes contradictory, so they need to be considered comprehensively. The most common method is F-measure (also known as F-Score).

F-measure is Precision and Recall weighted harmonic average:

When the parameterWhen =1, it is the most common F1-measure:

 

The situation is similar for multiple classifications, where each separation is averaged.

There are other indicators such as sensitivity, ROC and PR curves. The teacher did not speak.

The last example, take spam as an example, calculate the accuracy rate, recall rate