“This is the 10th day of my participation in the Gwen Challenge in November. Check out the details: [Last Gwen Challenge 2021]

Bayes method — Background

Bayesian classification: Bayesian classification is the general name of a class of classification algorithms, which are based on Bayesian theorem, so they are collectively called Bayesian classification.

Prior probability: probability based on previous experience and analysis. We use 𝑃(𝑌) to represent the initial probability assumed that 𝑌 has before training data.

A posteriori probability: the probability derived from an analysis of events that have occurred. With 𝑃 (𝑌 | 𝑋) represents the hypothesis 𝑋 set up under the affection of the probability of observed 𝑌 data, because it reflects the 𝑌 after see 𝑋 training data set up confidence.

Joint probability: Joint probability refers to the probability that multiple random variables meet their respective conditions in a multivariate probability distribution. The joint probability of 𝑋 and 𝑌 is expressed as 𝑃𝑋, 𝑌, 𝑌 (𝑌) or 𝑋 (∩ password). Joint probability: assuming that 𝑋 and 𝑌 both follow normal distribution, 𝑃(𝑋 < 5, 𝑌 < 0) is a joint probability, representing the probability that both conditions, Such as The probability of Both Conditions, are true. Represents the probability of two events occurring together. Bayes’ formula is as follows:

Naive Bayes method is a typical generative learning method. Method generated by the training data to study the joint probability distribution 𝑃 (𝑋, 𝑌), and then get a posteriori probability distribution 𝑃 (𝑌 | 𝑋). In particular, the use of training data to study 𝑃 (𝑋 | 𝑌) and 𝑃 (𝑌) estimates, get the joint probability distribution: 𝑃 (𝑋 𝑌) = 𝑃 (𝑋 | 𝑌) 𝑃 (𝑌)

Bayes’ principle

Naive Bayes method is a typical generative learning method. Method generated by the training data to study the joint probability distribution 𝑃 (𝑋, 𝑌), then get a posteriori probability points cloth 𝑃 (𝑌 | 𝑋). In particular, the use of training data to study 𝑃 (𝑋 | 𝑌) and 𝑃 (𝑌) estimates, get the joint probability distribution: 𝑃 (𝑋 𝑌) = 𝑃 (𝑌) 𝑃 (𝑋 | 𝑌) probability estimation method can be a maximum likelihood estimation and bayesian estimation.

The basic method of naive Bayes is conditional independence.

Ck represents the category and K represents the number of categories. This is a strong assumption. Because of this assumption, the number of conditional probabilities included in the model is greatly reduced, and the learning and prediction of naive Bayes method are greatly simplified. So naive Bayes method is efficient and easy to implement. The disadvantage is that the performance of classification is not necessarily very high.

Naive Bayes method uses Bayes theorem and the learned joint probability model for classification prediction.

Code implementation

We use bayesian method to achieve text classification

Let’s figure out what category 1XM belongs to

#coding:utf-8 # maximum likelihood estimation NaiveBayes algorithm import pandas as pd import numpy as np class NaiveBayes(object): def getTrainSet(self): DataSet = pd.read_csv('naivebayes_data.csv') dataSetNP = np.array(dataSet Shape [:,0: datasetnp. shape[1] # training data x1, X2 labels = dataSetNP[:, datasetnp. shape[1]-1] # Training data corresponding to the type Y return trainData, labels def classify(self, trainData, labels, features): P_y = {} P_y = {} P_y = {} P_y = {} P_y[label] = count(label)/float(len(labels)) # p = count(y)/count(y) # P_xy = {} for y in P_y.keys(): Y_index = [I for I, label in enumerate(labels) if label == y] # labels subindex of all values in y for j in range(len(features)): X_index = [I for I, x_index = [I for I, x_index = [I for I, x_index = [I for I, x_index = [I for I] feature in enumerate(trainData[:,j]) if feature == features[j]] xy_count = len(set(x_index) & set(y_index)) # Pkey = STR (features[j]) + '*' + STR (y) P_xy[pkey] = xy_count/float(len(labels)) P = {} for y in p_y. keys(): for x in features: pkey = str(x) + '|' + str(y) P[pkey] = P_xy[str(x)+'*'+str(y)] / float(P_y[y]) #P[X1/Y] = P[X1Y]/P[Y] print(P_y[y]) Print (pkey) # print(pkey) # print(pkey) # print(pkey) # print(pkey) # F [y] [y] * P = F [STR (x) + '|' + STR (y)] # P [y/x) = P (x/y) * P (y)/P (x), the denominator is equal, the more molecules can, So have F = P/X/y * P = P [y] x1 / y * P x2 / y * P [y] features_label = Max (F, Return features_label if __name__ == '__main__': Nb = NaiveBayes() # Labels = nb.getTrainSet() # x1,x2 features = [1,'X','M'] # result = Nb.classify (trainData, labels, Print (features,' belong to ',result)Copy the code