Author: CHEONG

AI Machine Learning and Knowledge Graph

Research interests: Natural language processing and knowledge mapping

The article contains a large number of formulas, if you need to obtain the original Word document containing the formula, you can follow the public account [AI Machine Learning and Knowledge Graph] and reply: GMM first lecture, you can add wechat [17865190919] into the learning exchange group, add friends when the remarks from nuggets. Original is not easy, reprint please inform and indicate the source!

In this paper, gaussian mixture model is briefly introduced from two perspectives. The next chapter will introduce the Learning problem solution of Gaussian mixture model in detail.

First, geometric Angle

Gaussian mixture model can be regarded as the superposition of multiple Gaussian distributions, that is, the weighted value of multiple Gaussian distributions, which can be expressed by the formula:

Where N(UK, σ K)N(U_K, \Sigma_k)N(UK, σ K) represents the KTH Gaussian distribution, and αk\ alpha_K αk represents the weight of the KTH Gaussian distribution in the mixed Gaussian model. Look at the figure below, each colored line represents a Gaussian distribution model, and multiple colored Gaussian models are superposed to form the black line, i.e. the mixed Gaussian model. We can clearly match the figure below with the gaussian mixture model above.

The figure above shows the Gaussian mixture model formed by the superposition of one-dimensional Gaussian distribution, and the figure below shows a Gaussian mixture model formed by the superposition of two two-dimensional Gaussian distributions:

2. Angle of mixed model

For the mixed model, it is necessary to introduce the hidden variable Z on the basis of the observed variable X. The meaning of the hidden variable Z represents the probability of which sample in the mixed Gaussian distribution belongs to the Gaussian distribution. If we assume that Z is a discrete random variable, and if the mixed Gaussian distribution consists of K Gaussian distributions, then:

The table shows a gaussian mixture model composed of C1, C2… ,ckc_1,c_2,… ,c_kc1,c2,… And the z implicit variable means that the probability of a sample belonging to the ith Gaussian distribution is PIp_IPi, and there are:

We can understand the implicit variable Z again from the perspective of sample generation: suppose we now have a die with uneven k-sides and use this die to generate sample data. Each face of this die represents a Gaussian distribution. Since the k side of the die is not even, the probability of each side being rolled is P1, P2… ,pkp_1,p_2,… ,p_kp1,p2,… Pk, so that through multiple dice rolls you can get a set of samples that make up the Gaussian mixture model mentioned in the table above.