A list,
1 overview of Gaussian mixture model
Gaussian density function estimation is a parameterized model. Gaussian Mixture Model (GMM) is an extension of a single Gaussian probability density function. GMM can smoothly approximate the density distribution of arbitrary shapes. There are two kinds of Gaussian Mixture models: Single Gaussian Model (SGM) and Gaussian Mixture Model (GMM). Similar to clustering, each Gaussian model can be regarded as a category according to different parameters of the Gaussian Probability Density Function (PDF). Input a sample X, its value can be calculated through PDF, and then determine whether the sample belongs to the Gaussian model through a threshold. It is obvious that SGM is suitable for the classification of problems with only two categories, while GMM is more refined because it has multiple models, which is suitable for the classification of multiple categories and can be applied to complex object modeling.
1.1 Single Gaussian model
1.2 Gaussian mixture model
2. Parameter estimation of gaussian mixture model
2.1 GMM under known sample classification
Ii. Source code
The function mix = gmm_init (ncentres, data, kiter, covar_type) % % input: % ncentres: number of mixed models % train_data: training data % kiter: number of iterations of kmeans %% Output: % mix: initial set of GMM parameters [dim,data_sz]=size(data'); mix.priors=ones(1,ncentres)./ncentres; mix.centres=randn(ncentres,dim); switch covar_type case 'diag' % Store diagonals of covariance matrices as rows in a matrix mix.covars=ones(ncentres,dim); case 'full' % Store covariance matrices in a row vector of matrices mix.covars=repmat(eye(dim),[1 1 ncentres]); otherwise error(['Unknown covariance type ', mix.covar_type]); end % Arbitrary width used if variance collapses to zero: Make it 'large' so % that centre is responsible for a reasonable number of points. GMM_WIDTH=1.0; % [mix. Centres,options,post]=k_means(mix. Centres,data); [mix.centres,post]=k_means(mix.centres,data,kiter); % Set priors depending on number of points in each cluster cluster_sizes = max(sum(post,1),1); % Make sure that no prior is zero mix.priors = cluster_sizes/sum(cluster_sizes); % Normalise priors switch covar_type case 'diag' for j=1:ncentres % Pick out data points belonging to this centre c=data(find(post(:,j)),:); diffs=c-(ones(size(c,1),1)*mix.centres(j,:)); mix.covars(j,:)=sum((diffs.*diffs),1)/size(c,1); % Replace small entries by GMM_WIDTH value mix.covars(j,:)=mix.covars(j,:)+GMM_WIDTH.*(mix.covars(j,:)<eps); end case 'full' for j=1:ncentres % Pick out data points belonging to this centre c=data(find(post(:,j)),:); diffs=c-(ones(size(c,1),1)*mix.centres(j,:)); mix.covars(:,:,j)=(diffs'*diffs)/(size(c,1)+eps); % Add GMM_WIDTH*Identity to rank-deficient covariance matrices if rank(mix.covars(:,:,j))<dim mix.covars(:,:,j)=mix.covars(:,:,j)+GMM_WIDTH.*eye(dim); end end otherwise error(['Unknown covariance type ', mix.covar_type]); end mix.ncentres=ncentres; mix.covar_type=covar_type; %============================================================= function [centres,post]=k_means(centres,data,kiter) [dim,data_sz]=size(data'); ncentres=size(centres,1); % number of clusters [ignore,perm]=sort(rand(1,data_sz)); % Generate random numbers in any order perm = perm(1:ncentres); % centres=data(perm,:); % specify an initial centre id=eye(ncentres); %Matrix to make unit vectors easy to construct for n=1:kiter % Save old centres to check for termination old_centres=centres; % Calculate posteriors based on existing centres =(ones(ncentres,1)*sum((data.^2)',1))'+... ones(data_sz,1)* sum((centres.^2)',1)-2.*(data*(centres')); % Assign each point to nearest centre [minvals, index]=min(d2', [], 1); post=id(index,:); num_points = sum(post, 1); % Adjust the centres based on new posteriors for j = 1:ncentres if (num_points(j) > 0) centres(j,:) = sum(data(find(post(:,j)),:), 1)/num_points(j); end endCopy the code