Handwritten number recognition Based on MATLAB GUI RBM neural network Handwritten number recognition

A list,

Boltzmann machine is a kind of neural network model, but RBM is the most used in practical applications. The model of RBM itself is very simple, just a two-layer neural network, so it cannot be considered as a category of deep learning strictly. However, Deep Boltzmann Machine (HEREINAFTER referred to as DBM) can be regarded as the extension of RBM. It is not difficult to study DBM after understanding RBM, so this paper mainly focuses on RBM.

Returning to the structure of RBM, it is a two-layer neural network, as shown in the figure below:The neuron of the upper layer constitutes the hidden layer, and the value of the neuron of the layer is hidden with h vector. The neuron of the next layer constitutes the visible layer, and the value of the neuron of the visible layer is represented by v vector. The hidden layer and the visible layer are fully connected, which is similar to DNN. The hidden layer neurons are independent of each other, and the visible layer neurons are independent of each other. The connection weights can be expressed by a matrix W. Unlike DNN, RBM does not distinguish between forward and reverse. The state of the visible layer can act on the hidden layer, and the state of the hidden layer can act on the visible layer. The bias coefficient of the hidden layer is vector B, while that of the visible layer is vector A.

Commonly used RBM is binary, that is, whether the hidden layer or the visible layer, their neurons are only 0 or 1. This paper deals only with binary RBM.

The structure of RBM model is summarized as follows: mainly weight matrix W, bias coefficient vectors A and B, state vector H of neuron in hidden layer and state vector V of neuron in visible layer.

2 RBM Probability Distribution RBM is based on an energy-based probability distribution model. How do you understand that? The first part is the energy function, and the second part is the probability distribution function based on the energy function.

For the given state vectors H and V, the current energy function of RBM can be expressed as:Given the energy function, we can define the state of RBM as given v, and the probability distribution of H is:Where Z is the normalized factor, similar to the normalized factor in Softmax, expressed as:With probability distribution, we now conditional distribution P (h | v) :Where Z ‘is the new normalized coefficient, expressed as:In the same way, we can also calculate P (v | h), there is no longer listed. So with the conditional probability distribution, now let’s look at the activation function of RBM, and when we talk about neural networks, we can’t even get around the activation function, but we didn’t mention that. The activation function of RBM based on conditional distribution is easy to derive due to the use of the energy probability model. We (hj with P = 1 | v) as an example is derived as follows.The sigmoID activation function is used in RBM from the visible layer to the hidden layer. In the same way, we can get the hidden layer to the visible layer using the sigmoID activation function. That is: 3. Loss function and optimization of RBM modelThe key to the RBM model is to figure out the parameters W, A and B in our model. What if I solve for it? For m samples of the training set, RBM generally adopts logarithmic loss function, that is, expectation minimization is as follows:For optimization, the first thing that comes to mind is of course gradient descent to iteratively figure out W, A, and b. The loss function of a single sample is −ln(P(V)). − Ln (P(V))Notice that V here refers to a particular training sample, and V refers to any sample.

Let’s take the gradient calculation of AI as an example:It uses:In the same way, you get the gradient of W and b. Instead of deriving the results, I will give them directly:Although the gradient descent method can theoretically solve the optimization of RBM, in practical application, due to the large amount of calculation of probability distribution, the probability distribution has 2^nv+ Nh cases, so the gradient sum of all samples is not directly calculated according to the above gradient formula. Instead, the McMc-based method is used to simulate and calculate the gradient loss of each sample and then calculate the gradient sum. The commonly used method is the contrast divergence method based on Gibbs sampling. As for the contrast divergence method, it will not be expanded here because MCMC knowledge is required.

Ii. Source code

function varargout = core_Test_gui2(varargin)
% CORE_TEST_GUI2 MATLAB code for core_Test_gui2.fig
%      CORE_TEST_GUI2, by itself, creates a new CORE_TEST_GUI2 or raises the existing
%      singleton*.
%
%      H = CORE_TEST_GUI2 returns the handle to a new CORE_TEST_GUI2 or the handle to
%      the existing singleton*.
%
%      CORE_TEST_GUI2('CALLBACK',hObject,eventData,handles,...) calls the local
%      function named CALLBACK in CORE_TEST_GUI2.M with the given input arguments.
%
%      CORE_TEST_GUI2('Property'.'Value',...). creates anew CORE_TEST_GUI2 or raises the
%      existing singleton*.  Starting from the left, property value pairs are
%      applied to the GUI before core_Test_gui2_OpeningFcn gets called.  An
%      unrecognized property name or invalid value makes property application
%      stop.  All inputs are passed to core_Test_gui2_OpeningFcn via varargin.
%
%      *See GUI Options on GUIDE's Tools menu.  Choose "GUI allows only one % instance to run (singleton)".
%
% See also: GUIDE, GUIDATA, GUIHANDLES

% Edit the above text to modify the response to help core_Test_gui2

% Last Modified by GUIDE v2. 5 17-Apr- 2021. 07:50:14

% Begin initialization code - DO NOT EDIT
gui_Singleton = 1;
gui_State = struct('gui_Name',       mfilename, ...
                   'gui_Singleton',  gui_Singleton, ...
                   'gui_OpeningFcn', @core_Test_gui2_OpeningFcn, ...
                   'gui_OutputFcn',  @core_Test_gui2_OutputFcn, ...
                   'gui_LayoutFcn', [],...'gui_Callback'[]);if nargin && ischar(varargin{1})
    gui_State.gui_Callback = str2func(varargin{1});
end

if nargout
    [varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:});
else
    gui_mainfcn(gui_State, varargin{:});
end
% End initialization code - DO NOT EDIT


% --- Executes just before core_Test_gui2 is made visible.
function core_Test_gui2_OpeningFcn(hObject, eventdata, handles, varargin)
% This function has no output args, see OutputFcn.
% hObject    handle to figure
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)
% varargin   command line arguments to core_Test_gui2 (see VARARGIN)

% Choose default command line output for core_Test_gui2
handles.output = hObject;

% Update handles structure
guidata(hObject, handles);

% UIWAIT makes core_Test_gui2 wait for user response (see UIRESUME)
% uiwait(handles.figure1);


% --- Outputs from this function are returned to the command line.
function varargout = core_Test_gui2_OutputFcn(hObject, eventdata, handles) 
% varargout  cell array for returning output args (see VARARGOUT);
% hObject    handle to figure
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)

% Get default command line output from handles structure
varargout{1} = handles.output;



function result_Callback(hObject, eventdata, handles)
% hObject    handle to result (see GCBO)
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)

% Hints: get(hObject,'String') returns contents of result as text
%        str2double(get(hObject,'String')) returns contents of result as a double


% --- Executes during object creation, after setting all properties.
function result_CreateFcn(hObject, eventdata, handles)
% hObject    handle to result (see GCBO)
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    empty - handles not created until after all CreateFcns called

% Hint: edit controls usually have a white background on Windows.
%       See ISPC and COMPUTER.
if ispc && isequal(get(hObject,'BackgroundColor'), get(0.'defaultUicontrolBackgroundColor'))
    set(hObject,'BackgroundColor'.'white');
end


% --- Executes on button press in classify.
function classify_Callback(hObject, eventdata, handles)
% hObject    handle to classify (see GCBO)
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)
 data = handles.data; data_new=[data data]; load linear_classify_weights2; % w1 (2*22*22+1) *256W_class for (256+1) *10
  
  N=1; % each time only1Dataprobs = [data_new ones(N,1)]; % after blending bias, dataprobs is1* (2*22*22+1)
  w1probs = (dataprobs*w1)>0; % w1probs is when the bias is not integrated1*256
  w1probs = [w1probs  ones(N,1)]; % after bias, w1probs is1* (256+1)
  targetout = exp(w1probs*w_class); % output layer is0*10% will output layer output100*10/repmat(sum(targetout,2),1.10); % compares the maximum generation in each row10*1, I maximum value per row, J maximum number per row, % represents the numerical result of recognition,1- 10(They represent numbers09 -) [J] I = Max (targetout, [],2);
Copy the code