· Using RNN(LSTM) to do handwritten digit recognition accuracy 98%+

Reference blog address, fixed a small Bug, convergence speed and accuracy are slightly improved. Use this model in Kaggle matches with 98%+ accuracy

import os
os.environ["KMP_DUPLICATE_LIB_OK"] ="TRUE"
import time
import tensorflow as tf
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cv2 

from  tensorflow.examples.tutorials.mnist import  input_data

mnist=input_data.read_data_sets("./2RNN/data",one_hot=True)

train_rate=0.002  
train_step=1001
batch_size=1000
display_step=10

frame_size=28
sequence_length=28
hidden_num=128
n_classes=10

"" where: train_rate is the learning rate, which is a hyperparameter, currently set by experience, and of course also adaptive. Batch_size: the number of samples per batch. RNN can also be trained by using stochastic gradient descent to feed data in batches instead of feeding the whole data set each time. Sequence_size: The length of each sample sequence. Since we want to input a 28x28 image as a sequence to the RNN for training, we need to serialize the image. One of the easiest ways to do this is to assume that there is some relationship between the rows and take each row of the picture as a dimension of the sequence. So sequence_size is set to 28. What is reflected in Figure 1 is the number of xi from left to right after the left loop is expanded. RNN cell number frame_size: Size of each component in a sequence. Because each component is a row of pixels, and a row of pixels has 28 pixels. So frame_size is 28. As reflected in Figure 1, each xi in the bottom-most variable input is a vector or matrix of length frame_size. Input cell number hidden_num: indicates the number of hidden layers. The empirical setting is 5, as shown in Figure 1, there are hidden_num hidden layer units from bottom to top. N_classes: set the number of classes to 10

x=tf.placeholder(dtype=tf.float32,shape=[None,sequence_length*frame_size],name="inputx")

y=tf.placeholder(dtype=tf.float32,shape=[None,n_classes],name="expected_y")

weights=tf.Variable(tf.random_normal(shape=[hidden_num,n_classes]))
bias=tf.Variable(tf.fill([n_classes],0.1))
Weights is the last layer of the network. Its shape is hidden_numXn_class. Bias The bias of the last layer.

# Define RNN network
def RNN(x,weights,bias) :
    x=tf.reshape(x,shape=[-1,sequence_length,frame_size])
    Convert the input to the accepted shapes of dynamic_rnn: batch_size,sequence_length,frame_size
    #rnn_cell=tf.nn.rnn_cell.BasicRNNCell(hidden_num)
    rnn_cell=tf.nn.rnn_cell.BasicLSTMCell(hidden_num)

    Output_size equals the number of hidden layers, state_size equals the number of hidden layers, but the two sizes are different for LSTM cells.
    For each xi of a sequence_length sequence [x1,x2,x3...], an RNN is run in the depth direction, each of which is processed by hidden_num hidden layer units.

    output,states=tf.nn.dynamic_rnn(rnn_cell,x,dtype=tf.float32)
    h = tf.matmul(output[:,-1,:],weights)+bias
    Output will be [batch_size,sequence_length, RNn_cell.output_size
    return (h)
    Output_size = [batch_size,rnn_cell.output_size] = [batch_size,hidden_num] so it can be multiplied by weights. This is why the weights shapes are initialized to [hidden_num,n_classes] in 2.5. And then normalized by Softmax.

predy=RNN(x,weights,bias)

cost=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=predy,labels=y))

opt=tf.train.AdamOptimizer(train_rate).minimize(cost)

correct_pred=tf.equal(tf.argmax(predy,1),tf.argmax(y,1))
accuracy=tf.reduce_mean(tf.to_float(correct_pred))

testx,testy=mnist.test.next_batch(batch_size)

saver=tf.train.Saver()

with tf.Session() as sess:
    srun = sess.run
    init =  tf.global_variables_initializer()
    srun(init)
    for t in range(train_step):
        batch_x,batch_y=mnist.train.next_batch(batch_size)
        _cost_val,_ = srun([cost,opt],{x:batch_x,y:batch_y})
        if(t%display_step==0):
            accuracy_val, cost_val = srun([accuracy,cost],{x:testx,y:testy})
            print(t,cost_val,accuracy_val)

    saver.save(sess,'./2RNN/ckpt1/mnist1.ckpt',global_step=train_step)



    
Copy the code