Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

First, RNN network structure

First, compare the structure similarities and differences between simple fully connected neural network and RNN. As shown in the figure below, we can find that RNN has more parameter H than fully connected neural network, so the neuron formula of RNN has one more term than that of fully connected neural network.

As for the training process, it is no different from the fully connected neural network, which is based on the method of gradient descent to continuously reduce the difference between the predicted value and the real value.

The formula on the left of the figure below is the output neuron formula, with which we can see that: the whole RNN structure shares one group (U, W, B), and the calculation formula of each hidden layer neuron H is composed of the current input X and the output of the previous hidden layer neuron.

Other RNN network structures

The most classic use of the above RNN structure is text (sequence) classification. You can design the input and output as you wish to accomplish the corresponding task. Let’s look at several common RNN constructs and their uses.

(1) SINGLE input multiple output RNN structure

(2) MULTI-input multi-output RNN structure

(3) MULTI-input multi-output RNN structure with unequal length of input and output

3. Advantages and limitations of RNN network

RNN is the advantage of network structure, its input can be multiple and orderly, it can simulate human reading order to read the text or other serialized data, and through the hidden layer neurons of coding, a hidden layer neurons of hidden information can be passed on to the next layer of neurons, forming a certain ability to remember, to better understand the serialized data.