One, foreword

Today is day 9, we will start the RNN series, complete the stock opening price forecast, the final R2 can reach 0.72, I will follow the CNN series with updates

My environment:

• Locale: Python3.6.5
• Compiler: Jupyter Notebook
• Deep learning environment: TensorFlow2.4.1

From column: 100 Examples of Deep Learning

What is RNN

The structure of traditional neural network is relatively simple: input layer – hidden layer – output layer

The biggest difference between RNN and traditional neural network is that each time it will bring the output of the previous time to the next hidden layer for training. As shown in the figure below:

Here’s a concrete example of how RNN works:

The user said “What time is it?” , our neural network will first break the sentence into five basic units (four words + one question mark)

Then, the five basic units are input to the RNN network in sequence. First, “What” is taken as the input of the RNN to get the output 01

The “time” is then input to the RNN network in sequence, resulting in the output 02.

As you can see, when you enter “time”, the output of the preceding “what” also affects the output of 02 (half of the hidden layer is black).

By analogy, we can see that all of the previous input results in subsequent output (you can see that the circle contains all of the previous colors).

When the neural network determines the intention, it only needs the output 05 of the last layer, as shown in the figure below:

Third, preparation

1. Set the GPU

Comment out this section of code if you are using a CPU.

import tensorflow as tf

gpus = tf.config.list_physical_devices("GPU")

if gpus:
    tf.config.experimental.set_memory_growth(gpus[0].True)  # Set GPU memory usage as required
    tf.config.set_visible_devices([gpus[0]],"GPU")
Copy the code

2. Load data

import os,math
from tensorflow.keras.layers import Dropout, Dense, SimpleRNN
from sklearn.preprocessing   import MinMaxScaler
from sklearn                 import metrics
import numpy             as np
import pandas            as pd
import tensorflow        as tf
import matplotlib.pyplot as plt
# Support Chinese
plt.rcParams['font.sans-serif'] = ['SimHei']  # is used to display Chinese labels normally
plt.rcParams['axes.unicode_minus'] = False  # is used to display the negative sign normally
Copy the code

data = pd.read_csv('./datasets/SH600519.csv')  # Read stock file

data
Copy the code

2426 rows × 8 columns

The opening price of the previous day (2426-300=2126) is used as the training set, the table counts from 0, 2:3 is extracted [2:3] column, the front is closed and the back is opened, so the opening price of 300 days after the opening price of column C is extracted as the test set """
training_set = data.iloc[0:2426 - 300.2:3].values  
test_set = data.iloc[2426 - 300:, 2:3].values  
Copy the code

4. Data preprocessing

1. The normalization

sc           = MinMaxScaler(feature_range=(0.1))
training_set = sc.fit_transform(training_set)
test_set     = sc.transform(test_set) 
Copy the code

2. Set the test set and training set

x_train = []
y_train = []

x_test = []
y_test = []

""" Using the opening price of the first 60 days as the input feature x_train and the opening price of day 61 as the input label Y_train for loop builds a total of 2426-300-60=2066 sets of training data. Build 300-60=260 sets of test data
for i in range(60.len(training_set)):
    x_train.append(training_set[i - 60:i, 0])
    y_train.append(training_set[i, 0])
    
for i in range(60.len(test_set)):
    x_test.append(test_set[i - 60:i, 0])
    y_test.append(test_set[i, 0])
    
# Disrupt the training set
np.random.seed(7)
np.random.shuffle(x_train)
np.random.seed(7)
np.random.shuffle(y_train)
tf.random.set_seed(7)
Copy the code

X_train :(2066, 60, 1) y_train:(2066,) x_test :(240, 60, 1) y_test :(240,)
x_train, y_train = np.array(x_train), np.array(y_train) # x_train = (2066, 60, 1)
x_test,  y_test  = np.array(x_test),  np.array(y_test)

Input requirements: [Number of samples to be fed, number of cyclic core time expansion steps, and number of features to be input for each time step]
x_train = np.reshape(x_train, (x_train.shape[0].60.1))
x_test  = np.reshape(x_test,  (x_test.shape[0].60.1))
Copy the code

5. Build models

model = tf.keras.Sequential([
    SimpleRNN(100, return_sequences=True), # Boolean. Whether to return the last output in the output sequence or the entire sequence.
    Dropout(0.1),                         # Prevent overfitting
    SimpleRNN(100),
    Dropout(0.1),
    Dense(1)])Copy the code

6. Activation model

# This app only measures the loss value, not the accuracy, so delete the metrics option and display only the loss value during each epoch iteration
model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
              loss='mean_squared_error')  # Loss function uses mean square error
Copy the code

Vii. Training model

history = model.fit(x_train, y_train, 
                    batch_size=64, 
                    epochs=20, 
                    validation_data=(x_test, y_test), 
                    validation_freq=1)                  # Number of epoch intervals tested

model.summary()
Copy the code

Epoch 1/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 6 s 123 ms/step - loss: 0.1809 - val_loss: 0.0310 Epoch 2/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 105 ms/step - loss: 0.0257 - val_loss: 0.0721 Epoch 3/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 85 ms/step - loss: 0.0165 - val_loss: 0.0059 Epoch 4/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 85 ms/step - loss: 0.0097 - val_loss: 0.0111 Epoch 5/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 90 ms/step - loss: 0.0099 - val_loss: 0.0139 Epoch 6/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 105 ms/step - loss: 0.0067 - val_loss: 0.0167... Epoch 16/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 95 ms/step - loss: 0.0035 - val_loss: 0.0149 Epoch 17/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 4 s 111 ms/step - loss: 0.0028 - val_loss: 0.0111 Epoch 18/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 4 s 110 ms/step - loss: 0.0029 - val_loss: 0.0061 Epoch 19/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 104 ms/step - loss: 0.0027 - val_loss: 0.0110 Epoch 20/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 90 ms/step - loss: 0.0028 - val_loss: 0.0037 the Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= simple_rnn (SimpleRNN) (None, 60, 80) 6560 _________________________________________________________________ dropout (Dropout) (None, 60, 80) 0 _________________________________________________________________ simple_rnn_1 (SimpleRNN) (None, 80) 12880 _________________________________________________________________ dropout_1 (Dropout) (None, 80) 0 _________________________________________________________________ dense (Dense) (None, 1) 81 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = Total params: 19521 Trainable params: 19521 Non - trainable params: 0 _________________________________________________________________Copy the code

Visualization of results

1. Draw the loss diagram

plt.plot(history.history['loss']    , label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title('Training and Validation Loss by K ')
plt.legend()
plt.show()
Copy the code

2. The prediction

predicted_stock_price = model.predict(x_test)                       # Test set input model for prediction
predicted_stock_price = sc.inverse_transform(predicted_stock_price) # restore the predicted data -- inversely normalize from (0,1) to the original range
real_stock_price = sc.inverse_transform(test_set[60:)# restore real data -- inversely normalize from (0,1) to the original range

Draw a comparison curve between the real data and the forecast data
plt.plot(real_stock_price, color='red', label='Stock Price')
plt.plot(predicted_stock_price, color='blue', label='Predicted Stock Price')
plt.title('Stock Price Prediction by K ')
plt.xlabel('Time')
plt.ylabel('Stock Price')
plt.legend()
plt.show()
Copy the code

(3) assessment

"" MSE: mean square error -----> Predicted value minus true value is squared, and the mean is calculated RMSE: root mean square error -----> Square root of mean square error MAE: mean absolute error -----> Predicted value minus true value is calculated, and the mean is calculated R2: Decision coefficient, can understand simple model to reflect the goodness of fit of important statistic articles detailing can refer to: https://blog.csdn.net/qq_38251616/article/details/107997435 "" "
MSE   = metrics.mean_squared_error(predicted_stock_price, real_stock_price)
RMSE  = metrics.mean_squared_error(predicted_stock_price, real_stock_price)**0.5
MAE   = metrics.mean_absolute_error(predicted_stock_price, real_stock_price)
R2    = metrics.r2_score(predicted_stock_price, real_stock_price)

print('Mean square error: %.5f' % MSE)
print('Root mean square error: %.5f' % RMSE)
print('Mean absolute error: %.5f' % MAE)
print('R2: %.5f' % R2)
Copy the code

Mean square error: 1833.92534 Root mean square error: 42.82435 Mean absolute error: 36.23424 R2: 0.72347Copy the code

Some codes in this paper refer to the related code in “Artificial Intelligence Practice: Tensorflow Notes” by Professor Cao Jian of Peking University

From column: 100 Examples of Deep Learning