1. Data sets
The dataset uses gesture recognition images with six different tags ranging from 0 to 5. portal
Ii. Principles and methods
- Deep neural network DNN is used to train the data set
- Softmax implements multiple classification and takes the maximum position as a prediction label
- Single-heat coding one-HOT implements vector mapping of labels 0-5
Three, library
import tensorflow as tf
import numpy as np
import tf_utils
import matplotlib.pyplot as plt
from tensorflow.python.framework import ops # for tf.get_variable rerun without overwriting the variable
Copy the code4. Data set processing
(a) View the image
train_x, train_y, test_x, test_y, classes = tf_utils.load_dataset()
index = 5
plt.imshow(train_x[index])
print("y =", np.squeeze(train_y[: ,index]))
print("train_x.shape:", train_x.shape)
Copy the code
(2) Tensile dimension reduction
# dimension reduction
train_x_flatten = train_x.reshape(train_x.shape[0] -1).T
test_x_flatten = test_x.reshape(test_x.shape[0] -1).T
print("train_x_flatten.shape:", train_x_flatten.shape)
Copy the codetrain_x_flatten.shape: (12288.1080)
Copy the code(3) Normalization
# normalization
train_x1 = train_x_flatten / 255
test_x1 = test_x_flatten / 255
Copy the code(4) Label independent thermal coding
# tag unique heat encoding
train_y1 = tf_utils.convert_to_one_hot(train_y, 6)
test_y1 = tf_utils.convert_to_one_hot(test_y, 6)
Copy the code(v) View data overview
print(Sample number of training set:, train_x1.shape[1])
print(Sample Size of test set:, test_x1.shape[1])
print("Training set image:", train_x1.shape)
print("Training Set tag:", train_y1.shape)
print("Test set image:", test_x1.shape)
print("Test Set tag", test_y1.shape)
Copy the codeSample number of training set:1080Sample number of test set:120Image of training set: (12288.1080Training Set tag: (6.1080) Test Set image: (12288.120) Test set tag (6.120)
Copy the codeFive, each process function
# generate placeholder
def creat_placeholder(n_x, n_y) :
x = tf.placeholder(tf.float32, [n_x, None], name="x")
y = tf.placeholder(tf.float32, [n_y, None], name="y")
return x,y
Initialize parameters
def initialize_parameters() :
w1 = tf.get_variable("w1"[25.12288],initializer=tf.contrib.layers.xavier_initializer(seed=1))
b1 = tf.get_variable("b1"[25.1],initializer=tf.zeros_initializer())
w2 = tf.get_variable("w2"[12.25], initializer = tf.contrib.layers.xavier_initializer(seed=1))
b2 = tf.get_variable("b2"[12.1], initializer = tf.zeros_initializer())
w3 = tf.get_variable("w3"[6.12], initializer = tf.contrib.layers.xavier_initializer(seed=1))
b3 = tf.get_variable("b3"[6.1], initializer = tf.zeros_initializer())
# w1 = tf.Variable(tf.random_normal([25, 12288]))
# b1 = tf.Variable(tf.zeros([25, 1]))
# w2 = tf.Variable(tf.random_normal([12, 25]))
# b2 = tf.Variable(tf.zeros([12, 1]))
# w3 = tf.Variable(tf.random_normal([6, 12]))
# b3 = tf.Variable(tf.zeros([6, 1]))
parameters = {
"w1": w1,
"b1": b1,
"w2": w2,
"b2": b2,
"w3": w3,
"b3": b3
}
return parameters
# forward propagation
def forward_propagation(X, parameters) :
w1 = parameters["w1"]
b1 = parameters["b1"]
w2 = parameters["w2"]
b2 = parameters["b2"]
w3 = parameters["w3"]
b3 = parameters["b3"]
z1 = tf.matmul(w1, X) + b1
a1 = tf.nn.relu(z1)
z2 = tf.matmul(w2, a1) + b2
a2 = tf.nn.relu(z2)
z3 = tf.matmul(w3, a2) + b3
#a3 = tf.nn.softmax(z3)
return z3
# Calculate the cost
def compute_cost(z3, Y) :
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=tf.transpose(z3), labels=tf.transpose(Y)))
return cost
Copy the code6. Neural network model building
# DNN model construction
def nn_model(train_x, train_y, test_x, test_y, learning_rate=0.0001, num_epoch=2000, minibatch_size=32, print_cost=True, is_plot=True) :
ops.reset_default_graph()
(n_x, m) = train_x.shape
n_y = train_y.shape[0]
x, y = creat_placeholder(n_x, n_y)
parameters = initialize_parameters()
z3 = forward_propagation(x, parameters)
cost = compute_cost(z3, y)
train = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
init = tf.global_variables_initializer()
seed = 3
costs = []
with tf.Session() as session:
session.run(init)
for epoch in range(num_epoch):
epoch_cost = 0
seed = seed + 1
num_minibatch = int(m / minibatch_size)
minibatches = tf_utils.random_mini_batches(train_x, train_y, minibatch_size, seed)
for minibatch in minibatches:
(minibatch_x, minibatch_y) = minibatch
_, minibatch_cost = session.run([train, cost], feed_dict={x:minibatch_x, y:minibatch_y})
epoch_cost = epoch_cost + minibatch_cost / num_minibatch
if epoch%5= =0:
costs.append(epoch_cost)
if print_cost and epoch%100= =0:
print("epoch =", epoch, "epoch_cost =", epoch_cost)
if is_plot:
plt.plot(np.squeeze(costs))
plt.title("learning_rate =" + str(learning_rate))
plt.xlabel("epoch")
plt.ylabel("cost")
plt.show()
parameters = session.run(parameters)
correct_prediction = tf.equal(tf.argmax(z3), tf.argmax(y))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
print("Training set accuracy:", accuracy.eval({x:train_x, y:train_y}))
print("Test set Accuracy:", accuracy.eval({x:test_x, y:test_y}))
return parameters
Copy the code7. Feed the data set training model
# Training model
parameters = nn_model(train_x1, train_y1, test_x1, test_y1)
Copy the codeepoch = 0 epoch_cost = 1.8915627219460225
epoch = 100 epoch_cost = 1.0814920374841401
epoch = 200 epoch_cost = 0.9045906644878967
epoch = 300 epoch_cost = 0.7937486514900668
epoch = 400 epoch_cost = 0.7092051903406779
epoch = 500 epoch_cost = 0.6288680759343233
epoch = 600 epoch_cost = 0.5487878710934612
epoch = 700 epoch_cost = 0.4825858437653743
epoch = 800 epoch_cost = 0.42227065834132105
epoch = 900 epoch_cost = 0.3695441674102436
epoch = 1000 epoch_cost = 0.31023081188852136
epoch = 1100 epoch_cost = 0.25832218676805496
epoch = 1200 epoch_cost = 0.23583885846715985
epoch = 1300 epoch_cost = 0.1781336689988772
epoch = 1400 epoch_cost = 0.15422452421802463
epoch = 1500 epoch_cost = 0.12832456477212184
epoch = 1600 epoch_cost = 0.10107426119573189
epoch = 1700 epoch_cost = 0.08046883931665708
epoch = 1800 epoch_cost = 0.06646379613966652
epoch = 1900 epoch_cost = 0.05467078682373871Accuracy of training set:0.99722224Test set Accuracy:0.75
Copy the code
8. Image prediction
(1) Image of predictive training set
index = 999
x = tf.placeholder(tf.float32, name="x")
plt.imshow(train_x[index])
image = (train_x[index]).reshape(64*64*3.1) # Pull the image down to column vectors
print(image.shape)
with tf.Session() as session:
image_prediction = session.run(forward_propagation(x, parameters), feed_dict={x:image})
prediction_label = np.squeeze(session.run(tf.argmax(image_prediction))) Argmax takes the maximum position of the vector after softmax classification
print("prediction_label:", prediction_label)
true_label = np.squeeze(train_y[: ,index])
print("true_label:", true_label)
if prediction_label == true_label:
print("Correct prediction!")
else:
print("Wrong prediction!")
Copy the code(12288.1)
prediction_label: 2
true_label: 2The prediction was correct!Copy the code
(2) Image of predictive test set
index = 46
x = tf.placeholder(tf.float32, name="x")
plt.imshow(test_x[index])
image = (test_x[index]).reshape(64*64*3.1)
print(image.shape)
with tf.Session() as session:
image_prediction = session.run(forward_propagation(x, parameters), feed_dict={x:image})
prediction_label = np.squeeze(session.run(tf.argmax(image_prediction)))
print("prediction_label:", prediction_label)
true_label = np.squeeze(test_y[: ,index])
print("true_label:", true_label)
if prediction_label == true_label:
print("Correct prediction!")
else:
print("Wrong prediction!")
Copy the code(12288.1)
prediction_label: 1
true_label: 2Wrong prediction!Copy the code
Nine,
- The difference between tf.get_variable() and tf.variable () is that tf.get_variable() is created when the Variable is not present and returns the Variable if it is not. A rerun, however, requires ops not to override, or an error will be reported.
- The Softmax classifier requires multiple output units, and in this example, six output units are required, so the resulting Z3 / A3 is the matrix of (6, 1), and argmax can be used to get the position where the maximum value is, that is, the label after the unique thermal coding.
- Xavier_initializer initializes W: For better information flow across the network, the variances of the outputs of each layer should be as equal as possible.
- The loss function tf.nn.softmax_cross_entropy_with_logits ⚠ is not supported by TF2.0 and can be used: tf.nn.softMAX_cross_entropy_with_logits_v2
- In tF.nn.softMAX_cross_entropy_with_logitS_v2 (logits= tF.transpose (Z3), labels=tf.transpose(Y)), logITS is the last linear output value of input forward propagation. Labels are labels entered with a single thermal code.
- Parameters = session.run(parameters) is not useless, it can save the final parameters to the session.
- The minibatch is divided randomly during each training round, specifying random seed changes.
- The training results can be observed: the training set works well, but the test set does not. In the test set, as in the example above, 2’s hands are so close together that it is easy to identify 1.
- _, minibatch_cost = session.run([train, cost], feed_dict={x:minibatch_x, y:minibatch_y}) Run train and cost functions at the same time.