Abstract: This article will share the principle knowledge of unsupervised learning Autoencoder, and then use MNIST handwritten number case for comparative experiment and cluster analysis.

This article is written by Eastmount in [Python Artificial Intelligence] 15. Unsupervised Learning Autoencoder principles and Clustering Visualization Cases.

What is Autoencoder

First of all, what is an Autoencoder? Self-coding is a form of neural network, note that it is an unsupervised learning algorithm. For example, now there is a picture, need to type it, and then restore the picture process, as shown below:

An image is compressed and then decompressed. When the image is compressed, the quality of the original image is reduced. When the image is decompressed, the original image is recovered with little information but contains all the key files. Why do you do that? Sometimes neural network need to input a lot of information, such as analysis of high-resolution images, the input will be tens of millions, neural network learning from tens of millions of is a very difficult work, at this point the need for compression, extraction of the original image is representative of the characteristics of information or compression input information, and then the compressed information into the neural network learning. This makes learning easier, so that’s where autocoding comes in.

As shown in the figure below, the white X of the original data is compressed and decompressed into black X, and then the error is calculated by comparing the two X’s, and then reverse transmission is carried out to gradually improve the accuracy of the self-coding.

Trained self-coding, the middle part is the essence of the original data, from the beginning to the end, we only used the input variable X, and did not use the label corresponding to the input variable, so self-coding is an unsupervised learning algorithm.

But when you actually use autocoding, you usually only use the first half of it, called an encoder, and you get the essence of the original data. Then we only need to create small neural network for training, which not only reduces the burden of neural network, but also can achieve good results.

The following figure shows self-coded data, which can summarize the characteristics of each type of data. If these data are placed on a two-dimensional image, each type of data can distinguish the original data with its essence. Self-coding can extract data features similar to PCA (principal component analysis), and can also be used for dimensionality reduction, which even exceeds THAT of PCA.

2.Autoencoder analysis MNIST data

Autoencoder algorithm belongs to unsupervised learning. It is a process of compressing data features and then decompressing the compressed features, similar to PCA dimensional-reduction compression.

The code for this article consists of two parts:

  • The first part: MNIST data set is used to compress and decompress the feature and compare the image after decompression and the image before compression to see if it is consistent. The experimental effect is similar to the image before compression.

  • The second part: Output the encoder results, compress them into two elements and visually display them. In the display picture, the same color represents the same type of picture, such as type 1 (number 1), type 2 (number 2), etc., and finally achieve unsupervised clustering.

The difference between supervised learning and unsupervised learning (1) Supervised learning methods must have training sets and test samples. Find patterns in the training set and apply them to the test sample. In unsupervised learning, there is no training set and only one set of data. (2) The method of supervised learning is to identify things, and the result of identification is to label the data to be identified. Therefore, the training sample set must consist of labeled samples. Unsupervised learning methods, on the other hand, have only the data set to be analyzed, with no labels in advance. If the data set is found to exhibit some kind of aggregation, it can be classified according to the natural aggregation, but not for the purpose of matching some kind of pre-classification label.

Let’s start coding!

First, open Anaconda, then select the “TensorFlow” environment you have set up and run Spyder.

Step 2: Import the extension package.

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
Copy the code

Third, download the data set.

Since the MNIST data set is the sample data of TensorFlow, we only need the following line of code to read the data set. If the dataset does not exist it will be downloaded online, if the dataset has already been downloaded it will be called directly.

Mnist = input_data.read_data_sets('MNIST_data', one_hot=True)Copy the code

Step 4, define parameters.

MNIST image is 28*28 pixels, and its N_input input feature is 784. Feature is continuously compressed, first compressed to 256, and then compressed to 128 after a hidden layer. Then zoom in on 128, unpack 256, and unpack 784. Finally, cost comparison is made between the extracted 784 features and the original 784 features, and the accuracy of Autoencoder is improved according to cost.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- initialization Settings -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # based parameter Settings learning_rate = 0.01 # Learning efficiency training_epochs = 5 #5 group training batch_size = 256 # Batch size display_step = 1 examples_to_show = 10 # Show 10 samples # Neural network input Settings N_input = 784 #MNIST input dataset (28*28) # hidden layer set n_hidden_1 = 256 # hidden layer n_hidden_2 = 128 # hidden layer weights = {'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])), 'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])), 'decoder_h1': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])), 'decoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_input])) } biases = { 'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])), 'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])), 'decoder_b1': tf.Variable(tf.random_normal([n_hidden_1])), 'decoder_b2': tf.Variable(tf.random_normal([n_input])) }Copy the code

The fifth step, write the core code, that is, define encoder and decoder functions to achieve compression and decompression operations.

Encoder is two layers, compressed to 256 and 128 elements, respectively. The decoder also consists of two layers, decompressed to 256 and 784 elements.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- compress and decompress function definition -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # Building the encoder def encoder(x): # Layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['encoder_h1'])) Allow biases['encoder_b1']) # Allow biases['encoder_b1'] # allow biases['encoder_b1'] # allow biases['encoder_b1'] # allow biases['encoder_b1'] biases['encoder_b2'])) return layer_2 # Building the decoder def decoder(x): Sigmoid (tf.add(tf.matmul(x, weights['decoder_h1'])), Allow biases['decoder_b1']) # allow biases['decoder_b1'] # allow biases['decoder_b1'] # allow biases (tf.add(tf.matmul(layer_1, weights['decoder_h2']) Biases [' decoder_b2 '])) return layer_2 # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the compression and decompression operation -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Decoder (encoder_op) decoder(decoder) decoder(decoder)Copy the code

Note that in the MNIST dataset, the xs data has a maximum value of 1 and a minimum value of 0, not the picture’s maximum value of 255, as it has been normalized by the sigmoid function here.

batch_xs, batch_ys = mnist.train.next_batch(batch_size) # max(x) = 1, min(x) = 0

Sixth step, define error calculation method.

Where, y_pred represents the predicted result, call decoder_op decompression function, decoder_op continues to call decoder decompression and encoder compression function, image data set X for processing.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- compared to forecast and actual results -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- y_pred = decoder_op # # prediction Input data class Labels (Labels) y_true = X # Define the minimum square deviation of loss error calculation Cost = tf.reduce_mean(tf.pow(y_true - y_pred, 2)) optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)Copy the code

The seventh step is to define the training and visualization code, which is divided into the core code of neural network operation.

Batch_x is the image data set obtained. Sess.run ([Optimizer, cost], feed_dict={X: batch_xs}) is used to calculate the error between the real image and the predicted image.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- training and visualization -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # initialization init = Tf.initialize_all_variables () # Tf.session () as sess: Sess.run (init) total_batch = int(mnist.train.num_examples/batch_size range(training_epochs): # Loop over all batches for i in range(total_batch): Batch_xs, batch_ys = mnist.train.next_batch(batch_size) # Max (x)=1 min(x)=0 Cost], feed_dict={X: batch_xs}) # Display error for each epoch if epoch % display_step == 0: print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c)) print("Optimization Finished!" )Copy the code

Step 8: Call matplotlib library to draw a map and visually compare the original image with the predicted image.

Encode_decode = sess.run(y_pred, feed_dict={X: Mnist.test. images[:examples_to_show]}) # compare the original plots with the predicted plots(2, 10, figsize=(10, For I in range(examples_to_show) a[0][i].imshow(np.reshape(mnist.test.images[i], (28, 28))) a[1][i].imshow(np.reshape(encode_decode[i], (28, 28))) plt.show()Copy the code

Step 9, run the code and analyze the results.

The output results are shown in the figure below, and the error keeps decreasing, indicating that our unsupervised neural network has learned knowledge.

Extracting MNIST_data\train-images-idx3-ubyte.gz Extracting MNIST_data\train-labels-idx1-ubyte.gz Extracting MNIST_data\t10k-images-idx3-ubyte.gz Extracting MNIST_data\t10k-labels-idx1-ubyte.gz Epoch: 0001 COST = 0.097888887 Epoch: 0002 cost= 0.087600455 Epoch: 0003 cost= 0.083100438 Epoch: 0004 cost= 0.078879632 Epoch: 0005 cost= 0.069106154 Optimization Finished!Copy the code

After 5 batches of training, the displayed results are shown in the figure below. Above is the real original image, and below is the image data after compression and decompression. Note that 5 batches of training is actually very small, and normal conditions require more training.

Complete code:

# -*- coding: utf-8 -*- """ Created on Wed Jan 15 15:35:47 2020 @author: xiuzhang Eastmount CSDN """ import numpy as np import tensorflow as tf import matplotlib.pyplot as plt from tensorflow.examples.tutorials.mnist import input_data # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- initialization Settings -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- based parameter Settings learning_rate = 0.01 # # learning efficiency Training_epochs = 5 #5 group training batCH_size = 256 # Batch size display_step = 1 examples_to_show = 10 # Show 10 samples # Neural network input Settings n_input = X = tf.placeholder("float", [None, N_hidden_1 = 256 # n_hidden_2 = 128 # n_hidden_2 = {'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])), 'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])), 'decoder_h1': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])), 'decoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_input])) } biases = { 'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])), 'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])), 'decoder_b1': tf.Variable(tf.random_normal([n_hidden_1])), 'decoder_b2': MNIST = input_data.read_data_sets("MNIST_data", One_hot = False) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the compression and decompression function definition -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # Building the encoder  def encoder(x): # Layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['encoder_h1'])) Allow biases['encoder_b1']) # Allow biases['encoder_b1'] # allow biases['encoder_b1'] # allow biases['encoder_b1'] # allow biases['encoder_b1'] biases['encoder_b2'])) return layer_2 # Building the decoder def decoder(x): Layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['decoder_h1'])), Allow biases['decoder_b1']) # allow biases['decoder_b1'] # allow biases['decoder_b1'] # allow biases (tf.add(tf.matmul(layer_1, weights['decoder_h2']) Biases [' decoder_b2 '])) return layer_2 # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the compression and decompression operation -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Construct model # compress: 784 => 128 encoder_op = encoder(X) # compress: 784 => 128 decoder_op = decoder(encoder_op) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- compared to forecast and actual results -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- y_pred = decoder_op # # prediction Input data class Labels (Labels) y_true = X # Define the minimum square deviation of loss error calculation Cost = tf.reduce_mean(tf.pow(y_true - y_pred, 2)) optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- training and visualization -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # initialization init = Tf.initialize_all_variables () # Tf.session () as sess: Sess.run (init) total_batch = int(mnist.train.num_examples/batch_size range(training_epochs): # Loop over all batches for i in range(total_batch): Batch_xs, batch_ys = mnist.train.next_batch(batch_size) # Max (x)=1 min(x)=0 Cost], feed_dict={X: batch_xs}) # Display error for each epoch if epoch % display_step == 0: print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c)) print("Optimization Finished!" Encode_decode = sess.run(y_pred, feed_dict={X: Mnist.test. images[:examples_to_show]}) # compare the original plots with the predicted plots(2, 10, figsize=(10, For I in range(examples_to_show) a[0][i].imshow(np.reshape(mnist.test.images[i], (28, 28))) a[1][i].imshow(np.reshape(encode_decode[i], (28, 28))) plt.show()Copy the code

3. Feature clustering analysis

The first part of the experiment was completed, which compared the 10 original images with the predicted ones. We then share the second part of the experiment to generate the cluster graph.

First, modify the parameters.

Modify as follows, learning efficiency set to 0.001, training batch set to 20.

Learning_rate = 0.001 # Learning efficiency training_epochs = 20 #20 group training batch_size = 256 # Batch size display_step = 1Copy the code

The second step is to increase the number of encoder and decoder layers and modify the parameters.

We set the hidden layer to 4, which is better. First it was compressed from 784 to 128, then to 64, 10, and finally to just 2 elements (features) to be displayed on a two-dimensional image. Weights and biases were updated, and encoder and decoder were set to 4 levels.

# n_hidden_1 = 128 # n_hidden_2 = 64 # n_hidden_3 = 10 # n_hidden_4 = 2 # n_hidden_4 = 2 # n_hidden_4 = 2 # 'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])), 'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])), 'encoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_3])), 'encoder_h4': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_4])), 'decoder_h1': tf.Variable(tf.random_normal([n_hidden_4, n_hidden_3])), 'decoder_h2': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_2])), 'decoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])), 'decoder_h4': tf.Variable(tf.random_normal([n_hidden_1, n_input])) } biases = { 'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])), 'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])), 'encoder_b3': tf.Variable(tf.random_normal([n_hidden_3])), 'encoder_b4': tf.Variable(tf.random_normal([n_hidden_4])), 'decoder_b1': tf.Variable(tf.random_normal([n_hidden_3])), 'decoder_b2': tf.Variable(tf.random_normal([n_hidden_2])), 'decoder_b3': tf.Variable(tf.random_normal([n_hidden_1])), 'decoder_b4': tf.Variable(tf.random_normal([n_input])), }Copy the code

The third step, modify the compression and decompression definition function, also increased to four layers.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- compress and decompress function definition -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # Building the encoder def encoder(x): Sigmoid (tf.add(tf.matmul(x, weights['encoder_h1'])), biases['encoder_b1'])) layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['encoder_h2']), biases['encoder_b2'])) layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(layer_2, weights['encoder_h3']), Call matmul function layer_4 = tf.add(tf.matmul(layer_3, weights['encoder_h4']), enable biases['encoder_b3'] biases['encoder_b4']) return layer_4 # Building the decoder def decoder(x): Layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['decoder_h1'])), biases['decoder_b1'])) layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['decoder_h2']), biases['decoder_b2'])) layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(layer_2, weights['decoder_h3']), biases['decoder_b3'])) layer_4 = tf.nn.sigmoid(tf.add(tf.matmul(layer_3, weights['decoder_h4']), biases['decoder_b4'])) return layer_4Copy the code

Step 4, finally modify the training code, we no longer watch its training results, but the results before it decompresses.

Encoder_result = sess.run(encoder_op, feed_dict={X: Mnist.test. images}) # scatter(encoder_result[:, 0], encoder_result[:, 1], c=mnist.test.labels) plt.colorbar() plt.show()Copy the code

The complete code is as follows:

# -*- coding: utf-8 -*- """ Created on Wed Jan 15 15:35:47 2020 @author: xiuzhang Eastmount CSDN """ import numpy as np import tensorflow as tf import matplotlib.pyplot as plt from tensorflow.examples.tutorials.mnist import input_data # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- initialization Settings -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- based parameter Settings learning_rate = 0.001 # # learning efficiency Training_epochs = 20 #20 group training batch_size = 256 # Batch size display_step = 1 examples_to_show = 10 # Show 10 samples # Neural network input set n_input X = tf.placeholder("float", [None, N_hidden_1 = 128 # N_HIDden_2 = 64 # N_HIDden_3 = 10 # N_HIDden_4 = 2 # weights = { 'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])), 'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])), 'encoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_3])), 'encoder_h4': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_4])), 'decoder_h1': tf.Variable(tf.random_normal([n_hidden_4, n_hidden_3])), 'decoder_h2': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_2])), 'decoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])), 'decoder_h4': tf.Variable(tf.random_normal([n_hidden_1, n_input])) } biases = { 'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])), 'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])), 'encoder_b3': tf.Variable(tf.random_normal([n_hidden_3])), 'encoder_b4': tf.Variable(tf.random_normal([n_hidden_4])), 'decoder_b1': tf.Variable(tf.random_normal([n_hidden_3])), 'decoder_b2': tf.Variable(tf.random_normal([n_hidden_2])), 'decoder_b3': tf.Variable(tf.random_normal([n_hidden_1])), 'decoder_b4': MNIST = input_data.read_data_sets("MNIST_data", "MNIST_data"),} One_hot = False) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the compression and decompression function definition -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # Building the encoder  def encoder(x): Sigmoid (tf.add(tf.matmul(x, weights['encoder_h1'])), biases['encoder_b1'])) layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['encoder_h2']), biases['encoder_b2'])) layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(layer_2, weights['encoder_h3']), Call matmul function layer_4 = tf.add(tf.matmul(layer_3, weights['encoder_h4']), enable biases['encoder_b3'] biases['encoder_b4']) return layer_4 # Building the decoder def decoder(x): Layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['decoder_h1'])), biases['decoder_b1'])) layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['decoder_h2']), biases['decoder_b2'])) layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(layer_2, weights['decoder_h3']), biases['decoder_b3'])) layer_4 = tf.nn.sigmoid(tf.add(tf.matmul(layer_3, weights['decoder_h4']), Biases [' decoder_b4 '])) return layer_4 # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the compression and decompression operation -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Construct model # compress: 784 => 128 encoder_op = encoder(X) # compress: 784 => 128 decoder_op = decoder(encoder_op) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- compared to forecast and actual results -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- y_pred = decoder_op # # prediction Input data class Labels (Labels) y_true = X # Define the minimum square deviation of loss error calculation Cost = tf.reduce_mean(tf.pow(y_true - y_pred, 2)) optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- training and visualization -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # initialization init = Tf.initialize_all_variables () # Tf.session () as sess: Sess.run (init) total_batch = int(mnist.train.num_examples/batch_size) # Loop over all batches for i in range(total_batch): Batch_xs, batch_ys = mnist.train.next_batch(batch_size) # Max (x)=1 min(x)=0 Cost], feed_dict={X: batch_xs}) # Display error for each epoch if epoch % display_step == 0: print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c)) print("Optimization Finished!" Encoder_result = sess.run(encoder_op, feed_dict={X: Mnist.test. images}) # scatter(encoder_result[:, 0], encoder_result[:, 1], c=mnist.test.labels) plt.colorbar() plt.show()Copy the code

This training process takes a little time, and the result is shown below:

The clustering result is shown in the figure below, which puts different colors in a pile and corresponds to different numbers. For example, the data set on the bottom left is clustered into the number 0 by unsupervised learning, and on the other side is another data set.

But the clustering results need to be improved, because this is just a simple example of Autoencoder. I hope this article can help bloggers to understand and understand unsupervised learning and Autoencoder algorithm, and the subsequent author will share more in-depth good cases.

References:

[1] Yang Xiuzhang, Yan Na. Python Network Data crawling and analysis from beginner to Master (Analysis) [M]. Beijing: Beihang University Press, 2018.

[2] “Mo Fan God” netease cloud video address

[3] study.163.com/course/cour…

[4] github.com/siucaan/CNN…

[5] github.com/eastmountyx…

[6] Zhou Zhihua. Machine Learning

[7] Deep learning (07) RNN- Recurrent neural networks -02- Implementation of Tensorflow – Lawlite

[8] github.com/lawlite19/D…

Click to follow, the first time to learn about Huawei cloud fresh technology ~