Quick start MXBoard: MXNet data visualization tool

It is very difficult to design and train deep neural networks, which usually involves a lot of adjustment, modification of network structure and trying various optimization algorithms and hyperparameters. From a theoretical point of view, the mathematical basis of deep neural network architecture is still not perfect, and related technologies are usually based on empirical results.

Fortunately, the inherent visual properties of data visualization can compensate for some of these shortcomings and create higher-level images that can help researchers in the training of deep neural networks. For example, if gradient data distribution can be plotted in real time during model training, disappearing gradient or explosion gradient can be quickly detected and corrected.

Gradient update distribution over time

In addition, the visualization of word embedding vector can clearly see that words converge into different manifolds in low-dimensional space, so as to maintain context proximity. Another example is data clustering: using the T-SNE algorithm to map high-dimensional data to lower-dimensional Spaces. Data visualization can be used extensively in deep learning in order to better understand the training process and the data itself.

The emergence of TensorBoard brings powerful visualization capabilities to TensorFlow users. We have received feedback from many different users, including enterprise users, who are using TensorFlow because of the rich feature set provided by TensorBoard. Can this powerful tool be used in other deep learning frameworks? Thanks to teamHg-memex’s efforts and their tensorboard_logger, we can now use a transparent interface to write custom data to event files, which can then be read by TensorBoard.

It is based on the MXboard we developed, a Python package for logging MXNet data frames and displaying them in TensorBoard. Follow these simple instructions to install MXBoard (https://github.com/awslabs/mxboard).

Note: To use all of MXBoard’s features, you need to install MXNet 1.2.0. Before the official release of MXNet 1.2.0, install PIP install — Pre MXNet nightly edition

MXBoard supports most of the data types of TensorBoard:

When designing the MXBoard API, we referred to the Tensorboard-PyTorch API. All of the recording apis are defined in a class called SummaryWriter. This class contains information about the file path, write frequency, queue size, and so on. To record new data of a specific data type, such as a scalar or an image, you simply call the corresponding API on the SummaryWriter object.

For example, suppose we want to plot a data distribution in which the standard deviation of the normal distribution decreases. Start by defining a SummaryWriter object, as follows:

Then create an NDArray in each loop with values from the normal distribution. We then pass NDArray to the add_histogram() function, specifying the number of bin’s and the circular index I, which will serve as the index for the data points. Finally, as with any file descriptor used in Python,.close() is called to close the file handle used by the SummaryWriter.

To visualize the diagram, you need to enter the working directory on the terminal and then start TensorBoard by typing the following command:

Then enter 127.0.0.1:8888 in the address bar of your browser. Click on HISTOGRAM and you should see the following rendering:

Visualize the narrowing of the normal distribution

Using what we have learned above, we try to accomplish the following two tasks:

1. Monitor and supervise learning and training

2. Understand the internal working principle of convolutional neural network

We will use the MNIST dataset in the Gluon Vision API and record it in real time using MXBoard:

• Cross entropy loss

• Verify and train accuracy

• Gradient data distribution

These can be used as indicators of training progress.

First, we define a SummaryWriter object:

We specified flush_secs=5 because we want to write records every five seconds to a log file so that we can track the real-time progress of the training in the browser.

We then recorded the cross entropy loss at the end of each lot:

At the end of each batch, we record the gradient as the HISTOGRAM data type, and the training and test accuracy as the SCALAR type.

We then run the Python training script and TensorBoard at the same time to visualize real-time training progress in the browser.

To reproduce the experiment, you can find the full code on Github.

The distribution of gradient renewal

Training indicators: cross entropy loss, training accuracy, validation accuracy

Visualizing convolution filters and feature maps as images makes sense for two reasons:

1. When training converges, the convolution filter shows clear pattern detection features, lines and unique colors. Convolution filters with non-convergent or over-fitting models display a lot of noise.

2. Observing the RGB reproduction of filters and feature maps can help us understand the features learned and meaningful to the network, usually edge and color detection.

Here, we use three pre-trained CNN models from MXNet Model Zoo, Inception-BN, Resnet-152, and VGG16. The filter of the first convolution layer is displayed directly in TensorBoard, together with the feature map obtained when applying the black Swan image. We can notice that networks can have different convolution kernel sizes.

• Inception-BN

Inception – BN: 7 x7 nucleus

• RESNET-152

RESNET – 152:7 x7 nucleus

• VGG16

VGG – x3 us nuclear

It can be seen that the filters of the three models show considerable smoothness and regularity, which is a common sign of a convergent model. Color filters are mainly responsible for extracting color-based features from images. Gray image is responsible for extracting general pattern and contour features of objects in the image. Visual image embedding

The last example is just as interesting. Embedding is a key concept in the field of machine learning, including computer vision and natural language processing (NLP). It is a mapping representation of high-dimensional data to low-dimensional space. In traditional image classification, the output of the penultimate layer of the convolutional neural network is usually connected to a fully connected layer with Softmax activation, which is used to predict the class or category to which the image belongs. If the network of this classification layer is stripped away, there is only one network left that outputs a feature vector for each sample, usually 512 or 1024 features per sample. This is called image embedding. We can call the add_embedding() API of MXBoard to observe the embedding distribution of data sets projected into 2D or 3D space. Images with similar visual features will be clustered together.

Here, we randomly selected 2304 images from validation and calculated their embeddings using ResNET-152, added the embeddings to the MXBoard log file and visualized them:

Resnet-152 embedded 3D projection using PCA algorithm

By default, we use PCA algorithm to project the embedding of 2304 image into 3D space. But the clustering effect is not obvious. This is because the PCA algorithm cannot maintain the spatial relationship between the original data points. Therefore, we use the T-SNE algorithm provided by the TensorBoard interface to better display the embedding. Building an optimal projection is a dynamic process:

Resnet-152 embedded 3D projection using t-SNE algorithm

After the t-SNE algorithm converges, it can be clearly seen that the data set is divided into several clusters.

Finally, we can use the TensorBoard UI to verify that the image classification is correct. We type “dog” in the upper right corner of the TensorBoard GUI. All images of validation datasets classified as “dog” tags are highlighted. We can also see that the clustering derived from the T-SNE projection is adjacent to the classification boundary.

Highlight the dog image

All the relevant code and instructions can be found here (https://github.com/reminisce/mxboard-demo).

As we can see from this MXBoard tutorial, visualization is a powerful tool for overseeing model training and gaining insight into how deep learning works. MXBoard provides MXNet with a simple, minimally invasive, easy-to-use centralized visualization solution for both scientific research and production environments. Best of all, to use it, all you need is a browser.

Link to original English text:

https://medium.com/apache-mxnet/mxboard-mxnet-data-visualization-2eed6ae31d2c