TensorSpace is a framework for building neural network 3D visualization applications. Developers can use the Keras-like TensorSpace API to easily create visual networks, load neural network models, and make 3D interactive rendering based on the loaded models in the browser. TensorSpace allows you to see neural network models more intuitively, and how they work through intermediate tensor operations to arrive at the final result. TensorSpace supports 3D visualization of properly preprocessed TensorFlow, Keras, and Tensorflow.js models.

  • Github address: github.com/tensorspace…
  • Chinese official website: tensorspace.org/index_zh.ht…
  • TensorSpace online learning model 3 d depth: tensorspace.org/html/playgr…

Figure 1– Interactive LeNet model created using TensorSpace

Application scenarios and features

TensorSpace is a library based on tensorflow. js, three. js and tween. js for 3D visualization of neural networks. By using TensorSpace, not only the structure of the neural network can be displayed, but also a series of processes such as the internal feature extraction of the network, the data interaction of the middle layer and the final result prediction can be presented.

By using TensorSpace, we can more intuitively observe and understand the neural network model developed based on TensorFlow, Keras and tensorflow.js. From an industrial development perspective, TensorSpace lowers the development threshold for applications related to front-end deep learning visualization. In general, TensorSpace has the following features:

  • Interaction – Use keras-like apis to build interactive 3D visualization models in the browser.
  • Intuitive — Observe and display the prediction data of the middle layer of the model, and intuitively demonstrate the process of model prediction.
  • Integration – Supports models trained using TensorFlow, Keras, and tensorflow.js.

Begin to use

Figure 2 – TensorSpace development flowchart

First we can download the latest tensorspace.js via NPM or YARN.

npm install tensorspace --save
Copy the code
yarn add tensorspace
Copy the code

TensorSpace. Js supports visualization of neural network models based on TensorFlow, Keras, and tensorflow.js training, but prior to visualization, the models need to be properly preprocessed to generate “TensorSpace compatible models”. For different deep learning model training libraries, we provide corresponding preprocessing tutorials TensorFlow model preprocessing tutorial, Keras model preprocessing tutorial and Tensorflow.js model preprocessing tutorial respectively.

The next step is to show the TensorSpace development process by quickly building a 3D LeNet model. All the code files and resource files used in this example can be found in the Examples/HelloWorld folder of the TensorSpace Github repository. Meanwhile, in order to highlight how to use the API of tensorspace.js, we used tensorflow. js to train a LeNet model in advance, completed the pretreatment process, and prepared the prediction data.

Before using the TENsorSpace.js API, you need to add tensorspace.js and its dependent libraries (tensorflow. js, three. js, Tween.js, TrackballControl.js) to the HTML page. Then create a Div as a “render container” for the TensorSpace model.


      
<html>
<head>
    <! -- Introduced dependencies on tensorflow.js, three.js, tween.js and TrackballControls.js -->
    <script src="tf.min.js"></script>
    <script src="three.min.js"></script>
    <script src="tween.min.js"></script>
    <script src="TrackballControls.js"></script> 
    <! Tensorspace.js -->
    <script src="tensorspace.min.js"></script>
</head>
<body>
  <! Create a div as a drawing container for the 3D network -->
  <div id="modelArea"></div>
</body>
</html>
Copy the code

Add the following Javascript code to the page, build the model using the API provided by tensorspace.js, load the preprocessed model, make predictions, and present them.

<script>
    // Create TensorSpace model
    let modelContainer = document.getElementById("container");
    let model = new TSP.models.Sequential(modelContainer);

    // Add a network layer to the TensorSpace model according to the LeNet structure
    // Input + 2 X (Conv2D & Maxpooling) + 3 X (Dense)
    model.add(new TSP.layers.GreyscaleInput({ shape: [28.28.1]})); model.add(new TSP.layers.Padding2d({ padding: [2.2]})); model.add(new TSP.layers.Conv2d({ kernelSize: 5.filters: 6.strides: 1 }));
    model.add(new TSP.layers.Pooling2d({ poolSize: [2.2].strides: [2.2]})); model.add(new TSP.layers.Conv2d({ kernelSize: 5.filters: 16.strides: 1 }));
    model.add(new TSP.layers.Pooling2d({ poolSize: [2.2].strides: [2.2]})); model.add(new TSP.layers.Dense({ units: 120 }));
    model.add(new TSP.layers.Dense({ units: 84 }));
    model.add(new TSP.layers.Output1d({
        units: 10.outputs: ["0"."1"."2"."3"."4"."5"."6"."Seven"."8"."9"]}));// Load the preprocessed LeNet model
    model.load({
        type: "tfjs".url: "PATH_TO_MODEL/mnist.json"
    });
    
    // Render the model and predict
    model.init(function() {
        console.log("Hello World from TensorSpace!");
        model.predict(data5);
    });
</script>
Copy the code

If all goes well, open your browser and you’ll see the model shown in Figure 3. After the model is built, interactive operations can be carried out on the model, such as opening and closing the network layer and controlling the 3D scene perspective.

This LeNet demo is hosted in CodePen. Click this link to go to CodePen to experience it online

Figure 3-Lenet model discrimination input “5”

Application show

This section will show some examples of 3D visual neural network models developed based on TensorSpace. Through these examples, you can experience different interactive models, including but not limited to: object classification, object detection, image generation, etc. We hope that by presenting these model examples, we can better and more intuitively illustrate TensorSpace application scenarios, operation methods, and presentation effects.

Click the “Online Demo” link to go to playground on TensorSpace’s official website to see the online application of the corresponding model. Considering the large size of some models (e.g. Vgg-16 > 500MB, AlexNet > 250MB…) As well as the network loading speed, some models may need longer loading time. Also, for a better UI interaction experience, we strongly recommend opening the online demo link on a medium or large screen (width > 750px).

  • LeNet online demo

  • AlexNet online demo

  • Yolov2-tiny Online demo

  • Resnet-50 online demo

  • VGG16 online demo

  • ACGAN online demo

  • MobileNetv1 online demo

Write in the last

If you have any questions about the use of TensorSpace, please send us your comments or PR in the Issues section on Github.

Finally, I would like to express my heartfelt thanks to the developers of tensorflow. js, three. js and Tween.js frameworks.