Monderson

Product: Alibaba Amoy department front end team

For a better reading experience, welcome to search [JavaScript deep learning]

1. Introduction

In order to solve the problem of heavy audit tasks of operation pictures, we launched the material intelligent audit project on The Double 11 of 2017. In this project, we got good results based on deep learning and have reviewed tens of millions of pictures so far. Subsequently, we also tried to make a JavaScript version of tensorflow-tens.js, but found many problems difficult to solve. Fortunately, TensorFlow has since released an official version of JavaScript, and recently released version 2.0. Deep learning has made extraordinary progress in artificial intelligence in recent years, far surpassing human performance in many tasks, and I believe it will continue to be widely used in more work scenarios. With the release of Tensorflow. js, our learning cost is further reduced, and I think the engineering and popularization of deep learning is the inevitable trend.

2. Introduction

This series does not require deep learning, avoids mathematical notation, and introduces concepts through code examples. The code examples use JavaScript, using the tensorflow.js (browser /Node.js) framework.

3. Hello World

Almost all deep learning-related tutorials will take Minist project as an example. Mnist is a classic data set of machine learning, including 60,000 training images and 10,000 test images. In this project, we will train an image recognition model based on these image data through deep learning. Images are also by far the most advantageous scenario for deep learning, and through this project we can also understand the basic usage of tensorflow.js and some of the core concepts of deep learning. To put it simply, solving a problem through deep learning is generally abstracted into the following process

3.1. Data preprocessing

For the browser environment, data processing is still cumbersome compared to other environments. This time, we use a single Sprite image to store Mnist, and the corresponding label is stored in binary. We can get it directly from Google’s URL.

3.1.1. Image data preprocessing

Image data processing is very useful in browser scenarios, whether image data set processing, model prediction upload image processing is often used

const MNIST_IMAGES_SPRITE_PATH =
    'https://storage.googleapis.com/learnjs-data/model-builder/mnist_images.png';
const MNIST_LABELS_PATH =
    'https://storage.googleapis.com/learnjs-data/model-builder/mnist_labels_uint8';
Copy the code

Converting images into binary data in the browser is required

const img = new Image();
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');
img.crossOrigin = ' ';
// Once the image is loaded, use Canvas drawImage and getImageData to get the binary data
img.onload = () = > {
  img.width = img.naturalWidth;
  img.height = img.naturalHeight;
  ctx.drawImage(img, 0, i * chunkSize, img.width, chunkSize, 0.0, img.width,chunkSize);
  const imageData = ctx.getImageData(0.0, canvas.width, canvas.height);
}
img.src = MNIST_IMAGES_SPRITE_PATH;
Copy the code

3.1.2. ArrayBuffer & DataView

In tensorflow. js, ArrayBuffer types are often used, such as Canvas, Fetch API, and File API.

// Canvas Uint8ClampedArray
const canvas = document.getElementById('myCanvas');
const ctx = canvas.getContext('2d');

const imageData = ctx.getImageData(0.0, canvas.width, canvas.height);
const uint8ClampedArray = imageData.data; 

// Fetch ArrayBuffer
fetch(url)
.then(function(response){
  return response.arrayBuffer()
})
.then(function(arrayBuffer){
  // ...
});

// File ArrayBuffer
const fileInput = document.getElementById('fileInput');
const file = fileInput.files[0];
const reader = new FileReader();
reader.readAsArrayBuffer(file);
reader.onload = function () {
  const arrayBuffer = reader.result;
  / /...
};

Copy the code

3.1.3. Test data and validation data

If the model is trained with full data, it is easy for the model to over-fit with the data set, which may not be able to deal with the new data generated in the future. In order to solve this problem, we generally divide the data into multiple pieces for training and verification.

“`javascript this.datasetLabels = new Uint8Array(await labelsResponse.arrayBuffer());

// Slice the the images and labels into train and test sets. this.trainImages = this.datasetImages.slice(0, IMAGE_SIZE * NUM_TRAIN_ELEMENTS); this.testImages = this.datasetImages.slice(IMAGE_SIZE * NUM_TRAIN_ELEMENTS); this.trainLabels = this.datasetLabels.slice(0, NUM_CLASSES * NUM_TRAIN_ELEMENTS); this.testLabels = this.datasetLabels.slice(NUM_CLASSES * NUM_TRAIN_ELEMENTS);

<br /> <a name="0Ryzs"></a> ## 3.2. Deep learning is essentially to build a multi-layer network model, which cannot realize feature extraction and association learning. With TensorFlow we can easily build our own deep web javascript function createDenseModel() {const model = tm.sequential (); model.add(tf.layers.flatten({inputShape: [IMAGE_H, IMAGE_W, 1]})); model.add(tf.layers.dense({units: 42, activation: 'relu'})); model.add(tf.layers.dense({units: 10, activation: 'softmax'})); return model; } // More details in the next chapterCopy the code

3.3. Training model

After the model is compiled, training can be carried out. Currently, tensorflow. js also provides TFJS-VIS to facilitate visualization of the training process

model.compile({
  optimizer,
  loss: 'categoricalCrossentropy'.metrics: ['accuracy']});await model.fit(trainData.xs, trainData.labels, {
  batchSize,
  validationSplit,
  epochs: trainEpochs
});
Copy the code


4. Online examples

Online demo official sample code

5. Subsequent

The core concepts in tensorflow.js will be introduced next week

6. Reference

Tfjs-examples Python deep learning