preface

Using JavaScript and frameworks like tensorflow.js is a great way to get started with machine learning. This paper mainly introduces three mainstream methods for machine learning-related applications using Tensorflow.js, and briefly illustrates the limitations of the front-end in this field.

In today’s increasingly hot concept of “end intelligence”, whether front-end students can find their own position in it is what we should pay attention to. Looking forward to today’s front-end responsibility has already exceeded the original industry to give the definition of front-end, is because there are so a group of people in continuous exploration, constantly expand their own field boundaries. Recently, I saw this earlier article and was interested in it. I translated it and mixed some of my own words, and took the code to run it, just as a Spring Festival entertainment.

Machine learning is often thought of as the domain of data science, the domain of Python developers. Over the past few years, however, a number of open source frameworks have enabled languages to do the same, including JavaScript. This article uses a few small examples to explore the possibilities of using tensorflow.js on the browser side.

What is machine learning?

Before we start coding, let’s talk briefly about what machine learning is and some of the concepts and terms involved.

define

Colloquially defined as the ability of a computer to learn from data without special programming.

If we compare that to traditional programming, what that means is that we let the computer recognize patterns in the data and generate predictions without human intervention.

In fraud detection, for example, there are no strict criteria for determining whether a transaction is fraudulent. Fraud can occur in any city, bank account, consumer, any time and so on. Trying to track this information manually is an impossible task.

However, using the vast amount of fraud data previously collected, we can train a machine learning algorithm to understand the patterns in the data and generate a model based on any transaction to predict the probability of fraud, eliminating the problem of telling the computer what to do and what to do.

The core concept

To help you understand the code examples that follow, let’s take a look at some common terms.

The Model (Model)

When you train an algorithm with a data set, a model is the output of a training process. It acts a bit like a function: it takes data as input and produces predictions as output.

Labels and Features

Label represents how to classify and label the entries in your dataset. For example, if we have a data set that describes different animals, our labels might be “cat”, “dog”, “snake”, etc.

Features describes the characteristic attributes of each item in your dataset, as in the above example, such as “whiskers, meow,” “noisy, woof,” “reptile, fierce,” etc.

With these things, a machine learning algorithm finds some connections between features, and labels are used to predict future results.

Neural Networks

Neural Networks are a series of machine learning algorithms that use artificial Neural network layers to simulate the workings of the human brain, which I won’t go into further in this article.

Now that we know some common terms, let’s take a look at what you can do with JavaScript and tensorflow.js.

Three main ways of use

  1. Use a pre-training model
  2. The migration study
  3. Use your own model

1. Use a pre-training model

Depending on the problem you are trying to solve, there may already be a model out there that is trained to solve a problem using a given data set, and you can adapt it slightly to apply it to your own code.

For example, we want to do a picture recognition function, can use a well-known picture classification model – MobileNet development. The model can be found at tensorflow.js.

Using the pre-training model is extremely simple, and you can either reference the CDN directly or install it locally as NPM modules (this method makes the training process a bit faster). Here is a simple API call:

const predictImage = async() = > {const img = document.getElementById('image');
    console.log("Model loading...");
    const model = await mobilenet.load();
    console.log("Model loaded!")
    const predictions = await model.classify(img);
    console.log('Forecast result:', predictions);
}
Copy the code

One thing this model does is recognize the image we uploaded and predict the content of the image, returning an array of three results, sorted in order of probability. Here’s what I got after two runs:

The full code can be seen here. This is how the pre-training model of Tensorflow.js is used in browsers.

2. Transfer learning

Transfer learning combines the ability to pre-train models with the ability to customize data sets, meaning that you can use the functionality of an existing model while adding your own sample data, rather than creating a model from scratch. This feature allows us to have a customizable classifier quickly and easily.

Transfer learning application example, this function is to realize the real-time collection of people head tilt direction samples and make prediction judgment subsequent actions. The model collects a certain number of samples from the left and right directions for learning. Finally, after clicking Test Prediction, our head tilt direction can be determined automatically (whether this function is familiar or not).

A KNN classifier is used here, which is combined with the MobileNet model for function customization. The full code can be seen here. Details will not be explained one by one, please refer to the original text.

Use your own model

The final approach is to define, train, and run a model entirely in the browser. To illustrate this example, let’s create a classic example of recognizing an iris.

Build a neural network to classify and identify three iris categories, namely Setosa, Virginica, and Versicolor(according to the dataset), based on an open source dataset.

At the heart of every machine learning project is a data set. The first step of the training model is to divide the data set into training set and test set. The reason is that we need to use the training set to train our algorithm, and the test set is used to test the accuracy of the prediction of our algorithm, so as to verify the usability of our model or whether it needs to be improved. The sample data set and code can be found here, containing 130+ pieces of data.

The data looks like this:

{
  "sepal_length": 5.1."sepal_width": 3.5."petal_length": 1.4."petal_width": 0.2."species": "setosa"
}
Copy the code

As you can see, the parameters, data sets, and so on are under our control and highly customizable. See, this is a simple neural network implemented by Tensorflow. js!

If you want to save the model for predicting other data, do the following:

await model.save('file:///path/to/my-model'); // in Node.js
Copy the code

limitations

Having said the three main model training methods in Tensorflow.js, the limitations of applying machine learning to the front end are also discussed.

1. The performance

When using a pre-trained model, external resources can seriously affect our application. For example, the size of some object detection models will exceed 10MB, which will significantly degrade the performance of our website. Make sure to consider the user experience and optimize resource loading speed to improve performance.

2. Quality of data

If you want to build your own model from scratch, you will need to collect your own data or use open source datasets. It is important to ensure the quality of data before proceeding with data processing. For example, if you want to build an emotion recognition model, make sure that the data set you use is accurate and diverse. If the data is not accurate enough, the output is probably not available.

3. Credibility

Using the open source pretraining model is a quick and labor-saving option. But it also means you don’t know how it’s been trained, whether the data set is comprehensive, which algorithm it is, and this kind of black box model also adds to the uncertainty of the prediction.

conclusion

All in all, using frameworks like JavaScript and tensorflow.js is still a good way to get started with machine learning. JavaScript is a great language for developers to learn about, even if the project is most likely implemented in a language like Python.

In this tutorial, we have only mentioned a few possibilities for using tensorflow.js. The number of tool libraries is mushrooming, and more specific machine learning frameworks are being developed to allow you to explore different areas. Examples include Magta. js for music and Guess.

As tools become more efficient and reliable, and machine learn-based applications built using JavaScript are just around the corner, the opportunity to push the boundaries is now. Go boys.