Yoha from the AI Department of Ant Financial Email: [email protected]

Ok, I admit that the title is a bit too big, but what I really want to say is “what is the role of er at the front end in the era of ARTIFICIAL intelligence, in artificial intelligence companies and in artificial intelligence departments”.

As we all know, the current or future front-end development has such several directions, front-end +Node.js, front-end + various ends (including IOT), front-end + data visualization, front-end + graphics. Wait, why are there so many plus signs after the front end? My understanding is that the function of the front end is changing, from the previous simple page to the business gradually. While this post is not a personal opinion on any of these directions, I would like to say that the front end +AI may also be a future direction (relevant, I am in the AI department after all, hahaha).

Algorithm fire in recent years, more and more people learn algorithm, of course, this also shows that the enterprise internal demand for algorithm talent. However, algorithm and engineering are inseparable, without engineering, algorithm is useless. Well, since there is engineering, then certainly not our front end, so what can we do front end? Let me give you a few conclusions and I’ll explain them one by one:

  1. Develop the company website? Ha ha ha ha ha ha, in fact, there is no problem;
  2. Development of engineering platform. Such as model service platform, data annotation platform and other large AI applications;
  3. Model visualization (not data visualization). Use visual means to explain the model and assist the algorithm to adjust the parameters;
  4. Front-end development algorithms. What? By spectrum? Rest assured, reliable! We have the experience of landing production environment! This article is not going to throw out tensorflow.js, keras.js, deeplearning.js, etc., and tell you that you can do algorithms with these frontend.

Of course, the above does not include all aspects, we are also welcome to discuss together

Development company official website

A: hello! The who! You don’t laugh! What happened to the development company website? AI company’s official website is really different from other companies! In addition to rotating graphics, full-screen page turning, HTML5 animation such high-end skills, do you do something else called AI official website? You can do face recognition, right? Can you control the camera gesture on the dot? Ok, so this simple model can be put on the front end and run. How do you say run? B: well… There is no introduction here, after all, it is not a tutorial. Interested students can talk to me privately.

Development of engineering platform

Engineering platform includes a wider range, in fact, all AI application platforms are engineering platforms, and engaged in AI application front-end generally belongs to the mid-stage front-end. What are the engineering platforms? Take 🌰 for example: data entry, data annotation platform; Modeling has modeling platform; The deployment of models includes model service platforms and a series of large platforms built around AI. And these platforms pose great engineering challenges to the front end. Tens of thousands to hundreds of thousands of code, all kinds of strange interactive operations, all kinds of visualization of big data, all kinds of visualization of rich media, and so on. But that’s not the point!! what? ! That’s not the point, right? So what’s the key? As I understand it, the key is to improve all aspects of your capabilities in the process of building these mid-platform applications, which will not only improve your front-end technology, but also greatly expand your vision, understanding of cross-field technology, understanding of the business of the company, and so on. This is the biggest difference between the mid-stage front end and the business front end and the engineering front end.

Model visualization

To be honest, model visualization does not belong to the category of front-end, nor does it completely belong to the category of algorithm or data visualization. Strictly speaking, it belongs to the cross category of front-end, data visualization and algorithm. So what exactly is model visualization?

Let me explain it this way. We all know that deep learning has always been a black box. Students engaged in deep learning affectionately call themselves “parameter tuning engineers”. Because all you know is that this neural network inputs data and goes through a bunch of calculations to get the result you want, but you don’t know why you get the result you want. What happened to this neural network trained model? Or why is the trained model inaccurate? How to adjust the training parameters to get a better model? The exact answer is not yet available. Therefore, model visualization is to explain the model and track the training process by means of visualization. Therefore, it is necessary to develop ER pair algorithm, to understand the visualization, to understand the front end, through a variety of visualization means to analyze the model, to help algorithm students to better tune parameters to understand the model. Tensorflow’s accompanying TensorBoard is a set of visual analysis tools, but it’s not enough.

So what are the requirements for the front end of model visualization?

Are d3.js and Canvas at least proficient in front-end technology? There must be a few common ways to reduce the dimension of high-dimensional data, right? Pca, T-SNE, autoencoder must have heard of? CNN, RNN, DNN these algorithms are familiar with the principle, right? Here we should also be able to see that model visualization needs such inter-disciplinary talents. Now there are many students in this direction.

Algorithm developed

Tensorflow. js, deeplearning.js, kera.js, etc. A colleague of Ali nanjing did a great job of using Tensorflow. js to teach little dinosaurs in Chrome. So, can the front-end algorithm be implemented in the production environment? ! Does it really add value to the business? The answer is: yes! First of all, there are three ways in which the front-end can perform HIGH performance computing:

  1. asm.js
  2. WebAssembly
  3. GPU

So, opencV has asM. js and WebAssembly versions, and we can do CV algorithms on the front end. Object tracking, image processing, feature detection, etc., did ok in the front end, and we landed in the production environment.

Wait, what about gpus? Speaking of gpus, of course we use tensorflow.js. So what can Tensorflow.js do in a production environment? The typical TensorFlow model is hundreds of megabytes. How does it run on the front end? It doesn’t matter, we have MobileNet, which is a neural network architecture for mobile models that dramatically reduces the number of model parameters, and can be used in browsers as well. So what can be done now? To name a few 🌰 :

  1. Face recognition
  2. Face than
  3. Object detection
  4. Signal detection
  5. Video tracking
  6. .

For example, the author’s recently transplanted TVNet algorithm based on Tensorflow. js Node can extract dense optical flow from video. And with the development of model compression technology, it is believed that there will be more models in the future to run on the front end.

Here’s a bit of GRU based algorithmic model I used to predict bitcoin’s rise and fall, just for fun:

const input = tf.input({
    shape: [timeStep, 9]
  });
  const inputReverse = tf.input({
    shape: [timeStep, 9]
  });
  const gruFwd = tf.layers.gru({
    units: 24,
    unitForgetBias: true
  });
  const gruBwd = tf.layers.gru({
    units: 24,
    unitForgetBias: true
  });
  const fwd = gruFwd.apply(input);
  const bwd = gruBwd.apply(inputReverse);

  const lstmOpt = tf.layers
    .activation({ activation: "relu" })
    .apply(tf.layers.add().apply([fwd, bwd]));
  let dense = tf.layers
    .dense({
      units: 24,
      activation: "relu"
    })
    .apply(lstmOpt);
  const outputs = tf.layers
    .dense({
      units: 2,
      activation: "softmax"
    })
    .apply(dense);
  model = tf.model({ inputs: [input, inputReverse], outputs: outputs });
  console.log("training...");
  let res;
  model.compile({
    loss: "categoricalCrossentropy",
    optimizer: "adam"
  });
  train_data = tf.tensor3d(train_data);
  train_label = tf.tensor2d(train_label);
  train_data_reverse = tf.tensor3d(train_data_reverse);

  res = await model.fit([train_data, train_data_reverse], train_label, {
    epochs: 100
  });
Copy the code

To sum up, I believe that the future of the front end in the DIRECTION of AI has a lot of promise! AI makes life better.FE makes AI better.

The last

Oh, write so much, how can not come to a hard wide: Ant financial artificial intelligence department recruit front end! The more the better, welcome all heroes to join us, let’s explore the future of front-end +AI! Email address at the beginning of the article.