What makes AutoML the “next generation of AI hacking”?

Before we talk about this, we need to understand the general steps of machine learning.

In fact, whether it is image recognition, speech recognition or other machine learning projects, the structure of the difference is very small, a good model requires a lot of experience to tune. The implementation process has the following steps:

• Data preprocessing

• Feature selection

• Model algorithm selection

• Adjustable parameter

• Re-optimization of the model after going online

• evaluation

In the context of the resurgence of AI wave, many enterprises are eager to try AI, but due to the global shortage of relevant talents, enterprises may not be able to recruit people, or they have to hire a lot of money. For example, top AI scientists in the US now make as much as NFL quarterbacks ($2.7 million per year on average) and even more than NBA stars ($8 million on average in 2017). So “automating machine learning” is a big draw for companies interested in trying or importing AI.

Deep learning technologies are often complex and difficult to develop from scratch. Some companies offer automated deep learning (ADL) platforms that help developers easily use deep learning, such as Microsoft’s CustomVisioning.AI, Google’s Cloud AutoML, Domestic companies such as the Fourth Paradigm and Zhiu Technology also launched AutoML platform. As the first in a series on AutoML, this article will focus on Google’s Cloud AutoML.

The concept of AutoML comes from a new concept proposed in 2012 by the academic community — Programming by Optimization (PbO), which literally means the development of Optimization programs. Essentially, it is to solve the problem of manual adjustment parameters during programming.

In January, Google announced Cloud AutoML. In the early hours of January 18, Beijing time, Li Feifei tweeted three times to announce Google’s latest AI product, Cloud AutoML Vision. “Everyone can customize machine learning models with this AI product without being proficient in machine learning.”

Link: https://cloud.google.com/automl

AutoML Vision, the first service from Cloud AutoML, offers automated development of custom image recognition systems. According to Google, even those with no machine learning expertise will be able to easily build custom image recognition models by understanding basic modeling concepts. Simply upload your own tag data into the system, and you get a trained machine learning model. The entire process, from data import to markup to model training, can be done through a drag-and-drop interface.

In addition to image recognition, Google plans to expand AutoML into translation, video and natural language processing in the future.

Google Cloud AutoML uses three core technologies: Neural Architecture Search Technology, Learning2Learn and Transfer Learning.

Through these techniques, parameters and structural adjustments are handed over to the machine. The idea is that the machine uses the neural architecture search technology to test and find a good combination of parameters and neural network structure. The AI developer thinks the combination is good, and after recognition, the machine remembers it and does it faster and more accurately the next time. Or you can transfer this combination to another application scenario, which is called transfer learning.

Neural architecture of AutoML

Typically, machine learning models are crafted by teams of engineers and scientists. The process of designing machine learning models by hand is difficult because the search space for all possible models can be combined so large that a typical 10-layer network typically has 10^10 candidate networks. For this reason, the process of designing networks often requires significant time and experimentation by people with significant machine learning expertise.

GoogleNet architecture. The design of this network required many years of careful experimentation and refinement from the initial version of the convolutional architecture.

To make the process of designing machine learning models more accessible, Google has been exploring ways to automate the design of machine learning models. Among many algorithms studied by them, evolutionary algorithms and reinforcement learning algorithms show great promise.

In an approach known internally at Google as Auto ML, a controller neural network can come up with a “sub-” model architecture that can then be trained and evaluated for quality in a particular task. This feedback is then used to inform the controller how to improve its next round of proposals. After thousands of iterations, a new architecture is generated. Finally, controller learning assigns high probability to spatial regions in order to achieve higher accuracy on the validation data set that is kept out, and to regions that score low on building spatial regions are less likely. Here’s what the process looks like:

This approach has been applied by Google to two highly benchmarked datasets in deep learning: image recognition using CIFAR-10 and language modeling using Penn Treebank. On these two data sets, the method can design models that are comparable to the most advanced models designed by machine learning experts.

So, what kind of neural network does it generate? One example: A recurring schema trained to predict the next word in a Penn Treebank dataset. Here on the left is a neural network designed by experts. On the right is the recurring schema created by the above method:

The architecture of machine selection does share some common features with human design, such as the use of addition to combine inputs with previously hidden states. However, there are some notable new elements: for example, the architecture chosen by the machine contains multiplication combinations (the blue node at the far left of the figure on the right, labeled “ELEM_mult”). This type of combination is not common for recurrent networks, probably because the researchers did not see a clear benefit. Interestingly, the simplest form of this approach has recently been proposed by human designers, who also believe that this multiplicative combination can actually mitigate the gradient extinction/explosion problem, suggesting that machine-selected architectures can discover a useful new type of neural network architecture.

This approach can also prove why certain types of neural networks work well. The architecture on the right here has many channels, so the gradient can be reversed, which may help explain why the LSTM RNN works better than the standard RNN.

The migration study

Transfer learning, as its name implies, is to Transfer the learned and trained model parameters to the new model to help the new model training.

Considering that most of the data or tasks are related, the learned model parameters (also known as the knowledge learned by the model) can be shared to the new model in some way through transfer learning, thus accelerating and optimizing the learning efficiency of the model without starting from scratch like most networks. Tabula rasa).

Cloud AutoML transfers the trained model to the new model training process through Transfer Learning. In this way, machine learning models can be trained with less data. This is particularly important in the medical field, where there is often insufficient training data for modeling rare diseases and special cases.

Learning2learn

Cloud AutoML automatically selects suitable models through learning2Learn function and automatically adjusts parameters with Hyperparameter Tuning technologies.

Jeff Dean, head of Google’s AI division, is excited about AutoML because it helps Google “automate problem solving,” but the use of AutoML poses unique problems.

“Because we’re using more systems than traditional hand-coded software, I think that presents us with a lot of challenges that we’re dealing with,” Dean said. If you’re learning from data and the data has already made biased decisions about it, the machine learning model of learning itself perpetuates those biases. So we’re doing a lot of work, working with others in the machine learning community, trying to figure out how to train machine learning models without bias.”

Another challenge: how to properly design safety-critical systems using AutoML to create AI for industries like healthcare. Decades of computer science best practices have been established so that these systems can be hand-coded, and machine-building machines must perform the same steps.

It’s one thing to make a mistake in classifying dogs, but it’s quite another to make a mistake in a safety-critical system, Dean said. “I think this is a very meaningful and important application direction for us, especially as we start to do machine learning in safer critical systems, like making decisions about healthcare or self-driving cars,” he said.

Other experts have weighed in on AutoML, along with questions from Google’s own bigshots.

One technology expert, who asked not to be named, told AI Frontier that Cloud AutoML’s first service is for Vision. The ImageNet data set is good and large enough that it does migrate well in most cases, and Vision is now an easier area to do. If it is NLP, CTR and other fields, it is much more difficult. People now have a bit of a “Google must be good” mentality, I have to say that Google PR ability is really good.

, of course, through the study of migration implementation AutoML this matter itself does give practitioners very big imagination space, can break the data island, a lower cost to solve more problems, such as electricity business data to do the recommendation of traditional industry, or a new company no data but can do things with other company or industry data.

Google says AutoML Vision offers a clean, graphical user interface that allows you to build new models by importing data and dragging and dropping components, and press reports have pointed out that you don’t need to write a line of code. “It’s easy not to write code, but it’s hard not to write code to get good results,” the expert told THE AI front.

AutoML has to be the next big thing for AI as a whole, and it could be a machine learning killer. But for most companies, even Google, AutoML is still a nascent field that everyone is trying to figure out, and it’s important to get ahead of the curve.

At Google’s I/O developer conference, however, there were no presentations on AutoML’s progress, aside from Jeff Dean’s occasional mention of AutoML, which gave a sense of the mystery and potential of this new technology.

In addition to Google, international giants such as Microsoft have also entered the game, as will be mentioned in future articles in this series. Of course, don’t assume that AutoML is only available to foreign users. A surprising number of Chinese companies are launching their own AutoML platforms. In this series, we’ll also introduce several homegrown AutoML platforms. Wait and see!

Refer to the link

http://developers.googleblog.cn/2017/11/automl.html

https://zhuanlan.zhihu.com/p/27792859

https://www.datasciencecentral.com/profiles/blogs/automated-deep-learning-so-simple-anyone-can-do-it

http://www.mittrchina.com/news/1582

https://www.zhihu.com/question/41979241/answer/123545914

https://www.jianshu.com/p/1430dcc71d15