The text/ZhenZi
Who should watch it? If you’re interested in using machine learning to upgrade your programming patterns, you shouldn’t miss this article.
What do I get? You can break through the mental barriers, the mindsets, and really understand the value of machine learning as a tool. Although in the “Double 11 front-end zero research and development how refined? In this series of articles and other technical articles in the Nuggets account, what do my team and I look forward to making the front end smart? What can be done? To be clear, however, the more feedback I actually get is: HMMM! Very cow force! What do I care? I am puzzled why the vision of liberating people from “cutting pictures” to “engineers” cannot move people. After a lot of research, communication and reflection, I found that the core problem is that people’s thinking has not changed, and they still view what intelligence does with the traditional front-end perspective and thinking. Today, with you break and crumb a detailed talk about how to do front-end intelligent way of thinking, with the change of thinking really understand the “front-end intelligent” essence.
First, the key problem of intelligent thinking
What are the key issues?
The key question is: Do you do it or do the model do it? When you do it you have to think it through, when you model it you have to define it. Define the problem clearly and think clearly what the problem is? Many people get confused when they hear this… Put simply, a well-defined problem means: What problem in what area are you trying to solve? Problem thinking is: what problem in what area are you trying to solve? And how to solve it? Hearing this, you will have two questions: can you solve it? How to solve?
Can you solve it?
It’s not a precise question, but what does it answer? To what extent? … There are a lot of ramifications, but let’s start with the macro dimension. The idea of intelligent problem solving comes from my analysis and thinking of some pain points in the front end and programming field. When I came to Hangzhou in September 2018, THE predecessor of IMGCook.com, “Da Vinci”, had been working for more than two years. In order to support Da Vinci’s design draft code generation ability, the data standardization and unified delivery system was built at the bottom. The upper layer constructed the ark, destiny stone and other business platform, but can solve the problem is very limited, only strictly abide by the design draft specifications and relatively simple problems can be “Da Vinci” to solve. Further analysis, Leonardo Da Vinci relied on OpenCV technology to identify the design draft, and OpenCV requires a large number of threshold constraints to achieve better results:
No matter how much we adjust the threshold in OpenCV, we always move around in the middle of various badcases, just like in the picture above, no matter how hard we try, we are crushed by the infinite combination of design possibilities and the infinite personal habits of the designer.
How to do? Machine vision models and algorithms to solve. I investigated a large number of machine vision models and algorithms:
Inspiration was gradually generated during the investigation. Through continuous exploration, good results were finally achieved on the two models of MaskRCNN and Yolo V3:
Image, text and controller can be recognized accurately based on machine learning and deep neural network. To sum up, the intelligent method of machine learning reinforcement can solve the problem that the traditional programming threshold cannot solve. Next, I’ll explain how you can use intelligence to solve a problem that traditional programming can’t: How?
Hard?
As the saying goes: attitude determines attitude, I first share myself and the team to correct the mentality of the process. The first question is: Is machine learning a difficult tool? You can guess the answer: not hard! Today, if only the application of machine learning in the field of mature framework, model is not difficult, you can click on the link to see, if the whole process for a junior programmer to use the threshold please call 110!
Install the command-line tool for managing Pipcook projects:
$ npm install -g @pipcook/pipcook-cli
Copy the code
Initialize a project:
$ mkdir pipcook-example && cd pipcook-example
$ pipcook init
Copy the code
Playground
If you are wondering what you can do in Pipcook and where you can check your training logs and models, you could start from Pipboard
$ pipcook board
Copy the code
You will see a web page prompt in your browser, and there is a MNIST showcase on the home page and play around there. If you want to train a model to recognize MNIST handwritten digits by yourself, you could try the examples below.
- pipeline-mnist-image-classification: pipeline for classific Mnist image classification problem.
- pipeline-databinding-image-classification: pipeline example to train the iamge classification task which is to classifify imgcook databinding pictures.
- pipeline-object-detection: pipeline example to train object detection task which is for component recognition used by imgcook.
- python-keras: example to use Python Keras library to train deep leraning network in js syntax and runtime.
See here for complete list, and it’s easy and quick to run these examples. For example, to do a MNIST
image classification, just run the following to start the pipeline:
$ node examples/pipeline/pipeline-mnist-image-classification.js
Copy the code
Documentation
Both Please refer to English | Chinese
How does it work?
Up pig: This is the end of the article \ ( ̄▽ ̄)/for the introduction of front-end intelligence
Audience master: frighten? (/ yanke’) / ~ ╧╧
Ladies and gentlemen, Sir Mo Fang, I’ll yard a little bit more ha! In the way of “talking about the past and the present”, let’s take a look at the traditional r&d process and see why the problems that Da Vinci couldn’t solve with OpenCV can be solved with machine learning and intelligence. Back to the key question: What’s the difference between defining a problem and solving it? In the solution of OpenCV, the definition of the problem is how to use OpenCV algorithm to extract image, Text, Controller, we first need to define: what is the image, what is the Text, what is the Controller? However, to solve the problem with intelligent thinking, we just tell the model: this is image, this is Text, this is Controller, and this is annotated data. Like I said to the eldest son: you take blocks to throw brother is wrong, the results after a few days the eldest son took the ball to throw brother…… Even if I was breaking down inside, the problem was that I didn’t make it clear that it was wrong to throw things at other people, and that there was a lot more to explain when the older son was jealous of the younger one and that he was partially taking care of himself. The same is true for OpenCV to extract image, Text, and Controller. It is too difficult to figure out the situations that these elements appear in the design draft and the rules of their own feature description. The Da Vinci team has been thinking about it for more than two years. What can we do with machine learning and intelligence? Simply give the model samples of correct and incorrect answers and ask the model to use brute force algorithms — machine learning — to find ideas from the samples. So, here is the most essential difference is that we don’t want to know specific how to solve, as long as choosing the right model, each model has its own area, can check their framework, model related documents), tidy up the right answer, sample data, tell model, training model, the answer was completed! How to organize the correct answer? We created a SimpleCook platform for collecting, organizing, and annotating data sets. Collecting means finding raw data. In the imgCook project, we used front-end technology to generate controls for identification on the SimpleCook platform. Puppeteer is the key technology here. It uses Headless browser to render the page, and then labels the image areas corresponding to IMGCook, Text, and Controller using either program rule judgment or manual annotation. The data set — the production of the correct answer is over. How do you train the model? Remember the Pipcook front-end machine learning framework described above? One command to do it: See here for complete list, and it’s easy and quick to run these examples. For example, to do a MNIST image classification, just run the following to start the pipeline:
$ node examples/pipeline/pipeline-mnist-image-classification.js
Copy the code
Just follow the Pipcook tutorial and replace the sample data from the tutorial with your own data set. That’s all. To sum up, the key questions of intelligent thinking are: organize the right answer, select the model, train the model, and then what? That’s it! ~ ~
Second, the value of intelligent thinking to solve problems
deterministic
First, the problem solving process of intelligent thinking is simple, and simple process represents certainty: there must be an answer. In fact, this is just like the confusion in life. Everyone has his own answer. Even if we tell the model the so-called correct answer, the model may not give us the expected answer to the unknown question after training, but the model will certainly give us an answer.
Secondly, from an experience point of view, as long as the problem is solved well in the industry, it should be solved well in your own field. For example, using an image classification model, it has been shown in rigorous experiments that the model can accurately identify a cat or a dog in a picture. If I tagged my cat and dog pictures and fed them to a model, the model would be trained to recognize other people’s pictures of cats and dogs, which is another level of certainty.
Finally, take a real example. For example, when you come to a new team, you can’t put all the faces and names together. After a period of practice, you communicate with each other every day, eat, drink and play together, and eventually you can remember everyone, and you can call any one of them by name. The same goes for the model, which at first couldn’t remember the faces of different people, but by labeling each person, each Angle, and each light condition with each person’s name, the model, with proper training, was able to call out each person’s name just as we do. That’s certainty.
robustness
First of all, the problem solving method of intelligent thinking is very simple. After the correct answer is told to the model, the model itself trains the parameters and weights of the model in the sample data, and summarizes the idea behind the answer by itself, which is very robust. Back to Da Vinci’s problem with OpenCV: Threshold fallacies. It is difficult to summarize and extract the characteristics of image, Text, and Controller in all cases. However, when the correct answer is available on a large enough sample, the model can extract a robust enough answer.
Secondly, those of you who are interested in algorithms can Google genetic algorithm and ant colony algorithm, etc., and you will find that all of the algorithms with strong robustness are beyond our expectations. To put it simply, it is difficult to summarize the patterns and ideas behind these algorithms. However, That doesn’t stop us from writing genetic algorithms and ant colony algorithms to train these robust patterns and algorithms in simulation or real world environments, because the way we write programs is inherently robust. Remember, in the past, when we write code, we always think before writing. Today, when we solve problems with intelligent thinking, we can write code without thinking clearly. Isn’t this way of software development robust enough?
evolutionary
When it comes to high availability on the server today, I talked to my colleagues at least 10 years ago about the “self-healing” of the system. Ten years later, few companies have come close to this goal. I’ll start by talking about the evolution of intelligent thinking, and I’ll end by sharing some of my explorations and thoughts about the “self-healing” of systems.
The evolution of intelligent thinking is reflected in the way we solve problems. In the past, we determined to understand the problem ideas before development, and realized the problem solving ideas in the development process, which is called Hardcode. When faced with problems that go beyond flexible design, we still have to write new code, constantly patching what we used to call “solutions.”
Intelligent thinking to solve the problem, we do not provide their thinking, model is extracted from the correct answer from their thinking, once encountered new situations we can put these things as a new feeding model, the model can self evolved, and similar problems to solve, this process is to realize the self evolution, The only thing we need to do is to form this evolutionary loop: the evaluation of model answers, the generation of positive and negative samples of new answers, and the construction of online training pathways.
The value of front-end intelligent thinking to solve problems lies in the certainty, robustness and evolution of problem solving. Ask, there is any programming language, technology, framework, scaffolding, tools… Can they give us that power?
Third, intelligent thinking training
Knowledge: Thought determines action
Having said all this, it’s no use if you don’t listen… What is the premise of listening? On reflection I think the first is to break the inertia. First of all, I admit that I am arrogant and opinionated as a programmer, because ever since I wrote my first function in Basic on Dos 6.22 in 1990, I printed out diamonds, circles, squares with “*” on the screen… I am sure I can write the whole world!!
Also that year, my father saved money and borrowed everything to buy me a 486 DX-100 computer, and I resolutely set foot on the road of “write the world”. Also in that year, I made every decision in my life: to go to technical secondary school, to drop out of technical secondary school after a semester to work, to be a waiter in a ballroom, to open a shop to sell videos and CDS, to close the shop for adult college entrance examination, to upgrade my law degree, to work as a salesperson in an ERP software company after graduation… All along the way, I made every decision on my own and no one would advise me, after all, I am the person who “writes the whole world”!!
Only by admitting my “headstrong” can I always remind myself to keep the “empty cup psychology”, which is an important reason why I learn things quickly, whether it is self-taught programming in the computer field to get to today, or self-taught electronic circuit design, embedded development, industrial design when starting a company…
Today, in the era of machine learning, new front-end intelligent thinking is needed to analyze, define and solve problems, and I happen to be able to clean myself up at any time and look at all new things and new technologies without inertia to find opportunities.
Ok: Behavior determines habit
There is no “impossible” in my vocabulary. I can learn anything as long as I train myself hard. So the question is: how? First, learn to restrain yourself, to understand that working with models can solve more complex and valuable problems, and to use evolution to solve problems once and for all and move on to new battlefields. It’s really hard to control and express myself. I’ve been training myself for almost ten years, but I still feel like I’m just starting out.
It is natural for programmers like me to write code to solve a problem. It is not direct to train the model to help us solve the problem. What can be done? I know that life lies in movement, but I returned home or a Ge You lying up Pad to see a movie.
How to do? I made three rules with myself:
-
1. Do not try to solve any problem by yourself, but see if there is a corresponding solution in the mature model;
-
2. Before writing code, force yourself to slow down and think about whether you can think the problem through and solve it thoroughly. If yes, write, if not, try to solve the problem intelligently;
-
3, in the front end of the field to constantly imagine some problems, with intelligent thinking to try, with the method of muscle memory to fully grasp the intelligent thinking.
Unity of
Having said that, you must be rubbing your hands, right? Welcome to join the tao Department technology front-end D2C intelligent team, which is responsible for the intelligent direction of alibaba’s front-end technology committee. No matter what problems we encounter in the cultivation of intelligent thinking, we have solutions!
Join us
How to get internship Offer from Ali?
For details, please see: How to obtain The Internship Offer of Ali Front End? Ali P8 to teach you how to prepare for interview
[social recruitment/school recruitment][Ali Tao department of technology – channel and D2C intelligent] front-end recruitment
Specific please check: [social recruitment/school recruitment] [Hangzhou] [Ali tao department of technology – channel and D2C intelligent] front-end recruitment: at this moment, only I belong to!