For more in-depth articles, check out the cloud computing channel: yq.aliyun.com/cloud
Recently, I had the opportunity to speak with decision-makers from companies involved in ai. Several of these executives have been asked by investors about their strategy in the direction of machine learning and about their existing machine learning projects. So why has machine learning suddenly become a topic of discussion in corporate boardrooms?
As we all know, computers have been designed to help humans solve all kinds of problems since the very beginning. Traditional software engineering is programming for a problem. In other words, we tell the computer an algorithm that can solve a problem and let it execute it. Many real-world problems can be described as algorithms. For example, in elementary school arithmetic, we use addition to solve counting problems. Once real problems are abstracted into algorithms, computers can execute them faster and more efficiently than humans.
Gradually, however, people are discovering the limits of this process. Problems such as image recognition (judging whether a photo is of a cat, for example) are so simple to humans that it is hard to describe them as efficient algorithms. Since the subject’s features can be hidden, we can’t simply use “four legs” or “two eyes” to determine whether a cat is a photo. Moreover, photos may only show parts of the cat, and the problem becomes more complicated by identifying specific parts of the cat.
These problems for traditional programming are the strengths of machine learning. Instead of telling the computer how to solve the problem, we use samples to train the computer to learn the algorithm itself. We trained the computer with lots of pictures labeled cats (supervised learning). In this way, the algorithm evolved and was eventually able to identify images of various cats.
Different from traditional software engineering, computers in machine learning examine marginal weights in neural networks. This is very similar to the process of learning in the human brain, which relies on neurons communicating with each other. And it is difficult for humans to give a comprehensive account of this marginal weight network. In this case, deep learning emerged and proved to be a success. Deep learning is one of many machine learning methods and has become a discipline in artificial intelligence, which is one of the main branches of computer science research. Back in 2012, a Team of Google researchers successfully trained a network of 16,000 computers to recognize cats, or any given object, by processing tens of thousands of video images. And one of them is deep learning.
Many real-world problems need to be solved by machine learning. This is because many problems often require us to detect certain features or patterns in data, such as identifying an object from an image, extracting target text from language, and detecting possible fraud from transaction data.
Here’s a simple example. Suppose we have a number of sensors sending and receiving data. In order to ensure that they work properly, we need to monitor them in real time, once one of them breaks down, we need to deal with them in time. By monitoring, we can detect certain fixed patterns in the data flow that led to the failure. Once you know these patterns, you can detect them during daily operations. Once this mode occurs, possible failures can be predicted in advance, thus improving operation and maintenance efficiency.
While the principle of machine learning is not new, it is gaining popularity. There are three main reasons for this. First, thanks to big data technology, we have ample sample data that can be used to train computers. Secondly, we now have unprecedented computing power, especially on the basis of cloud computing. Third, a series of open source projects have made it possible for almost anyone to exploit these machine learning algorithms for their own projects.
Machine learning is not a replacement for traditional software engineering, but a good complement to it. Machine learning provides many useful tools that allow us to solve more problems than traditional software engineering can. Machine learning opens up many new opportunities and is increasingly being used in existing systems.
Repeating operations that follow patterns is a typical example. Imagine an application system with more than 100 functions, but the average user uses only a few of them every day. By observing what the user is doing, the computer can learn and predict what the user will do next, thus improving efficiency. Another example is the allocation and transformation of data (for example, an ETL job for populating a data warehouse), in which the computer learns repeated data and objects, automates the steps and improves performance.
We can also be found in other fields such scene: according to different students (especially “” massively open online courses or MOOC courses) custom personalized learning materials, early diagnosis of diseases, correct orientation of online marketing target group, automatic identification data quality issues, or online dating website automatic matching.
Because of its capabilities, Spark (combined with Hadoop) has become a mainstream big data framework for machine learning. Talend is also moving in this direction, and it takes it a step further, using more efficient modeling jobs. When modeling is used, it reduces complexity and gives the underlying technology its own independence. Because these techniques are constantly being improved, only a small number of experts are able to use them.
While only a few experts need to really understand the algorithmic details of machine learning, the popularity of machine learning concepts is equally important. Finding patterns in large samples eventually expands the category of problems that computers can solve, specifically automated decision making, and that’s what computers learn. It gathers knowledge from training data and then uses that knowledge to make decisions about new data. On the one hand, we can directly use the training results of machine learning to make the decision-making process smarter and more accurate. On the other hand, we can also analyze and improve the results of machine learning in other areas and adapt them to our own business.
In short, computers are now capable not only of following explicit instructions (arithmetic, for example) but also of learning from samples (image recognition, for example). In different scenarios, these two methods have their own characteristics and advantages. However, if we can think differently and combine these two approaches, we may be one step closer to our ultimate goal in artificial intelligence.
This article is recommended by Beijing Post @ Love coco – Love life teacher, translated by Ali Yunqi Community organization.
Gero Presser is the founder and CEO of QuinScape GmbH in Germany. A Fu, Haitang, Li Feng
The article is a brief translation. For more details, please refer to the original text.