Let’s leave aside what deep learning is and how powerful it is. As an outstanding engineer in the information industry, first of all, you need to know what deep learning really brings. What will be the impact of deep learning on society and various industries in the future? What role will Chinese companies with deep learning talent play in the world? What opportunities and challenges will deep learning bring to us?
“The road is long.” Deep learning requires a lot of data and huge computing resources, which brings us to the GPU world. If you think of NVIDIA (NVDA) today as just a graphics card company, it’s a bit of an understatement. Today, Tesla’s Autopilot 2.0, driving warning system, unmanned acquisition system, intelligent logistics system, etc., are all filled with NVIDIA. Deep learning is what makes NVIDIA the unicorn of the new ai companies.
Deep learning is just beginning to revolutionize every field. It’s as exciting as the dawn. Recently, there are more and more news about driverless cars, assisted driving and smart speakers. Both big companies and new companies are talking about artificial intelligence. Why? Because deep learning has dramatically lowered the bar for technology. Deep learning is an opportunity for startups to overtake on curves, as long as they have enough data and a little bit of hardware and time.
The most definitive book on deep learning is Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. However, there are not many books on the market that are related to deep learning and combine knowledge principles with case studies. The original intention of Principles and Practices of Deep Learning is to help more people understand deep learning and get involved in the field of artificial intelligence.
Principles and Practices of Deep Learning
By Chen Zhongming and Peng Lingxi
For more than a year, the author spent most of his spare time writing, writing sample code, and even thought about giving up, but persevered. Isn’t success achieved by persistence over and over again? Life, more or less also have some dreams, worth us to pay!
“Principles and Practice of Deep Learning” learning suggestions
We strive to improve our abilities and learn deep learning and ARTIFICIAL intelligence, not to compete with others, but to surpass ourselves. There is much advice on the way to success, and for the study of this book, here are a few.
Learn Python basics. The code throughout the book is written in the Python programming language, but the book does not explain much Python code, but focuses on the principles and ideas of deep learning. Therefore, you are expected to install Python 3 by yourself first and understand the features and basic usage of the Python programming language. After a certain understanding, you can directly use the sample code in this book, which will be helpful for understanding deep learning.
Practice is the sole criterion for testing truth. Every line of code and function are the author personally implementation, although different servers and there are some differences between different versions of the framework, this book to readers may also code has been slightly out of date, but I still hope you personally to try to realize the code in this book, “on paper it eventually sleep shallow” after all, practice is the sole criterion for testing truth.
Don’t get bogged down in the framework. Deep learning is like “recipe”, data is “meat and vegetables”, and deep learning framework is “pan and spatula for cooking”. Who says you must use a “Caffe” for braised potatoes or a “Google” spatula for sauteed lotus root? We just need to learn the recipe, study the properties of the “pot” and “spatula” and practice, and we can make a delicious dish. It is hoped that readers will not be confused about whether the book uses Keras framework or Tensorflow framework. Instead, they will focus on the principles and case studies of deep learning.
Read the literature. Reading a good book is like a conversation with a fine man. The knowledge of deep learning changes with each passing day. In the era of rapid knowledge update and iteration, we need to master the essential content of knowledge. One of the best ways to master the essential content is to read papers and literatures related to knowledge points to understand and think about why we need to do this and what are the advantages and disadvantages of this method. By reading more relevant literature, we can better grasp the essence of deep learning knowledge.
Deep Learning Principles and Practice features
A large number of legends, easy to understand. The author himself drew a large number of illustrations, and tried to restore the algorithm ideas of deep learning, decompose and analyze obscure algorithms, and express complex problems with legends. Vivid legends can also bring readers fun to read, learn algorithm knowledge happily, and experience the essence of deep learning algorithm.
Simplified formula, vivid metaphor. Deep learning and machine learning books usually contain a large number of complex and lengthy algorithm formulas. In order to avoid confusing readers, this book tries its best to unify formulas and symbols, simplify relevant formulas, and explain them with vivid metaphors. While enlightening readers, it also exercises their ability to analyze and solve problems.
Algorithm principle, code implementation. When introducing the principle of deep learning and related algorithms, not only the corresponding formula is given, but also the code for realizing and solving the formula is given, so that readers can clearly understand the function, input and output of the algorithm. The combination of principles and codes makes readers more intimate with the algorithm implementation of deep learning.
Simple, careful analysis. Understanding deep learning requires certain machine learning knowledge. Chapter 1 of this book introduces the relationship between deep learning and machine learning, and briefly introduces the content of machine learning. In terms of content arrangement, each chapter introduces the application scenario, structure and usage mode of the model framework in turn. Finally, a real case is used to comprehensively analyze the model structure. The aim is to enable readers to grasp the essence of deep learning.
Introductory practice, case reenactment. The real world examples at the end of each chapter do not directly stack the code, but illustrate the reasons and benefits of using the algorithmic model. Start with a simple background and implement an actual engineering project using the deep learning knowledge described above. Practice can be used to test readers’ mastery of knowledge in time and lay a practical foundation for deep learning for readers.
What is deep learning
In early 2016, AlphaGo, developed by Google DeepMind, defeated South Korean go master Lee Se-dol. As in 1997, when IBM’s Deep Blue computer beat Garry Kasparov, the world’s leading chess grandmaster, the media began to hype the era of artificial intelligence. At the same time, capital began to chase after companies related to the AI industry, and the frenzy of acquisition of AI enterprises swept in.
Artificial Intelligence, Machine Learning and Deep Learning have been conflated in much of the media describing DeepMind’s triumph. While DeepMind uses all three of these techniques in AlphaGo, strictly by definition, its real use is deep learning.
Here’s how artificial intelligence, machine learning, and deep learning relate to each other. As shown in Figure 1, artificial intelligence includes machine learning, and deep learning is one of the important branches of machine learning. The three are the relationship between inclusion and inclusion.
Since the first proposal of artificial intelligence in 1950s, the problems of artificial intelligence have been basically finalized, which can be roughly divided into six specific directions: problem solving, knowledge reasoning, planning problem, uncertainty reasoning, communication perception and action, and learning problem. There are three main directions of machine learning: classification, regression and correlation analysis. Finally, deep learning is the deep expansion of neural network in machine learning.
FIG. 1 The relationship between deep learning, machine learning and artificial intelligence
The concept of artificial intelligence has been around since 1950, while machine learning has been around 1980, and the latest craze, deep learning, has been around 2010.
In fact, artificial intelligence includes algorithms we often use. For example, in problem solving, the most classical algorithms are A* search algorithm and A-B pruning algorithm. For example, learning problem in artificial intelligence contains most of the content of machine learning. At this stage, there have been a lot of materials to introduce the algorithms related to machine learning, and the famous ten machine learning algorithms are: Decision tree, support vector machine SVM, random forest algorithm, logistic regression, Naive Bayes, KNN algorithm, K-means algorithm, Adaboost algorithm, Apriori algorithm, PageRank algorithm.
FIG. 2 Sample diagram of deep neural network
The definition of deep learning has been constantly changing in the past decade. Most scholars believe that deep learning should “have more than two layers of neural network”. Figure 2 shows an example of neural network of deep learning, which is characterized by:
· Have more neurons;
· More complex network connection mode;
· Have amazing computing capacity;
· Can automatically extract high-dimensional features of data.
The deep network introduced in this paper mainly refers to neural networks with deep layers, including artificial neural network (ANN), convolutional neural network (CNN) and recurrent neural network (RNN).
General approach to machine learning
Machine learning can be divided into two main categories according to methods: supervised learning and unsupervised learning. Supervised learning mainly consists of classification and regression, while unsupervised learning mainly consists of clustering and association analysis. Deep learning is a kind of supervised learning.
Supervised learning in machine learning refers to the use of algorithms to parse structured or unstructured annotated data, learn from the data, acquire specific structural models in the data, and then use these structures or models to predict unknown new data. Generally speaking, supervised learning is to find the expression model of the data by analyzing the data, and then apply the model to the newly input data to make decisions.
In the training stage (the stage of data analysis), feature extraction is first required according to the original data, which is called feature engineering. After getting the features, we can use decision tree, random forest and other Model algorithms to analyze the features or relations between the data, and finally get the Model about the input data (Model). In the prediction stage, the data is also extracted according to the method of feature engineering, and the model obtained in the training stage is used to predict the feature vectors and finally get the Labels to which they belong.
FIG. 3 General methods of supervised learning
Each different model in a machine learning algorithm has its own rules to interpret the incoming data and then make predictions and judgments about the incoming data. For example, the model of decision tree is to build a tree structure, where each node represents a type of data and each leaf node represents a type of category. The linear regression model uses the linear regression equation to create a set of parameters to represent the relationship between the input data, while the neural network has a set of weighted parameter vectors to represent the relationship between nodes.
So far, we have all kinds of machine learning algorithms that can model data. After modeling the data through the algorithm, we can learn the correlation between the data, extract the high-dimensional features of the data, and the trained new model can predict similar new data. But is the reality really that good? Does each of the steps in Figure 3 take the same amount of time?
The best part of machine learning is computer vision, which requires hardware encoding to get a color or grayscale image. As an example, if we want to identify the road of the stop sign, not directly on the image sliding window in the first place, but rather to edge detection, image preprocessing such as median filtering, gaussian filtering operation, after get the engineers think you can use the image, to analyze the image, create a series of object candidate box.
Then feature detection is carried out for each window through the algorithm of image detection, and feature of each window is extracted through the algorithm of machine learning, or data can be compressed through the algorithm of data mining, so as to extract more abstract feature data information. Then we need to define a classifier to judge the letters in the parking sign. For example, a simple SVM classifier can be used to classify the abstract information with four letters “STOP”. So far, it is able to use the algorithm of machine learning to perceive the image and identify whether the input image is a parking sign.
Although the above image classification results are basically available, they are far from achieving the purpose of full automatic recognition. When there is rain or snow, the signs on the side of the road are not so clearly visible, or the signs are blocked by trees, the characteristics of the signs will change. At this time, traditional machine learning algorithms are not so effective, which is also a big problem that has been perplexing scholars. Therefore, various rotation invariance and illumination invariance algorithms have been proposed successively, but no one algorithm is particularly effective.
Over time, deep learning quickly changed all that.
Reasons for choosing deep learning
Before the advent of deep learning, machine learning engineers often spent days, weeks, or even months collecting data and sifting through it, trying various feature extraction methods to extract data, or combining several different features to classify and predict data. When the author used traditional machine learning to detect data, he spent a lot of time trying to use various algorithms, feature extraction methods and classifiers to extract data features. It takes a lot of trial and error to find a specific method model that meets the business requirements.
With the outbreak of deep learning, the latest deep learning algorithms have far surpassed the traditional machine learning algorithms in data prediction and classification accuracy. Deep learning does not require us to extract features by ourselves, but to automatically screen data and automatically extract high-dimensional features of data. The general method of deep learning (see Figure 4) has less feature engineering than the general method of supervised learning in traditional machine learning (see Figure 3), saving engineers a lot of working time.
FIG. 4 General methods of deep learning
On the one hand, deep learning saves engineers’ time, reduces their workload and improves their work efficiency, so that engineers can devote more time and energy to more valuable research directions. On the other hand, the effects of deep learning have begun to surpass traditional machine learning algorithms in many fields, and even achieve better results than human beings predicted in some fields. In addition, deep learning can be seamlessly combined with big data and can input huge data sets to carry out the end-to-end learning process of big data. This simple concept has attracted numerous researchers.
Deep learning is not only a problem in the field of computer science, but also a combination of more questions about neural networks, involving many fields such as biology and neuroscience. There is still a huge space for development. Deep learning is a gold mine, and we may only be seeing the tip of the iceberg…
Thoughts before deep learning
It may sound a little highbrow, but the best way to get deep learning right is to answer these four questions:
· What data needs to be input into the deep network model?
· What kind of data do you want to extract from the deep network model?
· Which deep network model is more suitable for the data at hand?
· What results do we expect from the new data according to the deep network model?
The first question is what kind of data to input into the model. When answering this question, we have a general idea of what kind of data to choose as a sample, whether it is in the form of pictures, documents, or voice. Once we are clear about what type of data body to extract from the model, we can more clearly define the loss function of the network model.
The third question enables us to further clarify the context and implement the problem into the details of neural network, for example, how many layers of network are roughly used in CNN, and how many layers of loops and time steps should be defined in RNN. The last question will be more conducive for us to combine the algorithm and model of deep learning into engineering projects and really help us solve practical problems.
If we can answer the above four questions well, it shows that we have a full understanding and analysis of the problems we encounter, and we can find the appropriate workflow or working mode for specific tasks (data and scenes), and quickly establish our own deep learning model!
Principles and Practices of Deep Learning
By Chen Zhongming and Peng Lingxi
This book systematically and comprehensively introduces all aspects of deep learning knowledge step by step, including technical experience, use skills and practical cases. This book introduces in detail the common network models related to deep learning at present (ANN, DNN, CNN, RNN), as well as the algorithm principles and core ideas of different network models. This book uses a large number of examples to analyze the network model, these cases can deepen readers’ understanding of the network model.
In addition, the book also provides complete advanced content and corresponding cases, so that readers can have a comprehensive and in-depth understanding of deep learning knowledge and skills, to achieve the purpose of learning to apply.
The 3rd anniversary activity month of asynchronous community is open, more gifts are waiting for you
