The first part is the foundation of deep learning

Chapter 1 What is Deep Learning

1.1 Artificial intelligence, machine learning and deep learning

First, let’s clarify the relationship between artificial intelligence, machine learning and deep learning through a diagram

1.1.1 Artificial intelligence

1.1.2 Machine learning

Machine learning offers a new programming paradigm in addition to classical programming. In classical programming, the input is data and rules defined on the data, and the output is an answer derived from the data and rules. Machine learning takes data and answers as inputs, and trains them to produce common rules that can be applied to the new data and enable the computer to produce its own answers.

1.1.3 Learning representations from data

Three elements of machine learning:

  • Input data points. For example, if your task is speech recognition, these data points might be files that record the sounds of people speaking. If your task is to tag images, then this data may be images.
  • Examples of expected data. The target data for training may be the text corresponding to sound files for speech recognition tasks, while the expected output may be tags such as “dog” and “cat” for image tagging tasks.
  • A way to measure the effectiveness of an algorithm. Measures the gap between the current and expected output of the computing algorithm. The measured result is a feedback signal that regulates the way the algorithm works. This adjustment step is what we call learning.

1.1.4 “Depth” of deep learning

Deep learning is a subfield of machine learning: it’s a new way of learning representations from data, with an emphasis on learning from connected layers that correspond to increasingly meaningful representations. The depth value in deep learning refers to the number of layers of the learning model. In deep learning, these layers are almost learned through neural network models.

The deep network can be thought of as a multistage distillation of information, with information passing through successive filters and increasing purity.With each layer of filtering, the information entropy of data decreases and the meaning of data becomes clearer.

1.1.5 Use a Picture to understand how deep learning works

The specific operations of the neural network on the data of each layer are stored in the weight of the layer, and the data change of each layer is defined by its weight, so the weight is also called parameter. A neural network system often contains hundreds or more parameters. The learning goal of neural networks is to find the correct values of these weights. However, it is not easy to find the correct values of these hundreds or even thousands of parameters, since small changes in each parameter will lead to changes in other parameters of the entire system. First we need to find an appropriate function to measure the distance between the predicted value of the system output and the actual target value of the data. This function is the loss function, also known as the objective function, in neural networks. The loss function can be used to calculate a loss value that measures how well the network works on this example. The basic technique of deep learning is to use this loss value as a feedback signal to fine-tune the weight parameter to reduce the loss value corresponding to the current example. This tuning is done by the optimizer, which implements the back-propagation algorithm, which is at the heart of deep learning. At the beginning, the parameters of the neural network are randomly assigned, and the loss value is naturally large. But as the neural network processes more and more data and the parameters tend to be more and more correct, the loss value will be smaller and smaller.

1.1.6 Machine learning before deep learning

Some commonly used models and algorithms before deep learning:

  • Naive Bayes algorithm: Used to solve classification problems.
  • Logistic regression algorithm: Although it has “regression” in its name, it is actually a classification algorithm. These two algorithms are still useful today
  • Support vector machine
  • The decision tree
  • Random forests
  • Lifting machine

These algorithms will be described in detail in another book, Machine Learning in action, which I will skip here.

1.1.7 Summary of this section

  1. Understand the general steps and principles of deep learning
  2. Master some basic concepts in deep learning such as weight, parameters, loss function, back propagation, optimizer

Next chapter: The mathematical foundations of neural networks