Hello everyone, I am Yu Feng. Today I want to share with you my four steps of deep learning summarized by myself. It is inevitable that I make mistakes
Or the old saying, I am Yufeng, I hope that the article I share can help you and more friends. Welcome to forward or reprint ah!
preface
As one of the most popular fields, deep learning has become increasingly popular in both job hunting and scientific research as long as it has something to do with it. So there are more and more friends of deep learning.
Of course, it is also possible to learn if you want to. In deep learning, persistence is essential, because there is a lot of knowledge involved and it is not easy to learn in a short time. Many people are the entry to give up, if you want to learn, you must insist on oh!
Here are my four steps to deep learning. Of course, the fourth step is out of reach for many of us. The first three steps are good enough.
Self summary, if there is wrong or lack of place, but also hope not to spray, also welcome leaders more advice.
Step 1: Get started
Basic Mathematics:
Have you studied all three of the major math courses: linear algebra, probability theory and numerical computation?
Such as linear algebra matrix and vector calculation, norm, eigenvalue decomposition, singular value decomposition, determinant calculation and so on.
For example, probability distributions in probability theory, marginal probability, conditional probability, various probability distributions, expectation, variance, covariance, maximum likelihood estimation, Bayes, and so on.
For example, gradient optimization, constraint optimization, least squares and so on in numerical calculation.
Fundamentals of machine learning:
Do you know some basic concepts of machine learning: overfitting, underfitting, supervised learning and the corresponding support vector machine and other common algorithms, unsupervised learning and the corresponding principal component analysis, clustering and other common algorithms. Tree model, random gradient descent and so on.
Fundamentals of deep learning:
1. Neuron
2. Weights
3. Bias
4. Activation Function
5. Neural Network
6. Input/Output/Hidden Layer
7. MLP(Multi-layer perceptron)
8. Forward Propagation
9. Cost Function
10, Gradient Descent
11. Learning Rate
12. Backpropagation
13. Batches
14. Cycles (Epochs)
15. Dropout
16. Batch Normalization
17. Filters
18. Convolutional Neural Network (CNN)
Pooling
20. Padding
21. Data Augmentation
22. “Recurrent Neuron”
23. Recurrent Neural Network (RNN)
24. Vanishing Gradient Problem
25. Exploding Gradient Problem
Artificial neural network
BP neural network
28. L1, L2 regularization and other regularization optimization methods,
29. Basic stochastic gradient descent, momentum, some optimization methods of Adam,
Wait a few basic concepts and knowledge point.
Introduction to information
Books:
Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville
Machine Learning by Zhihua Zhou
Statistical Learning Methods (2nd edition) by Li Hang
Deep Learning in Python by Francois Chollet
Machine Learning in action: SciKit-Learn and TensorFlow by Aurelien Geron
100 pages of Machine Learning by Andriy Burkov
Course selection
Wu En of machine learning, deep learning courses (code word, pay attention to the public number: feather peak reply “Wu En” can obtain) : www.coursera.org/specializat…
CMU: Deep learning: www.cs.cmu.edu/~rsalakhu/1…
CMU: introduction to deep learning: simons.berkeley.edu/talks/tutor…
Stanford University: Theory of Deep Learning (Stat385) : stats385.github. IO /
Stanford University: Deep Learning based Natural Language Processing (CS224n)
www.youtube.com/playlist?li…
Convolutional Neural Networks in Visual Recognition (CS231N) :
cs231n.stanford.edu/
Li Hongyi: Deep learning (2017)
Link | www.bilibili.com/video/av977…
Deep Learning for Hands-on Learning — PyTorch 2021
Link | www.bilibili.com/video/BV1Fb…
Website recommendation:
www.duozhishidai.com/
Tensorflow. Google. Cn/tutorials/k…
keras.io/examples
keras.io/zh/
www.deeplearningbook.org/
Competencies to be acquired after this stage:
1. Able to handle small data sets
2. Able to build a simple network and train some small data sets such as handwritten data sets
3. Understand common networks like CNN, Resnet, VGG, etc.
Step 2: Advance
To advance is to go in one direction. You can’t go in all directions at the beginning, so going in one direction is the wisest choice.
Go deep into a field
At present, deep learning mainly includes the following areas:
Computer vision
Biometric recognition: face recognition, gait recognition, pedestrian ReID, pupil recognition;
Image processing: classification/segmentation, classification annotation, map search map, scene segmentation, vehicle license plate, OCR, AR;
Video analysis: Security monitoring, smart city;
2. Natural language processing
Voice recognition (Siri, Cortana, IFlytek), text data mining, text translation;
3. Data mining
Consumption habits, weather data, recommendation system, knowledge base (expert system);
Game 4.
Role simulation, AlphaGo (reinforcement learning);
5. Composite applications
Unmanned vehicles, unmanned aerial vehicles and robots;
Just pick one of the directions above to get started, and then dig deeper.
At this time, we should read some literature in the professional field. Literature websites recommend:
1. ArXiv retrieves papers
Official website: arxiv.org/[1\]
There will be relevant open source addresses, data sets, etc. It will be helpful to reproduce the paper.
2. papers with code
Official website: Paperswithcode.com/\ [3\]
There’s a lot of good data sets, there’s a lot of open source databases that are really good for research.
Should have the ability
1. Large data processing ability
At this time, it can process several GB of data sets, and can complete data expansion through data amplification methods and data processing functions, so that it can acquire more data sets to train its own data sets.
Finish your own projects
If you are a researcher, you should be able to apply a complex model or custom network to your task, train your data set, and tune and optimize your model to the best effect.
If you are not a scientist, you should find a training project or participate in some competitions to train yourself, complete the code development according to specific tasks, and make the best effect of the model through tuning and optimization.
3. Deep learning theory
At this point, your deep learning should not only know the basis, but the principle of the basis. When you tune or add a function, you should know the specific function of the function and the principle of realizing the function. For example, the advantages and disadvantages of various activation functions, the principle of various optimization functions, and the role of various hyperparameters, etc.
At this time, when looking back at books or watching videos, I will have a different understanding, from ignorant to have some experience, and can complete the implementation of relevant projects according to relevant tutorials, rather than just following the code and repeatedly looking for a bug for a wrong code.
Step 3: Be professional
Has enough understanding to their field, in my mind should be able to form a review of the model, such as yolov1 is put forward that year, have what kind of impact is put forward, is proposed in order to solve what, YOLOv2 what are improved, and the improvement principle, and can solve the problem of what, and so on all have a clear understanding.
In addition, I have made some achievements in my research direction, such as adding a method of my own on the basis of a certain model, or proposing a new model to solve the existing pain points in a certain direction, so as to achieve better results.
Of course, this does not include the kind of magic change, this professional level is at least in the top meeting posted more than one article of the people.
I also continue to explore in the second stage, and to the third stage grope, hope to see this article with you to work together, common progress.
Step 4: The top dog
This level is like Ng Enda, Li Feifei, Zhou Zhihua these big guys, of course, we reach this level is too difficult. Most of us can reach the professional level is already very good, as for the top level, or leave it to those genius leaders!
Hello, my friends, I am Yufeng, a non-professional programmer who entered the big factory from shuangfei dream and finally got his wish. Now I work as an algorithm engineer in a large factory.
During my master’s degree, I devoted myself to all kinds of scientific research competitions, from little white to national champion. During my graduate study, I visited Tsinghua University and wrote seven invention patents and three scientific research articles. Public number: Yufeng code word, pay attention to you can view my graduate student three years of counter-attack journey.
This and the public account will share some programming and scientific research articles, please look forward to.
At the same time, I am also the chief of B station UP: Yufeng code word, daily share technical video, welcome to onlooker.