From Facebook’s research to DeepMind’s legendary algorithms, deep learning has reached the pinnacle of data science. It’s amazing innovation, incredible breakthroughs, and we’re just getting started in this space!
But if you’re new to the field, the word “depth” might make you suspicious. Deep learning is one of the hottest topics in the industry today, but unfortunately, it is unfamiliar and mysterious to most people. Many people have the impression that in-depth study involves a lot of math and statistics.
If you have a similar question about deep learning but don’t know how, when or where to ask – this is the place for you. This article answers most of what you want to know.
In this article, we’ll dispel some myths about deep learning and answer some widely asked questions about the field. We also have plenty of resources to get you started.
Here’s the exciting part: Deep learning isn’t as hard as most people think. Read on to find out more!
directory
Q1: What is deep learning? Why is it so popular?
Q2: Is deep learning hype or real life applications?
Q3: What is the difference between deep learning and machine learning?
Q4: What are the prerequisites for further study?
Q5: Is it necessary to have a research background for deep learning in order to start it?
Q6: Which tool/language should I choose to build a deep learning model?
Q7: Why is GPU necessary to build deep learning models?
Q8: When (and where) is deep learning applied?
Q9: Do we need a lot of data to train deep learning models?
Q10: Where can I find some basic project ideas to practice in-depth learning?
Q11: What are the free learning resources for deep learning?
Q12: FAQ in Deep Learning?
Q13: What is the future of deep learning?
Q1: What is deep learning and why is it so popular now?
Deep learning is just one branch of machine learning that has shown surprising promise in recent years. That’s because deep learning has a lot in common with how the human brain functions. The superiority of the human brain is an obvious fact, and it is considered the most creative, versatile, and effective self-learning model ever devised.
Now, let’s take an example to understand the power of the deep learning model:
What do you see in the picture above?
The obvious answer is “a car,” right? Despite the sand, greenery, clouds and many other things in the picture, our brain labels the image as a car. That’s because our brains have learned to recognize the main object of an image.
This ability to extract useful information from large amounts of extraneous data makes deep learning special. With the increasing amount of data, we want to have better, more data input into our model. Deep learning models get better as the amount of data increases.
Although deep learning has been around for years, major breakthroughs in these technologies have come only in the last few years. This is because of two main reasons – as we have seen before, the first is the increase in the production of data from various sources. The infographic below shows this trend succinctly. The second problem is the increase in hardware resources required to run these models. Gpus are becoming a requirement for running deep learning models, which are several times faster than before, and they help us build larger and deeper learning models in a relatively short amount of time.
That’s why deep learning has become a buzzword in the data science industry.
Q2: Deep learning is hype
Or is there a real life application?
Deep learning has seen many practical applications in recent years. From Netflix’s famous movie recommendation system to Google’s self-driving cars, deep learning is already transforming many businesses and promises to revolutionize almost every industry. Deep learning models are being applied to everything from diagnosing cancer to winning a presidential election, creating art and writing literature to making real life money. So it would be wrong to say that this is just a hyped topic.
Some of the major applications of deep learning that tech companies are adopting include:
The text is translated into hundreds of languages
Simplified speech recognition technology
Effective computer vision field
Facial feature detection
Locate malignant cells and other foreign bodies
However, some argue that deep learning is overblown because the tag data needed to train deep learning models is not readily available. Even with the data, the computational power needed to train these models is expensive. Therefore, because of these barriers, people do not experience the powerful power of deep learning and call it hype.
The following blogs can be used to build some real life deep learning applications:
Six Deep Learning Applications that Beginners can Build in Minutes (in Python)
https://www.analyticsvidhya.com/blog/2017 /02/6-deep-learning-applications-beginner-python/
Q3: What is the difference between deep learning and machine learning?
This is one of the most important questions most of us need to understand. This comparison can be made in the following three directions:
1. Data dependence
The most important difference between deep learning and traditional machine learning is performance as the data size increases. Deep learning algorithms don’t perform as well when the data is small. That’s because deep learning algorithms need a lot of data to fully understand it. At this point, traditional machine learning algorithms and manual rules prevail. The following chart summarizes this fact.
2. Feature engineering
Feature engineering is the process of applying domain knowledge to feature extraction in order to reduce the complexity of data and make the model more intuitive for learning algorithms. The process is difficult, time-consuming and expensive in terms of time and expertise.
In machine learning, the features of most applications need to be identified by experts and then hand-coded by domain and data type.
For example, features can be pixel values, shapes, textures, positions, and directions. The performance of most machine learning algorithms depends on the accuracy of feature recognition and extraction.
Deep learning algorithms attempt to learn advanced features from data. This is a very unique part of deep learning and a major step ahead of traditional machine learning. Thus, deep learning reduces the task of developing a new feature extractor for each problem. For example, convolutional neural networks will try to learn low-level features, such as edges and lines in the network layer at first, then parts of the human face, and finally a high-level representation of the face.
3. Interpretability
Finally, we consider interpretability as a factor in comparing machine learning and deep learning. This factor is the main reason why deep learning needs to be considered before it can be applied to industry.
Let me give you an example. Suppose we use deep learning to automatically grade essays. It does a pretty good job of scoring, close to human performance. But there’s a problem. It does not say why the score was given. In fact, mathematically, you can figure out which nodes in the deep neural network are activated, but we don’t know what neurons should model and what those layers of neurons are doing together. So we can’t explain the results.
On the other hand, machine learning algorithms, such as decision trees, give us clear rules about why it chose this, so it’s particularly easy to explain the reasoning behind it. Therefore, algorithms such as decision trees and linear /Logistic regression are mainly used in industry because of their interpretability.
If you want a more in-depth comparison between machine learning and deep learning, I suggest you browse the following blogs:
Deep Learning vs. Machine Learning – The essential difference you Need to know!
https://www.analyticsvidhya.com/blog/2017 /04/comparison-between-deep-learning-machine-learning/
Q4: What are the prerequisites for exposure to deep learning?
Getting started exploring deep learning is not as difficult as one might think. Before you dive in, you should know some basic basics. In-depth study requires an understanding of the following topics:
For more details on the prerequisites, see below
Python Deep Learning Tutorial
https://www.analyticsvidhya.com/blog/2016/08 /deep-learning-path/
Q5: Do I need to study for a PhD
To launch a career in deep learning?
Not at all. A PhD is not a required course for deep learning. You can learn, experiment and start your career without going to school. The point of any job or position is usually to show what you can do, not your degree per se.
Having said that, if you choose to combine this with deep learning, a PhD in a specific field, such as linguistics at NLP, will certainly speed up your learning.
Q6: Which tool/language should I choose
To build a deep learning model?
Python is recommended because it has a robust ecosystem for machine learning. The Python ecosystem is made up of developers and programmers who provide open source libraries and support to users of the Python community. This makes the task of writing complex code for various algorithms much easier, and the implementation and experimentation techniques much easier.
At the same time, Python is a more general programming language that can be used for development and implementation. This greatly simplifies the transition from development to operations. That is, a deep learning product that can predict airline ticket prices can not only be developed in Python, but can also be connected to your website in the same form. This is why Python has become a universal language.
In addition, I recommend beginners to use an advanced library like Keras. This makes it easier to experiment because it provides an abstract representation of unnecessary information hidden in the algorithm. And allow access to parameters that can be tuned to improve the performance of such models. Let’s take an example to understand this:
When you press the button on the TV remote control, do you need to be concerned about the background processes taking place inside the remote control? Do you need to know what signal is sent when you press that button, or how it’s amplified?
No need, right?
Because it might take a physicist to understand this process, but for a cripple sitting in his bedroom, it’s just an overinformation.
In addition to Python, there are other contenders in deep learning, such as R, Julia, C, and Java. For alternatives to libraries, you can look at TensorFlow, Pytorch, Caf 2, DL4J, and more. We should also keep abreast of their latest developments.
If you’re not familiar with programming, there are guI-based software that doesn’t require coding and can build deep learning models, such as LOB or Google’s AutoML.
Q7: Why is GPU necessary to build deep learning models?
When you train a deep learning model, two main operations are performed:
In forward passing, the input is passed through a neural network and the output is generated after the input is processed. In reverse transmission, we correct the weights of the neural network according to the errors of forward transmission.
Both of these operations are essentially matrix multiplication. A simple matrix multiplication can be represented by the following diagram:
Here we can see that each element in the first row of the first array is multiplied by the first column of the second array. Thus, in a neural network, we can take the first array as the input to the neural network, and the second array as the weight of the network.
The task seems easy. Now to give you an intuitive sense of large-scale deep learning, VGG16 (convolutional neural network with 16 hidden layers is a commonly used deep learning application), it has 140 million parameters; AKA weights and offsets. You’re just giving the network an input, and think of all the matrix multiplication! If we use traditional methods, it takes years to train the system.
We note that the computation set of the neural network consists of multiple matrix multiplications. So how can we go faster?
We just need to do everything at the same time, not one after the other. In a nutshell, this is why we use gpus (graphics processing units) instead of cpus (central processing units) to train neural networks.
Q8: When (and where) are neural networks applied?
Deep learning has become one of the focus of people’s attention. Its “deeper” versions have made huge breakthroughs in areas such as image recognition, speech and natural language processing.
Now that we know how powerful it is, the main question arises: when can neural networks be used, and when can they not? The field is now like a gold mine, with many discoveries being made every day. Here are a few things to keep in mind to be part of the gold rush:
Try to think of the deep learning model as a child. It starts by observing how the parent walks. It then tries to walk on its own, and with each step, the child learns how to complete a specific task. He may fall over a few times, but after a few failed attempts, he learns how to walk. If you don’t let it walk, it may never learn to walk. The more opportunities you can give your child, the better.
Deep learning algorithm is a representation learning algorithm. These algorithms break down complex problems into simpler forms, making them understandable (or “representable”). Think of the process of deep learning as having to chew your food before you swallow. This is more difficult for traditional (non-representational learning) algorithms.
Each question has its own characteristics. So the data determines how you solve the problem. For example, if the problem is sequence generation, recursive neural networks are more suitable. However, for image-related problems, convolutional neural networks may be better.
Neural networks were “discovered” a long time ago, but they have only started to shine in the last few years, largely because computing resources are now better and more powerful. If you want to use these networks to solve real life problems, get ready to buy some high-end hardware!
Q9: We need a lot of data
To train deep learning models?
Indeed, we need a lot of data to train a typical deep learning model. However, we can often overcome this problem by using something called transfer learning. Let me elaborate.
One of the barriers to using deep learning models in industrial applications is the small amount of data. Examples of data needed to train some popular deep learning models are as follows:
However, task-specific deep learning models can be used for different problems in the same domain even if the data volume is not very large. This technique is called transfer learning.
For example, we have a set of 1000 images of cats and dogs labeled 1 and 0(cat 1, dog 0), and we have another set of 500 test images that we need to classify. So instead of deep learning the data from 1000 images, we can retrain the data using a pre-trained VGGNet model and use it to classify unlabeled sets of images. In your application, a pre-trained model may not be 100% accurate, but it takes advantage of other advantages and saves a lot of work.
Take a look at the following article to get a more intuitive idea of using a pre-trained model:
Techniques for using pretraining models in transfer learning and deep learning
Transfer learning and the art of using Pre-trained Models in Deep Learning
Q10: Where can I find basic project ideas
To practice deep learning?
To practice deep learning, it takes more than ideas. We also need labeled data to test whether our ideas can be implemented using deep learning.
For starters, it is strongly recommended to start with MNIST data. The dataset contains handwritten numbers and their actual labels, that is, numbers between 0 and 9. You can even evaluate your model in a number recognition contest.
Age Test Challenge is a great program for intermediate users. The dataset consisted of facial images of Indian film actors. The task is to predict a person’s age based on his or her facial features. For the sake of simplicity, this question has been transformed into a multi-category question that includes three categories: youth, middle age and old age.
You can also refer to a list of exciting deep learning datasets and problems.
25 Open Datasets for Deep Learning Every Data Scientist Must Work With
Q11: What are the free learning resources for deep learning?
As a relatively new technology, there are not enough content and tutorials available for beginners. However, free content and resources related to deep learning are steadily growing. Learning resources can be classified according to different applications of deep learning.
-
Image processing:
-
Ng’s convolutional neural network
https://www.coursera.org/learn/convolutional-neural-networks
-
Architecture understanding of convolutional neural networks
Architecture of Convolutional Neural Networks (CNNs) demystified
-
Beginner’s Guide to convolutional Neural networks
Architecture of Convolutional Neural Networks (CNNs) demystified
-
Machine learning is fun! Part three: Deep learning and convolutional neural networks
https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721
-
Image network classification based on deep convolutional Neural Network
http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
-
R-CNN
https://arxiv.org/pdf/1311.2524v5.pdfc
-
The depth of the convolution
https://www.cv-foundation.org/openaccess /content_cvpr_2015 /papers/Szegedy_Going _Deeper_With_2015_CVPR_paper.pdf
-
Sequence prediction/time prediction
-
Ng’s sequence model
https://www.coursera.org/learn/nlp-sequence-models
-
Essentials of deep learning: Introduction to LSTM
Essentials of Deep Learning : Introduction to Long Short Term Memory
-
Irrational validity of recursive neural networks
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
-
Understand LSTM networks
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
-
Recursive neural networks and LSTMS
https://ayearofai.com/rohan-lenny-3-recurrent-neural-networks-10300100899b
-
Natural language processing
-
Deep learning in natural language processing
http://cs224d.stanford.edu/
-
Distributed representation of words and phrases and their composition
https://arxiv.org/pdf/1310.4546.pdf
-
Sequence learning based on neural network
https://arxiv.org/pdf/1409.3215.pdf
-
Deep Speech: Extended end-to-end speech recognition
https://arxiv.org/pdf/1412.5567.pdf
A list of additional resources is also available on the following blogs:
-
Python Deep Learning Tutorial
A Complete Guide on Getting Started with Deep Learning in Python
-
Great deep learning resource
https://github.com/ChristosChristofidis/awesome-deep-learning
Q12: FAQ in Deep Learning?
Some common questions that may be asked in deep learning are:
Please note that this is not an exhaustive list of interview questions. You can test yourself on important questions in deep learning by taking the skills quiz below.
30 questions that test a data scientist’s deep learning skills
https://www.analyticsvidhya.com/blog/2017/08/skilltest-deep-learning/
40 questions that test a data scientist’s deep learning skills
https://www.analyticsvidhya.com/blog/2017/04/40-questions-test-data-scientist-deep-learning/
45 basic questions that test data scientists
https://www.analyticsvidhya.com/blog/2017/01/must-know-questions-deep-learning/
Q13: What is the future of deep learning?
Deep learning has made great progress in recent years, but there is still much untapped potential. We are still in the early stages of this field and every day seems to bring new breakthroughs. One of the applications we do see is in the automotive industry, where deep learning can revolutionize driving by enabling self-driving cars. While we can’t predict the future, we can see deep learning models requiring less and less input from human data scientists and researchers.
In the near future, we are sure to see a trend in which knowledge of deep learning becomes an essential skill for every data science practitioner. In fact, you’ve probably noticed a new job called “Deep learning Engineer.” This person is responsible for deploying and maintaining the deep learning model used across the company. Needless to say, the industry will be in great demand for such people.
For now, one limitation of the DL is that it only does what humans ask it to do. It takes a lot of data to learn its goals and repeat the process. This leads to bias in some applications. We can see that over time, this improvement will eliminate some of the deviations in the training process.
Over time, we may even stop distinguishing deep learning from other types of learning. It will become a popular and commonly used area and will not need to do special branding, marketing or sales promotion.
In many cases, researchers trained DL models and were still unable to explain the “why” behind them. “It worked out pretty well, but why would you tune a hyperparameter that way?” Hopefully, with the rapid development of the DL, we’ll see this “black box” concept become a thing of the past and we can explain the intuition behind the decisions it makes.
endnotes
These are the answers to the most common questions on our portal and elsewhere, and the answers that people who want to join the deep learning bandwagon are looking for.
Is there anything else you want to clarify about deep learning? Use the comments section below to ask questions; Or jump on to our discussion portal and we’ll help you!
The original article was published on June 19, 2018
Author: VIDHYA Group
This article is from “Data THU”, a partner of the cloud community. For more information, follow “Data THU”.