- Which Deep Learning Framework is Growing Fastest?
- By Jeff Hale
- The Nuggets translation Project
- Permanent link to this article: github.com/xitu/gold-m…
- Translator: ccJia
- Proofreader: luochen1992, zhmhhu
In September 2018, I wrote an article comparing all the major deep learning frameworks in terms of demand, engineering usage, and popularity. TensorFlow is the absolute king, and Pytorch is a young challenger with an eye on the ball.
Half a year on, has the situation changed?
To answer this question, I looked at online job boards (e.g. Indeed, Monster, LinkedIn, SimplyHired), Google searches, GitHub activity, media posts, ArXiv posts, and Quora followers. Finally, based on these data, the growth chart of demand, usage and popularity is drawn for readers to understand.
Integration and updating
We found that both TensorFlow and PyTorch had some major releases.
PyTorch 1.0 was pre-released in October 2018, at the same time as Fastai 1.0 was released. These two releases are important milestones in the maturation of deep learning frameworks.
TensorFlow 2.0 Alpha was also released in April 2019. While adding many new features, the user experience has been improved. And it more closely integrates Keras as its high-level API.
methods
In this article, Keras and Fastai are included in the comparison because of their dependency on TesnsorFlow and PyTorch, which can also be used as a metric to evaluate TensorFlow and PyTorch.
As for other deep learning frameworks, I won’t discuss them. Many readers will certainly find frameworks such as Caffe, Theano, MXNET, CNTK, DeepLearning4J and Chainer valuable. However, while they have a lot of advantages, they are not as good as TensorFlow and Pytorch, and they have very little overlap with either framework.
Our analysis was performed between March 20 and 21, 2019, and the data source is in Google’s table.
I use Plotly as a visualization library, and refer to my Kaggle Kernel for more details.
Ok, now let’s look at the results of each category.
Changes in online job requirements
I scour data from several major online job boards, such as Indeed, LinkedIn, Monster, and SimplyHired, to determine which frameworks are in highest demand right now.
The search keyword I used was machine learning, followed by the corresponding library name. For example, the keyword we use to evaluate TensorFlow is Machine learning TensorFlow. For historical comparison reasons, this keyword is used for search. If machine learning is not added, the results will not be significantly different. Our search area is set to the United States.
I took data from March 2019 and subtracted data from six months earlier, and here’s what I found:
In the job list, TensorFlow grew slightly more than PyTorch. Keras also grew, roughly half as much as TensorFlow. Fastai is still not in demand.
It is worth noting that PyTorch has seen significant growth compared to TensorFlow on all other job sites except LinkedIn. But in absolute terms, TensorFlow has nearly three times as many jobs as PyTorch or Keras.
Changes in the average search activity on Google
The number of searches on the largest search engines can be used as an indicator of popularity. I’ve collected the last year’s worldwide searches for Machine Learning and Artificial Intelligence categories on Google Trends. Unfortunately, Google does not provide absolute values, but only a set of relative values.
I took the average of my interest scores over the last six months and compared them to the average of my interest scores six months ago.
In the last six months, related searches for TensorFlow have declined while related searches for PyTorch have increased.
The chart below shows the trend of people’s interest in the past year.
New Media articles
New media is the primary place to distribute data science articles and tutorials, and I hope you’ll enjoy it. I did a Google search using Medium.com as a filter, and over the past six months TensorFlow and Keras have had similar numbers of articles published, while PyTorch has had slightly fewer.
Keras and Fastai, as high-level apis, are popular among practitioners of the new frameworks, and there are many tutorials showing how to use them well.
New articles on arXiv
ArXiv is the online repository where most of the current deep learning academic papers are published. I used Google to search for frameworks mentioned in new papers published over the past six months, and TensorFlow came out ahead.
New activity on GitHub
Git’s recent activity is another indicator of framework popularity. The chart below shows the number of likes, clones, followers, and contributors.
TensorFlow is the leader in all options. However, PyTorch is close to TensorFlow in terms of followers and contributors, and Fastai has many new contributors.
No doubt some Keras contributors are still digging into the TensorFlow library. It’s also worth noting that both open source projects were started by Google employees.
The number of new Quora followers
I also included a reference metric I had previously missed — the number of people following new topics on Quora.
Over the past six months, the vast majority of fans of new topics have been around TensorFlow, with PyTorch and Keras lagging far behind.
Once I have all the data, I will organize it into a new metric.
The calculation of the increment score
Here’s how I created the increment score:
-
Scale the scores of all metrics to between 0 and 1.
-
Subcategories such as online jobs and GitHub activity.
-
Weight each category according to the percentages below.
-
And just to make it easier to understand, let’s multiply all of our scores by 100.
-
The score of the corresponding classification of each frame is accumulated to get the final increase score.
Online work accounts for a third of the total, simply because the immediate financial benefits are most tangible. This division is also intended to better balance the options. Unlike my other 2018 Power Score analysis, I did not include KDNuggets usage surveys (no new data) and publications (not many new books in the last 6 months).
The results of
Here’s what the table looks like:
Here are the scores for each category:
Here’s the final score:
TensorFlow is currently the most in demand and fastest growing framework, and that’s not going to change anytime soon. PyTorch has also grown rapidly, as evidenced by the large number of jobs required for its use and demand. Of course, Keras has come a long way in the last six months. Finally, fastai’s growth is less than ideal, but it is the youngest of these frameworks. So both TensorFlow and PyTorch are frameworks worth learning about.
Study suggest
If you are going to learn TensorFlow, I suggest you start with Keras. Chollet’s Deep Learning with Python and Dan Becker’s DataCamp Course on Keras are two good introductory books. Using Keras as a high-level API in TensorFlow 2.0 can be called only through tf.keras. Here is a quick start tutorial for TensorFlow 2.0 written by Chollet.
If you are ready to learn PyTorch, I suggest starting with Fast. Ai’s MOOC article Practical Deep Learning for Coders, V3. You can learn the principles of deep learning and the foundations of Fastai and PyTorch.
So what is the future for TensorFlow and PyTorch?
Future directions
I’ve often heard PyTorch work better than TensorFlow. This is because PyTorch has a more stable API and is closer to Python usage. There is also a native model transformation tool, ONNX, that can be used to speed up reasoning. Also, Pytorch shares many of numpy’s commands, lowering the barrier to learning.
As Cassie Kozyrkov, Google’s chief ARTIFICIAL intelligence decision officer, explains, all of the improvements to TensorFlow 2.0 are about the user experience. TensorFlow now has a more intuitive API, more rational Keras integration, and more straightforward execution options. These changes, and the widespread use of TensorFlow, will help it remain popular for years to come.
In addition, TensorFlow recently announced an exciting plan to develop a version of TensorFlow for Swift. Swift was originally a programming language built by Apple. Swift has many advantages over Python in terms of execution and development speed. Fast. Ai will use the Swift version of TensorFLow in some of its advanced MOOCs, according to Jeremy Howard, co-founder of Fast. The language may not be ready for another year or two, but it will certainly improve on existing deep learning frameworks. As you can see, collaboration and cross-learning between languages and frameworks is taking place.
Advances in quantum computing will also affect deep learning frameworks. While a working quantum computer is still several years away, Google, IBM, Microsoft and others are already thinking about how to combine quantum computing with deep learning. Frameworks need to adapt to this new technology.
conclusion
You’ve seen the growth of TensorFLow and PyTorch. They both have good advanced apis: tf.keras and Fastai, which lower the bar for beginners. You’ve already seen the new features and future directions of the framework.
If you need to use the diagrams in this article, you can go to my Kaggle Kernel to get them.
I hope you find this comparison helpful. If it does help, share it with your favorite social media channels so that more people can benefit.
Thank you for browsing!
If you find any mistakes in your translation or other areas that need to be improved, you are welcome to the Nuggets Translation Program to revise and PR your translation, and you can also get the corresponding reward points. The permanent link to this article at the beginning of this article is the MarkDown link to this article on GitHub.
The Nuggets Translation Project is a community that translates quality Internet technical articles from English sharing articles on nuggets. The content covers Android, iOS, front-end, back-end, blockchain, products, design, artificial intelligence and other fields. If you want to see more high-quality translation, please continue to pay attention to the Translation plan of Digging Gold, the official Weibo, Zhihu column.