Google’s Tensorflow and Facebook’s PyTorch have been popular deep learning frameworks in the community. So which framework is best for the deep learning project at hand? In this paper, the author gives his own suggestions on the function effects, advantages and disadvantages of the two frameworks as well as installation and version update.

From the builtin,
Translated by Vihar Kurama, Heart of the Machine, with participation by Wu Pan and Du Wei.


If you’re reading this, you’ve probably started your own deep learning journey. If you’re not familiar with the field, deep learning uses “artificial neural networks,” a special kind of brain-like architecture. The goal is to develop human-like computers that can solve real-world problems. To help develop these architectures, tech giants like Google, Facebook, and Uber have released several frameworks for Python deep learning environments, which make it easier to learn, build, and train different types of neural networks. This article will detail and compare two popular frameworks: TensorFlow and PyTorch.

directory

  • Google’s TensorFlow

  • Facebook PyTorch

  • What can we build with TensorFlow and PyTorch?

  • Contrast PyTorch with TensorFlow

  • Advantages and disadvantages of PyTorch and TensorFlow

  • PyTorch and TensorFlow installation, version, update

  • TensorFlow or PyTorch? My advice

Google’s TensorFlow

TensorFlow is an open source deep learning framework created by developers at Google and released in 2015. The official study is published in the paper TensorFlow: Large-scale Machine Learning on Heterogeneous Distributed Systems.

The paper address: http://download.tensorflow.org/paper/whitepaper2015.pdf

TensorFlow has been widely used by companies, enterprises, and startups to automate work tasks and develop new systems, and has been praised for its distributed training support, scalable production and deployment options, and support for multiple devices such as Android.

Facebook PyTorch

PyTorch is one of the newest deep learning frameworks, developed by the team at Facebook and opened source on GitHub in 2017. See the paper “Automatic Differentiation in PyTorch” for more information on its development.

The paper address: https://openreview.net/pdf?id=BJJsrmfCZ

PyTorch is growing in popularity because of its simplicity, ease of use, support for dynamic computed graphs, and efficient memory usage. More on that later.

What can we build with TensorFlow and PyTorch?

Neural networks were originally used to solve simple classification problems such as handwritten number recognition or camera recognition of car registration plates. But with recent frameworks and advances in Nvidia’s high performance graphics processing units (Gpus), we can train neural networks on terabytes of data and solve far more complex problems. One notable achievement is that convolutional neural networks implemented in TensorFlow and PyTorch have achieved current best performance on ImageNet. The trained model can be used in different applications, such as target detection, image semantic segmentation and so on.

Although neural network architectures can be implemented based on any framework, the results are not the same. There are a number of parameters in the training process that are tied to the framework. For example, if you are training a data set on PyTorch, you can use a GPU to enhance the training process since they run on CUDA (a C++ back end). TensorFlow can also use a GPU, but it uses its own built-in GPU acceleration. Therefore, depending on the framework you choose, the time to train your model will always vary.

TensorFlow top level project

Magenta: explore a machine learning will be used for creating tool in the process of open source projects: https://magenta.tensorflow.org/

Sonnet: This is a tensorflow-based software library for building complex neural networks: https://sonnet.dev/

Ludwig: this is a don’t need to write code can be training and testing of deep learning model kit: https://uber.github.io/ludwig/

PyTorch Top level project

CheXNet: use the depth study to analyze the chest X-ray, can realize the radiologist pneumonia monitoring level: https://stanfordmlgroup.github.io/projects/chexnet/

PYRO: This is a general-purpose probabilistic programming language (PPL) written in Python, supported by PyTorch at the back end: https://pyro.ai (https://pyro.ai/)

Horizon: A Platform for Applied Reinforcement Learning: https://horizonrl.com (https://horizonrl.com/)

These are just a few of the frameworks and projects built on TensorFlow and PyTorch. You can find more on TensorFlow and PyTorch’s GitHub and website.

Contrast PyTorch with TensorFlow

The key difference between PyTorch and TensorFlow is the way they execute code. Both frameworks work on the tensor, the underlying data type. You can think of tensors as multi-dimensional arrays like the one shown below.



Mechanism: dynamic graph definition and static graph definition

The TensorFlow framework consists of two core building blocks:

  • A software library for defining computational graphs and the runtime for executing those graphs on a variety of different hardware.

  • A computational graph with many advantages (which will be covered shortly).

A computational graph is an abstract way of describing a computation as a directed graph. A graph is a data structure consisting of nodes (vertices) and edges. It is a collection of vertices connected by pairs of directed edges.

When you run the code in TensorFlow, the graph is defined statically. All communication with the outside world is performed via TF.sessionObject and TF.placeholder, which are tensors that are replaced by external data at run time. For example, look at the following code snippet:

Here is a statically generated graph in TensorFlow before running the code. The core advantage of computational graphs is parallelization or dependency driving scheduling, which makes training faster and more efficient.

Similar to TensorFlow, PyTorch has two core modules:

  • On-demand and dynamic build of computational graphs

  • Autograd: Perform automatic differentiation of dynamic graphs

As you can see in the figure below, the graph changes and executes nodes as the execution progresses, without special session interfaces or placeholders. Overall, the framework is more tightly integrated with the Python language and feels more local most of the time. As a result, PyTorch is a more Pythonic framework, while TensorFlow feels like a completely new language.


Depending on the framework you’re using, there’s a lot of difference in software. TensorFlow provides a way to implement dynamic diagrams using the TensorFlow Fold library, and PyTorch’s dynamic diagrams are built in.

Distributed training

A major difference between PyTorch and TensorFlow is data parallelism. PyTorch optimizes performance by taking advantage of Python’s native support for asynchronous execution. With TensorFlow, you have to write code by hand and fine-tune each operation to run on a particular device to achieve distributed training. However, you can reproduce all of the functionality in PyTorch into TensorFlow, but that requires a lot of work. The following code snippet shows a simple example of implementing distributed training with PyTorch as a model:

visualization

TensorFlow has an advantage when it comes to visualizing the training process. Visualization helps developers track the training process and facilitate debugging. The visual library for TensorFlow is called TensorBoard. PyTorch developers use Visdom, but Visdom provides very simple and limited functionality, so TensorBoard is better at visualizing the training process.

TensorBoard features:

  • Track and visualize loss and accuracy metrics

  • Visual computation diagrams (operations and layers)

  • View histograms of weights, deviations, or other tensors over time

  • Display image, text, and audio data

  • Analyze the TensorFlow program

Visualize training in TensorBoard

The characteristics of Visdom

  • To deal with the callback

  • Draw diagrams and details

  • Management environment

Visualize training in Visdom

Production deployment

TensorFlow is the clear winner when it comes to deploying trained models into production. We can deploy the model in TensorFlow directly using TensorFlow serving, which is a framework using the REST Client API.

With PyTorch, production deployment was easier in the latest stable version 1.0, but it did not provide any framework for deploying models directly on the network. You must use Flask or Django as the back-end server. So, if performance is a concern, TensorFlow serving is probably a better choice.

Define a simple neural network with PyTorch and TensorFlow

Let’s compare how to declare a neural network in PyTorch and TensorFlow.

In PyTorch, the neural network is a class, and we can use the torch. Nn package to import the layers necessary to build the architecture. All layers are first declared in the __init__() method and then defined in the forward() method to traverse the input X across all layers of the network. Finally, we declare a variable model and assign it to the defined schema (model = NeuralNet()).


Keras has recently been incorporated into the TensorFlow library, a neural network framework that uses TensorFlow as a back end. From then on, the syntax of the declaration layer in TensorFlow is similar to that of Keras. First, we declare variables and assign them to the type of architecture we are going to declare, in this case a Sequential() architecture.

Next, we add layers directly in sequence using the model.add() method. The layer type can be imported from tf.layers, as shown in the following code snippet:


TensorFlow andPyTorch The advantages and disadvantages of

Both TensorFlow and PyTorch have their advantages and disadvantages.

TensorFlow benefits:


  • Simple built-in advanced API

  • Visual training using TensorBoard

  • Production deployment is easy with TensorFlow Serving

  • Easy mobile platform support

  • Open source

  • Good documentation and community support

Disadvantages of TensorFlow:

  • Static figure

  • Debug method

  • Difficult to modify quickly

The advantages of PyTorch

  • Python-like code

  • Dynamic figure

  • Easy and quick editing

  • Good documentation and community support

  • Open source

  • Many projects use PyTorch

Disadvantages of PyTorch:

  • Visualization requires a third party

  • Production deployment requires an API server

PyTorch and TensorFlow installation, version, update

Both PyTorch and TensorFlow have recently released new versions: PyTorch 1.0 (first stable release) and TensorFlow 2.0 (beta). Both versions have significant updates and new features that make the training process more efficient, fluid and powerful.
If you want to install the latest versions of these frameworks on your own machine, you can install them using source code build or via PIP.

PyTorch installation

MacOS and Linux 
pip3 install torch torchvisionCopy the code
Windows 
Pip3 install https://download.pytorch.org/whl/cu90/torch-1.1.0-cp36-cp36m-win_amd64.whlpip3 to install https://download.pytorch.org/whl/cu90/torchvision-0.3.0-cp36-cp36m-win_amd64.whlCopy the code
TensorFlow installation

MacOS, Linux and Windows

# Current stable release for CPU-onlypip install TensorFlow # install TensorFlow 2.0 Betapip install Tensorflow = = 2.0.0 - walkCopy the code
To check whether the installation is successful, follow these steps using a command prompt or terminal.

TensorFlow or PyTorch?My advice

TensorFlow is a very powerful and mature deep learning library with strong visualization capabilities and multiple options for advanced model development. It has production-oriented deployment options and supports mobile platforms. On the other hand, the PyTorch framework is young, has much stronger community mobilization, and is Python-friendly.
My advice is that if you want to develop and build AI-related products more quickly, TensorFlow is a good choice. PyTorch is recommended for research developers because it supports fast and dynamic training.


Original link:
https://builtin.com/data-science/pytorch-vs-tensorflow