Heart of Machine, by Qiu Lulu.

In mid-August, Google Brain member Martin Wicke announced in a public email that a new version of the open source framework, TensorFlow 2.0 preview, would be released before the end of the year. Today, at Google’s Developer conference in Shanghai, Heart of the Machine learned exclusively about a major change that will change Eager Execution to TensorFlow’s default Execution mode. This means that TensorFlow, like PyTorch, has moved from writing static diagrams to writing dynamic diagrams.


Google Developers Conference

On the second day of Google’s Developer conference, TensorFlow will be featured throughout the main session.

Feng Yifei, a software engineer from Google Brain, shared the new dynamics of TensorFlow programming interface, mainly introduced tf.Keras, tf.Data and other high-level libraries, and summarized a series of suggestions for developers to use TensorFlow. Include:

  • Build the prototype with Eager mode

  • Use Datasets to process data

  • Feature Columns are used to extract features

  • Build the model with Keras

  • Use Canned Estimators

  • Package the model with SavedModel

Eager mode will become the default mode in TensorFlow 2.0, allowing developers to build prototypes more easily and efficiently.

In the conversation after her talk, Feng mentioned that the beta version of TensorFlow 2.0 will be released by the end of this year, and the official version is expected to be released in Q1 or Q2 next year. Once the Eager mode is set to the default, an AutoGraph model can be automatically converted into an AutoGraph map once the prototype is ready. An AutoGraph map can also be optimized further, or build your own by turning off Eager mode.

We note that AutoGraph, released just two months ago, has left TF.contrib to become part of the official TF library, and in the design document, the engineer mentions, “In preparation for TF 2.0, We moved AutoGraph from tensorflow/contrib/to Tensorflow/Python/AutoGraph. AutoGraph can still be in tensorflow. Contrib. AutoGraph access, until tensorflow contrib cancelled.”

Google AI software engineer Anna Kim, a familiar face of Google’s Developer conference, also appeared at this year’s keynote, She mentioned that TensorFlow engineers post their latest design proposals on the TensorFlow Community’s Request for Comments, and she encouraged developers to browse and give their opinions on the latest design ideas of the engineers.

Martin Wicke’s proposal for sunset Tf. contrib, for example, is still in the feedback phase, with a number of developers contributing their own ideas under the pull Request.

The RFC address is as follows:

Github.com/tensorflow/…

In this afternoon’s talk, more engineers from Google will share design ideas and details related to Eager mode.


TensorFlow course

TensorFlow was developed by the Google Brain team based on DistBelief, Google’s first internal DL system, a general-purpose computing framework that has become the most popular open source tool for machine learning.

DistBelief, TensorFlow’s predecessor, was an internal DL tool developed by Google in 2011. Its Inception Network, based on DistBelief, won the ImageNet Challenge in 2014. Although DistBelief was already being used in a wide range of products within Google, it relied too heavily on Google’s internal architecture to open source. After refining DistBelief, Google officially released TensorFlow 0.5.0, an open source computing framework, in November 2015. Compared with DistBelief, TensorFlow’s computing framework is more general, its computing resources are arranged more rationally, and it supports more deep learning algorithms and platforms.

Tf-based projects prolifed in the first year after TensorFlow became open source: more than 480 people contributed directly to TF, including Google developers, outside researchers, independent developers, students, and senior developers from other companies. At the time, TensorFlow was already the most popular machine learning project on GitHub.

In its first year as open source, TensorFlow has added support for distributed training, iOS, raspberry PI development boards, and integration with the widely-used big data architecture. In addition, Google released the best-performing image classification model at the time, Inception- Resnet-v2, and answered thousands of questions on GitHub, StackOverflow, and TensorFlow mailing lists.

Google officially announced TensorFlow 1.0 at the first TensorFlow developer conference last February. In terms of speed, it achieves a 58-fold increase in distributed training on 64 Gpus for Inception V3. In terms of flexibility, TensorFlow 1.0 introduced high-level apis, such as tf.Layers, tF.Metrics and TF.Losses modules, and formally integrated the KerAS library into TF through TF.Keras.

Since then, TensorFlow has released a number of important updates, including Eager Execution for dynamic graphs, TensorFlow Lite for mobile deep learning, tensorflow.js for JavaScript developers, And autotransform Python into TF graphs.

In TensorFlow 2.0 planning, switching to the default mode of Eager Execution can have a big impact on developers, since we no longer need to write a full static graph and open a Session to run it. Instead, like PyTorch, Eager Execution is a run-defined interface, which means we can call it on Python to compute directly. This approach is very intuitive, so you can expect that getting started with TensorFlow will be much easier in the future.

Here’s what Google Brain’s Martin Wicke said about TensorFlow 2.0 in a public email in mid-August:


Planning for TensorFlow 2.0

Since its open source release in 2015, TensorFlow has become the most widely used machine learning framework in the world, accessible to a wide range of users and use cases. Since then, TensorFlow has been updated with rapid advances in computing hardware, machine learning research and commercial deployment.

To reflect these rapid changes, Google developers are already working on the next version of TensorFlow. TensorFlow 2.0 will be an important milestone with a focus on ease of use. Here are some of the expectations users have for TensorFlow 2.0:

  • Eager Execution will be a core feature of 2.0. It better aligns user expectations of programming models with TensorFlow practices and should make TensorFlow easier to learn and apply.

  • Support for more platforms and languages, and improve compatibility and reciprocity among these components through standardization of exchange formats and API alignment.

  • Outdated apis will be removed and duplication reduced to avoid confusion for users.

According to the email, Google plans to release a preview version of TensorFlow 2.0 before the end of the year.


Compatibility and continuity

TensorFlow 2.0 provides an opportunity for error correction and corrections that are not allowed in semantic versioning.

To simplify the conversion, the TensorFlow team will create a conversion tool that will either update Python code to use a TensorFlow 2.0-compatible API, or issue a warning if the automatic conversion is not possible. Similar tools have played a huge role in the transition to 1.0.

Not all improvements are automatic. For example, TensorFlow discourages the use of apis without direct docking. In this case, we will provide a compatibility module (tensorFlow.pat.v1) that contains the full TensorFlow 1.x API, which will be maintained throughout the TensorFlow 2.x cycle.


Disk Compatibility

Google stated that they do not plan to make major changes to the SavedModels or GraphDef stores, and plan to include all current kernels in 2.0. However, the 2.0 changes mean that variable names in the original checkpoint have to be converted before they can be compatible with the new model.


tf.contrib

TensorFlow’s contrib module goes beyond what can be maintained and supported by a single repository. Larger projects are best maintained separately, and we will incubate smaller extensions with the TensorFlow main code. So after publishing TensorFlow2.0, we’ll stop publishing tf.contrib. Over the next few months, we will work with contrib’s respective owners to develop detailed migration plans, including how to advertise your TensorFlow extension in TensorFlow’s community pages and documentation. For each contrib module, we have the following options: a) integrate the project into TensorFlow; B) Move it to a separate repository; C) Remove it completely. This does mean that all tf.contrib projects will be deprecated, and today TensorFlow will stop adding new tf.contrib projects.