The Tensorflow team has let it be known that Tensorflow 2.0 is coming soon, a major milestone release that focuses on simplicity and ease of use. I am looking forward to the arrival of Tensorflow 2.0, so I translated this document from the Tensorflow team: Effective Tensorflow 2.0: Best Practices and What’s Changed. The original address: https://medium.com/tensorflow/effective-tensorflow-2-0-best-practices-and-whats-changed-a0ca48767aff, a little cut. Click to read the original article to jump to the article, need to climb the wall oh!

In a recent article, we mentioned that TensorFlow 2.0 has been redesigned to focus on developer productivity, simplicity, and ease of use.

For an in-depth look at what’s changed and how best practices are applied, check out the new Effective TensorFlow 2.0 guide (posted on GitHub). This article gives a brief overview of the contents of that guide. If you are interested in these topics, please head over to the guide for more information!

Overview of major Changes

There are many changes in TensorFlow 2.0 that can improve user productivity, including removing redundant apis, making apis more consistent (unified RNN, unified optimizer), and better integration of Eager execution with the Python runtime.

Many RFCS (check them out if you’re new to them!) The changes and thinking involved in developing TensorFlow 2.0 have been explained. This guide shows what development should look like in TensorFlow 2.0. It is assumed that you have some familiarity with TensorFlow 1.x.

API to clean up

Many apis are missing or changing places in TF 2.0, and some are being replaced with equivalent 2.0 versions — TF.summary, TF.keras.metrics, and TF.keras.Optimizers. The easiest way to automatically replace with the new method is to use the V2 upgrade script.

Eager to perform

TensorFlow 1.x requires the user to manually splice together the abstract syntax tree (graph) by calling the TF. * API. The user then needs to manually compile the abstract syntax tree by passing a set of output and input tensors to the session.run() function call. In contrast, TensorFlow 2.0 executes immediately (as Python usually does), and in TF 2.0, graphics and sessions feel more like implementation details.

Reducing global variables

TensorFlow 1.x relies heavily on implicit global namespaces. When tf.variable () is called, it is put into the default graph, where it remains, even if the Python Variable pointing to it is forgotten. You can then restore the TF.variable, but only if you know the name under which it was created. This is difficult if variable creation is out of your control. As a result, various mechanisms proliferate to try to help users find their variables again.

TensorFlow 2.0 removes all of these mechanisms (Variables 2.0 RFC) and enables the default mechanism: Trace Variables! If you lose track of tF.variable, it will be garbage collected.

Functions, not sessions

The session.run() call is almost like a function call: you specify an input and a function to call, and then you return a set of output. In TensorFlow 2.0, you can use tf.function() to decorate Python Functions to mark them for JIT compilation, causing TensorFlow to run them as a single graph (Functions 2.0 RFC).

This mechanism allows TensorFlow 2.0 to reap all the benefits of graphical mode:

  • Performance: can optimize functions (node pruning, kernel fusion, etc.)
  • Portability: Functions can be exported/re-imported (SavedModel 2.0 RFC), allowing users to reuse and share modular TensorFlow functions.

With the ability to freely distribute Python and TensorFlow code, you can take full advantage of Python’s expressiveness. However, portable TensorFlow performs -move, C++, and JS without the Python interpreter context. To help users avoid rewriting code when adding @tf.function, * AutoGraph * will convert part of Python constructs into their TensorFlow equivalents.

TensorFlow 2.0 convention recommendations

Refactoring code into smaller functions

A common usage pattern in TensorFlow 1.x is the “sink” strategy, in which the set of all possible computations is prearranged and the selected tensor is then evaluated by session.run(). In TensorFlow 2.0, users should refactor their code into smaller functions that are called as needed. In general, it is not necessary to use tF.function to decorate these smaller functions, only to use tF.function to decorate high-level computations – for example, a step of training or forward passing of a model.

Use Keras layers and models to manage variables

Keras models and layers provide convenient variables and trainable_variables attributes that collect all associated variables recursively, making it easy to manage variables locally to where they are used.

The Keras layer/model inherits from tF.train.checkpointable and integrates with @tf.function, which makes it possible to obtain checkpoints directly or export savedModels from Keras objects. You don’t have to use the Keras’s.fit()API for these integrations.

Combine tf.data.datasets and @tf.function

Feel free to use regular Python iterations when iterating over training data loaded into memory. Otherwise, tF.data. Dataset is the best way to transfer training data from disk. Data sets are iterable (not iterators) and work like any other Python iteration in Eager mode. You can take advantage of the dataset asynchronous prefetch/streaming feature by wrapping your code in tf.function(), which replaces Python iterations with equivalent graphic operations using AutoGraph.

@tf.function
def train(model, dataset, optimizer):
 for x, y in dataset:
  with tf.GradientTape() as tape:
   prediction = model(x)
   loss = loss_fn(prediction, y)
  gradients = tape.gradients(loss, model.trainable_variables)
  optimizer.apply_gradients(gradients, model.trainable_variables)
Copy the code

If you use the keras.fit () API, you don’t need to worry about dataset iteration.

model.compile(optimizer=optimizer, loss=loss_fn)
model.fit(dataset)
Copy the code

Control flow using AutoGraph and Python

AutoGraph provides a way to convert data-dependent control flows into equivalent graphical patterns, such as TF.cond and TF.While_loop.

Data correlation control flows often occur in sequential models. Tf.keras.layers.rnn encapsulates RNN cells, allowing you to statically or dynamically expand the loop. For demonstration purposes, you can re-implement dynamic expansion as follows:

class DynamicRNN(tf.keras.Model):

  def __init__(self, rnn_cell):
    super(DynamicRNN, self).__init__(self)
    self.cell = rnn_cell
 
  def call(self, input_data):
    # [batch, time, features] -> [time, batch, features]
    input_data = tf.transpose(input_data, [1, 0, 2])
    outputs = tf.TensorArray(tf.float32, input_data.shape[0])
    state = self.cell.zero_state(input_data.shape[1], dtype=tf.float32)
    for i in tf.range(input_data.shape[0]):
      output, state = self.cell(input_data[i], state)
      outputs = outputs.write(i, output)
    return tf.transpose(outputs.stack(), [1, 0, 2]), state
Copy the code

Aggregate data using TF.metrics and log with tF.summary

Finally, a complete set of TF.summary notations will be available soon. You can access version 2.0 of TF.summary using the following command:

from tensorflow.python.ops import summary_ops_v2
Copy the code

The next step

This article provides a brief guide to Effective TF 2.0 (if you’re interested in these topics, go there to learn more!). For more information on TensorFlow 2.0, we also recommend these recent articles:

  • Contribute to TensorFlow: SIG, RFC, tests, and documentation
  • What are symbolic and imperative apis in TensorFlow 2.0?
  • Standardizing Keras: A guide to advanced apis in TensorFlow 2.0