My friends, for reprint please indicate the source: blog.csdn.net/jiangjunsho…

At tensorFlow1.x, the code defaults to graph execution. Starting with TensorFlow2.0, the code changes to eager execution. Eager execution of every step of the code will take place immediately. Graph Execution, on the other hand, combines all the code into a graph. The eager execution of an eager love story is like a one-night stand. Graph execution is like a marriage where you wait to get to know each other and not execute immediately after a long time of accumulation.

In eager mode, code is natural and easy to write, and debugging is easy because the code is executed immediately. In Graph mode, the code is more efficient. And because graph is really just a data structure made up of operation instructions and data, it can be easily exported, stored, and even later run in other non-Python environments (because graph is a data structure that defines operation instructions and data, So anywhere you can interpret the operations and data, you can run the model); And because graph is a data structure, different runtime environments can interpret the operations and data as they like. This way, the resulting code will be more consistent with the current runtime environment, which will make the code execute more efficiently.

For those of you who don’t understand, graph is a data structure… . Let me draw an analogy here to help you understand. Let’s say graph contains two data x and y, plus an operation instruction “add x and y.” When the C++ environment runs the graph, the “add x and y” operation is translated into the corresponding C++ code, and when the Java environment runs the graph, it is interpreted as the corresponding Java code. Graph is just a bunch of data and instructions, and how to execute those commands depends on the environment in which you’re running.

In addition to the above, Graph has a number of internal mechanisms to make code run more efficiently. In summary, Graph Execution enables tensorFlow models to run faster, more efficiently, more parallelistically, and better for different environments and devices.

Graph runs efficiently, but its code is not as simple as eager. In order to take into account the advantages of both modes, tF.function comes into being. Using tf.function, you can one-click wrap the eager code into a graph.

Since it is encapsulated as graph, why shouldn’t the word function in the name be tf.graph? The purpose of tf.function is to convert python function into tensorflow function with graph. So it makes sense to use the word function. The following code should help you understand better.

Import tensorflow as tf import timeit from datetime import datetime # Define a Python function. def a_regular_function(x, y, b): x = tf.matmul(x, Y) x = x + b return x # 'a_function_that_uses_a_graph' is a TensorFlow 'Function' Tf.function (a_regular_function) # define some tensorflow tensors. X1 = tf.constant([[1.0, 2.0]]) y1 = tf.constant([[2.0], [3.0]]) b1 = tf.constant(4.0) Orig_value = a_regular_function(x1, y1, b1).numpy() # Tenforflow Function can be called directly in Python. Just like using Python's own function. tf_function_value = a_function_that_uses_a_graph(x1, y1, b1).numpy() assert(orig_value == tf_function_value)Copy the code

Tf.function applies not only to top-level Python functions, but also to embedded Python functions. Take a look at the code below to see how it works.

def inner_function(x, y, b): X = tf.matmul(x, y) x = x + b return x # Turn 'outer_function' into a tensorflow 'function' using tf.function. Note that the previous code uses tf.function as a function, which is used as a modifier. Both approaches are supported. @tf.function def outer_function(x): Y = tf.constant([[2.0], [3.0]]) b = tf.constant(4.0) return inner_function(x, y, B) # tf.function builds a graph that contains not only 'outer_function' but also the 'inner_function' called within it. Outer_function (tf) constant ([[1.0, 2.0]]). Numpy ()Copy the code

Output result:

array([[12.]], dtype=float32)
Copy the code

If you’ve used TenforFlow 1.x before, you’ll notice that building graphs in 2.x no longer requires tF.session and Placeholder. Makes the code much simpler.

We often mix Python code with TensorFlow code in our code. In graph transformations using tF.function, tensorFlow code is transformed directly, while Python code is transformed by a library called AutoGraph.

The same Tensorflow function may generate different graphs. Because the input type of each tf.graph must be fixed, if a new data type is passed in when the tensorFlow function is called, the call will generate a new Graph. The input type and dimension are called signatures. Tensorflow function generates graphs based on signatures. New graphs are generated when new signatures are encountered. The following code should help you understand.

@tf.function def my_relu(x): return tf.maximum(0., x) # All three graphs are stored in the tenforflow function my_relu. Print (my_relu (tf) constant (5.5))) print (my_relu ([1, 1])) # python array print (my_relu (tf) constant ([3 - (3)]))) # tf arrayCopy the code

Output result:

Tf. Tensor (5.5, shape = (), dtype = float32) tf. The Tensor ([1. 0.], shape = (2), dtype = float32) tf. The Tensor ([3. 0.], shape = (2), dtype=float32)Copy the code

If the same input type is called, no new type is generated.

The following two calls will not generate a new graph.print (my_relu(tf.constant(-2.5))) # This data type is the same as' tf.constant(5.5) 'above. Print (my_relu (tf) constant ([- 1, 1]))) #. The data type and the above ` tf constant ([3 - (3)]) `.Copy the code

Because a Tensorflow function can contain multiple graphs, it can be said that tensorflow function is polymorphic. This polymorphism enables the Tensorflow function to arbitrarily support different input types, which is very flexible. And since a specific graph is generated for each input type, this also makes code execution more efficient!

The following code prints out three different signatures

print(my_relu.pretty_printed_concrete_signatures())
Copy the code

Output result:

my_relu(x)
  Args:
    x: float32 Tensor, shape=()
  Returns:
    float32 Tensor, shape=()

my_relu(x=[1, -1])
  Returns:
    float32 Tensor, shape=(2,)

my_relu(x)
  Args:
    x: float32 Tensor, shape=(2,)
  Returns:
    float32 Tensor, shape=(2,)
Copy the code

Above you learned how to convert a Python function to a TenforFlow function using tf.function. But there is more to learn to use tF.function correctly in real development. I’m going to take you through them. Boys of the 88th Division, don’t hold back. Follow me!

By default, the code in the TenforFlow function is executed in Graph mode, but it can also be executed in eager mode. Look at the code below.

@tf.function def get_MSE(): print("Calculating MSE!" ) # let tf.config. run_functions_wait (True) get_MSE(y_true, This code is used to cancel the previous setting tf.config. run_functions_15 (False)Copy the code

In some cases, the same Tensorflow function may run differently in Graph and eager modes. Python’s print function is one such special case. Look at the code below.

@tf.function def get_MSE(y_true, y_pred): print("Calculating MSE!" ) sq_diff = tf.pow(y_true - y_pred, 2) return tf.reduce_mean(sq_diff) y_true = tf.random.uniform([5], maxval=10, dtype=tf.int32) y_pred = tf.random.uniform([5], maxval=10, dtype=tf.int32) error = get_MSE(y_true, y_pred) error = get_MSE(y_true, y_pred) error = get_MSE(y_true, y_pred)Copy the code

Output result:

Calculating MSE!
Copy the code

Were you surprised to see the output? Get_MSE is called three times, but the Python print function inside is executed only once. Why is that? Since the Python print function is only executed when the graph is created, and the input arguments in the above three calls are of the same type, only one graph is created once, so the Python Print function is called only once.

To compare Graph with eager, let’s look at the output in eager mode.

# Enable forced eager mode tf.config. run_functions_wait (True) error = get_MSE(y_true, y_pred) error = get_MSE(y_true, Error = get_MSE(y_true, y_pred) # Cancel the eager mode tf.config. run_functions_wait (False)Copy the code

Output result:

Calculating MSE!
Calculating MSE!
Calculating MSE!
Copy the code

Look! In eager mode, print is executed three times. PS: If you use tf.print, then print three times in both graph and eager mode.

Another feature of graph Execution is that it does not execute useless code. Look at the code below.

Def unused_return_eager(x): # If the passed x contains only one element, the following code will report an error because the following code is the second element to fetch x. Tf. gather(x, [1]) # unused return x try: Print (unused_return_eager (tf) constant ([0.0]))) except tf. Errors. InvalidArgumentError as e: print (f '{type (e) __name__} : {e}')Copy the code

The above code is run in eager mode, so each line of code is executed, so the above exception occurs and is caught. The following code, which runs in Graph mode, does not raise an exception. Since tf.Gather (x, [1]) doesn’t actually do anything (it just fetches the second element of X without assigning or changing any variables), it’s not executed at all in Graph mode, so no exceptions are raised.

@tf.function def unused_return_graph(x): tf.gather(x, [1]) return x try: Print (unused_return_eager (tf) constant ([0.0]))) except tf. Errors. InvalidArgumentError as e: print (f '{type (e) __name__} : {e}')Copy the code

Graph is more efficient than eager. We can use the following code to figure out how much more efficient graph mode is than eager mode.

x = tf.random.uniform(shape=[10, 10], minval=-1, maxval=2, dtype=tf.dtypes.int32)

def power(x, y):
  result = tf.eye(10, dtype=tf.dtypes.int32)
  for _ in range(y):
    result = tf.matmul(x, result)
  return result
print("Eager execution:", timeit.timeit(lambda: power(x, 100), number=1000))
Copy the code

Output result:

Eager execution: 1.8983725069999764
Copy the code
power_as_graph = tf.function(power)
print("Graph execution:", timeit.timeit(lambda: power_as_graph(x, 100), number=1000))
Copy the code

Output result:

Graph execution: 0.5891194120000023
Copy the code

You can see from the code above that Graph takes nearly three times less time to execute than Eager. Of course, because the specific calculation content is different, the efficiency improvement degree is also different.

Graph improves performance, but there is a cost to converting graph. For some code, it may take longer to convert graph than to run graph. So try to avoid repeating graph transformations when you write your code. If you find that the model is inefficient, check to see if there are any duplicate transformations. You can tell if there is a repeat transformation by adding the print function (remember that print is called every time graph is converted). Look at the code below.

@tf.function def a_function_with_python_side_effect(x): print("Tracing!" ) # An eager-only side effect. return x * x + tf.constant(2) print(a_function_with_python_side_effect(tf.constant(2))) print(a_function_with_python_side_effect(tf.constant(3)))Copy the code

Output result:

Tracing!
tf.Tensor(6, shape=(), dtype=int32)
tf.Tensor(11, shape=(), dtype=int32)
Copy the code

Graph was converted once and print was called only once, because the parameter types were the same.

print(a_function_with_python_side_effect(2))
print(a_function_with_python_side_effect(3))
Copy the code

Output result:

Tracing!
tf.Tensor(6, shape=(), dtype=int32)
Tracing!
tf.Tensor(11, shape=(), dtype=int32)
Copy the code

Print is called twice above. Ah? Why is that? Print print print print print print print print print print print print print print print print print print Because the input parameter is a Python type, a new graph is created each time for a new Python type. Therefore, it is better to use the tenforFlow data type as the input parameter to function.

Finally, I give some suggestions related to TF.function:

  • When switching between eager and graph modes is required, tF.config.run_functions_gbit/s should be used for obvious annotation.

  • Tenforflow Variables (tf.variables) should be created outside of python function and their values should be modified inside. This advice also applies to other people using tf. The Variables of tenforflow objects (for example keras. The layers, keras. Models, tf, optimizers).

  • Avoid internal functions that rely on externally defined Python variables.

  • Try to include as much computational code as possible in one tf.function rather than multiple tF.functions to maximize code execution efficiency.

  • It is best to use the tenforFlow data type as the input parameter to function.