Introduction to TensorFlow

This guide gets you started programming in TensorFlow. Before using this guide, install TensorFlow. To get the most out of this guide, you should know the following:

  • How to program in Python.
  • At least a little bit of knowledge about arrays.
  • Ideally, something about machine learning. However, if you don’t know much about machine learning, this is still the first guide you should read.

TensorFlow provides several apis. The lowest level OF API – TensorFlow Core – gives you complete programming control. We recommend TensorFlow Core for machine learning researchers and others who need fine control of their models. The higher-level API is built on top of the TensorFlow core. These higher-level apis are generally easier to learn and use than TensorFlow Core. In addition, higher-level apis make repetitive tasks easier and more consistent across different users. Advanced apis like TF.Estimator can help you manage data sets, estimators, training, and reasoning.

This tutorial starts with the TensorFlow core tutorial. Later, we show how to implement the same model in TF.Estimator. Understanding the TensorFlow core principles will give you a good mental model of how internals work when you use more compact, higher-level apis.

tensor

Data center units in TensorFlow are tensors. A tensor consists of a set of primitive values that form any number of dimensions. The grade of a tensor is its dimension. Here are some examples of tensors:

3 # a rank 0 tensor; a scalar with shape [][1, 2., 3.]# a rank 1 tensor; a vector with shape [3][[1., 2., 3.], [4.# a rank 2 tensor; a matrix with shape [2, 3][[1., 2., 3.]], [[7.# a rank 3 tensor with shape [2, 1, 3]
Copy the code

TensorFlow core tutorial

Import tensor flow

The TensorFlow program’s specification import statement looks like this:

import tensorflow as tf
Copy the code

This gives Python access to all TensorFlow classes, methods, and symbols. Most documents assume that you have already done this.

Calculation chart

As you might imagine, the TensorFlow core program consists of two separate parts:

  • Build the computation diagram.
  • Run the calculation diagram.

A computational graph is a series of graph TensorFlow operations arranged as nodes. Let’s build a simple computational graph. Each node takes zero or more tensors as inputs and produces tensors as outputs. A node of one type is a constant. Like all TensorFlow constants, it takes no input, but outputs an internally stored value. We can create two floating point Tensors node1, node2 as shown below:

Node1 = tf.constant(3.0, dType =tf.float32) Node2 = tf.constant(4.0)# also tf.float32 implicitly
print(node1, node2)
Copy the code

The final print statement is generated

Tensor("Const:0", shape=(), dtype=float32) Tensor("Const_1:0", shape=(), dtype=float32)
Copy the code

Note that the print node does not output the value 3.0, 4.0 as you would expect. Instead, they are nodes that produce 3.0 and 4.0, respectively, at the time of evaluation. To actually evaluate the nodes, we must run the computed graph in the session. The session encapsulates the control and state of the TensorFlow runtime.

The following code creates a Session object and then calls its run method to run enough graphs to evaluate node1 and node2. By running the calculation in the session, the graph looks like this:

sess = tf.Session()
print(sess.run([node1, node2]))
Copy the code

We see expected values for 3.0 and 4.0:

[3.0, 4.0]
Copy the code

We can build more complex calculations by combining the Tensor nodes with operations (which are also nodes). For example, we can add our two constant nodes and generate a new graph like this:

from __future__ import print_function
node3 = tf.add(node1, node2)
print("node3:", node3)
print("sess.run(node3):", sess.run(node3))
Copy the code

The last two print statements are generated

node3: Tensor("Add:0", shape=(), dtype=float32) sess. Run (node3) : 7.0Copy the code

TensorFlow provides a utility called TensorBoard that displays a picture of a computed graph. Here is a screenshot showing how TensorBoard visualizes the graphics:

So far, this graph is not particularly interesting, because it always yields a constant result. Graphs can be parameterized to accept external input called placeholders. A placeholder is a promise followed by a value.

a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b  # + provides a shortcut for tf.add(a, b)
Copy the code

The first three lines are a bit like a function or lambda, where we define two input parameters (a and b) and then operate on them. We can evaluate this graph by using the feed_dict parameter of the run method to feed the placeholder with specific values from multiple inputs.

printSess.run (adder_node, {a: 3, b: 4.5})print(sess.run(adder_node, {a: [1, 3], b: [2, 4]}))
Copy the code

Lead to the output

7.5 [3.7.] In TensorBoard, the graph looks like this:

We can make the calculation diagram more complicated by adding another operation. For example,

add_and_triple = adder_node * 3.
printSess. run(add_and_triple, {a: 3, b: 4.5})Copy the code

To produce output

22.5 In TensorBoard, the above calculation diagram is as follows:

In machine learning, we usually need a model that can take arbitrary inputs, such as the one above. To make the model trainable, we need to be able to modify the graph to get a new output with the same input. Variables allow us to add trainable parameters to the graph. They are constructed as a type and an initial value:

W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W*x + b
Copy the code

Tf.constant is initialized when called, and its value never changes. By contrast, variables are not initialized when called tf.variable. To initialize all variables in the TensorFlow program, you must explicitly call a special operation like this:

init = tf.global_variables_initializer()
sess.run(init)
Copy the code

It is important to implement a handle to the initTensorFlow subgraph to initialize all global variables. Before we call sess.run, the variable is uninitialized.

Since x is a placeholder, we can evaluate several values of the linear_model simultaneously, as shown below:

print(sess.run(linear_model, {x: [1, 2, 3, 4]}))
Copy the code

To produce output

[0.0.30000001 0.60000002 0.90000004]Copy the code

We’ve created a model, but we don’t know how good it is. To evaluate the training data model, we need a Y placeholder to provide the required values, and we need to write a loss function.

The loss function measures how far the current model is from the data provided. We will use the standard loss model of linear regression, which adds the square of the triangle between the current model and the supplied data. Linear_model-y creates a vector where each element is an error increment for the corresponding example. We called TF.Square to fix the error. We then summarize all the squared errors to create a scalar that abstracts the error tf.reduce_sum from all the examples using the following method:

y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
Copy the code

Generating lost value

23.66
Copy the code

We can manually reassign the values of the W and b to raise the values of the -1 and 1 variables, perfectly initialized to provide the values of tf.variable, but can use operations such as tF.assign to change. For example, W=-1 and b=1 are the best parameters for our model. We can change W and B so:

fixW = tf.assign(W, [-1.])
fixb = tf.assign(b, [1.])
sess.run([fixW, fixb])
print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
Copy the code

The final print shows that the loss is now zero.

0.0
Copy the code

We guess the “perfect” values of W and B, but machine learning on the dot automatically finds the correct model parameters. We’ll show how to do this in the next section.

tf.train API
Copy the code

A full discussion of machine learning is beyond the scope of this tutorial. However, TensorFlow provides optimizers that can gradually change each variable to minimize the loss function. The simplest optimizer is gradient descent. It modifies each variable according to the magnitude of its loss derivative with respect to that variable. In general, calculating symbol derivation manually is tedious and error-prone. Thus, TensorFlow can automatically generate derivatives that describe the model tf.gradients only using this function. For simplicity, the optimizer will usually do this for you. For example,

Optimizer = tf. Train. GradientDescentOptimizer (0.01) "train" = optimizer. Minimize (loss)Copy the code
sess.run(init) # reset values to incorrect defaults.
for i in range(1000):
  sess.run(train, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]})
Copy the code
print(sess.run([W, b]))
Copy the code

Leading to the final model parameters:

[array (dtype = [0.9999969]float32), array (dtype = [0.99999082]float32)]
Copy the code

Now we’re done with machine learning! While this simple linear regression model does not require much core TensorFlow code, more complex models and methods need more code to feed data into the model. Therefore, TensorFlow provides a higher level of abstraction for common patterns, structures, and functions. We’ll learn how to use these abstractions in the next section.

The trainable linear regression model completed by the complete program is shown below:

import tensorflow as tf

# Model parameters
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W*x + b
y = tf.placeholder(tf.float32)

# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizerOptimizer = tf. Train. GradientDescentOptimizer (0.01) "train" = optimizer. Minimize (loss)# training data
x_train = [1, 2, 3, 4]
y_train = [0, -1, -2, -3]
# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
  sess.run(train, {x: x_train, y: y_train})

# evaluate training accuracy
curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))
Copy the code

Run time generation

W: [-0.9999969] B: [0.99999082] Loss: 5.69997E-11Copy the code

Note that the losses are very small numbers (very close to zero). If you run this program, your losses may not be exactly the same as those described above, because the model is initialized with pseudo-random values.

This more complex program can still be visualized in TensorBoard

tf.estimator
Copy the code

Tf.estimator is an advanced TensorFlow library that simplifies machine learning mechanisms and includes the following:

  • Running training cycle
  • Run the evaluation cycle
  • Management data set

Tf.estimator defines a number of common models.

Basic usage Note how much simpler the linear regression program is

# NumPy is often used to load, manipulate and preprocess data.
import numpy as np
import tensorflow as tf

# Declare list of features. We only have one numeric feature. There are many
# other types of columns that are more complicated and useful.
feature_columns = [tf.feature_column.numeric_column("x", shape=[1])]

# An estimator is the front end to invoke training (fitting) and evaluation
# (inference). There are many predefined types like linear regression,
# linear classification, and many neural network classifiers and regressors.
# The following code provides an estimator that does linear regression.
estimator = tf.estimator.LinearRegressor(feature_columns=feature_columns)

# TensorFlow provides many helper methods to read and set up data sets.
# Here we use two data sets: one for training and one for evaluation
# We have to tell the function how many batches
# of data (num_epochs) we want and how big each batch should be.x_train = np.array([1., 2., 3., 4.]) y_train = np.array([0., -1., -2., -3.]) x_eval = np.array([2., 5., 8., 1.]) y_eval = np. Array ([1.01, 4.1, 7, 0.]) input_fn = tf. The estimator. The inputs. Numpy_input_fn ({"x": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)
train_input_fn = tf.estimator.inputs.numpy_input_fn(
    {"x": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(
    {"x": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)

# We can invoke 1000 training steps by invoking the method and passing the
# training data set.
estimator.train(input_fn=input_fn, steps=1000)

# Here we evaluate how well our model did.
train_metrics = estimator.evaluate(input_fn=train_input_fn)
eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
print("train metrics: %r"% train_metrics)
print("eval metrics: %r"% eval_metrics)
Copy the code

When you run it, something similar happens

train metrics: {'average_loss': 1.4833182 e-08,'global_step': 1000, 'loss': 5.9332727 e-08}eval metrics: {'average_loss': 0.0025353201.'global_step': 1000, 'loss': 0.01014128}
Copy the code

Notice how our estimates have higher losses, but are still close to zero. It means we are learning correctly.

Custom model TF.Estimator does not lock you into a predefined model. Suppose we want to create a custom model that is not built into TensorFlow. We can still keep the high level abstraction tF.estimator for data sets, feeding, training, etc. To illustrate, we will show how to implement our own equivalent model to LinearRegressor using our knowledge of the low-level TensorFlow API.

To define an applicable custom model tf.estimator, we need to use tF.estimator.estimator. Tf) estimator) LinearRegressor is actually a subtype tf. The estimator. The estimator. Estimator We simply provide Estimator with a function model_fn to illustrate how TF.Estimator evaluates predictions, training steps, and losses, rather than classifications. The code is as follows:

import numpy as np
import tensorflow as tf

# Declare list of features, we only have one real-valued feature
def model_fn(features, labels, mode):
  # Build a linear model and predict values
  W = tf.get_variable("W", [1], dtype=tf.float64)
  b = tf.get_variable("b", [1], dtype=tf.float64)
  y = W*features['x'] + b
  # Loss sub-graph
  loss = tf.reduce_sum(tf.square(y - labels))
  # Training sub-graphGlobal_step = tf. Train. Get_global_step () optimizer = tf. Train. GradientDescentOptimizer "train" = (0.01) tf.group(optimizer.minimize(loss), tf.assign_add(global_step, 1))# EstimatorSpec connects subgraphs we built to the
  # appropriate functionality.
  return tf.estimator.EstimatorSpec(
      mode=mode,
      predictions=y,
      loss=loss,
      train_op=train)

estimator = tf.estimator.Estimator(model_fn=model_fn)
# define our data setsx_train = np.array([1., 2., 3., 4.]) y_train = np.array([0., -1., -2., -3.]) x_eval = np.array([2., 5., 8., 1.]) y_eval = np. Array ([1.01, 4.1, 7., 0.]) input_fn = tf. The estimator. The inputs. Numpy_input_fn ({"x": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)
train_input_fn = tf.estimator.inputs.numpy_input_fn(
    {"x": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(
    {"x": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)

# train
estimator.train(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did.
train_metrics = estimator.evaluate(input_fn=train_input_fn)
eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
print("train metrics: %r"% train_metrics)
print("eval metrics: %r"% eval_metrics)
Copy the code

Run time generation

train metrics: {'loss': 1.227995 e-11,'global_step': 1000}
eval metrics: {'loss': 0.01010036.'global_step': 1000}
Copy the code

Note that the contents of the custom model_fn() function are very similar to the manual model training loop of the underlying API.

The article is translated from the official English document. Please check my wechat official account and let’s study together.

Personal Website:

http://chaodongyang.com