Write up front: Keep up with The Times and start learning machine learning, with the mindset of trying to see what all the directions look like before graduation, and find it a promising and interesting field (Keng). // Then it’s time to fill in the math…

0 TensorFlow introduction

Just the introduction of small white, understanding is not deep, direct to see the official introduction

GitHub Description: Computation using data flow graphs for scalable machine learning

website: Is an open source software library that uses data flow diagrams for numerical calculations. The nodes in the graph represent mathematical operations, and the edges in the graph represent multi-dimensional arrays (tensors) passed between the nodes.

0.1 What is TensorFlow?

It means tensor,It means flow. Therefore, it can be directly viewed as an open source library for computing data flow diagrams composed of tensors.

0.2 What can TensorFlow do?

At present, it is mainly used for machine learning, which is a bit unapproachable. The author understands that data can be converted into vector description and the corresponding computational flow diagram can be used. Let me give you an example, though it would be inappropriate not to know. Let’s say we’re calculating, you can build a binary tree

The ordered traversal of this binary tree is the expression above, which means that this expression can be transformed into a graph that looks like a binary tree, andI can calculate this graph. Here’s the code. It doesn’t matter if you don’t understand it, as long as you understand that the flow of the code is the calculation of a graph (binary tree). How do you use it in the next chapter.

# coding: utf-8
import tensorflow as tf

a, b, c, d = tf.constant(1), tf.constant(2), tf.constant(3),tf.constant(4)
add = tf.add(a,b)
mul = tf.multiply(add, c)
sub = tf.subtract(mul, d)
with tf.Session() as sess:
    print(sess.run(sub))
# output: 
# 5
Copy the code

0.3 TensorFlow installation

I won’t go into the details here, but I believe you should have a running environment by clicking on this article. If not here recommend two websitesEnglish: official website 和 English :** College translationThen introduce my environment:

Pay attention toSet toTo run. If not, you can read more online tutorials to learn more about the virtual environment.

1 first TensorFlow

Well, with the previous introduction, you should have been able to useSo let’s start with how to code.

1.1 Basic Grammar

In fact, grammar is not accurate. Grammar is(in this case, Python), mainly introduces calling the computation library to implement this particular computation. On the sameWebsite tutorial

1.1.1 Cell Description

As you can see in the calculation diagram, the two main things are points (leaf and non-leaf nodes) and lines. My understanding is that dots represent data and lines represent operations. I don’t know if that’s right, but here we go. Here are some common “dots” :

constant

c = tf.constant(2)
Copy the code

variable

v = tf.Variable(2)
Copy the code

A placeholder

p = tf.placeholder(tf.float32)
Copy the code

The above code is the least able to use the principle of passable parameters, interested can go to see the source code, here is mainly toSyntactic pull-up is used first and then you dig into why it is necessary to design such a data structure to compute graphs.

Here are the “lines” :

arithmetic

add = tf.add(a, b)
sub = tf.subtract(a, b)
mul = tf.multiply(a, b)
div = tf.divide(a, b)
Copy the code

Other no longer introduced, details can seeThe source code. For example, the above operation is defined in.

1.1.2 Calculation Process

After knowing the common data and calculation methods, the following calculation process is introduced:

# coding: utf-8
import tensorflow as tf

# Step1: Create data
a, b, c, d = tf.constant(1), tf.constant(2), tf.constant(3),tf.constant(4)

# Step2: Construct the calculation diagram
add = tf.add(a,b)
mul = tf.multiply(add, c)
sub = tf.subtract(mul, d)

# Step3: calculate
with tf.Session() as sess:
    print(sess.run(sub))
Copy the code

The example above is a standard constant calculation procedure, you can tryprint(a, add)Look at what you’ve created, and you’ll find that it’s aAnd the value inside is 0. You can guess, it’s just printing, it’s not counting. LookSource:

# ops.py
def __repr__(self):
    return "<tf.Tensor '%s' shape=%s dtype=%s>" % (self.name, self.get_shape(),self._dtype.name)

@property
def name(self):
    """The string name of this tensor."""
    if not self._op.name:
        raise ValueError("Operation was not named: %s" % self._op)
    return "%s:%d" % (self._op.name, self._value_index)
Copy the code

If you’ve learned to calculate constants, can you do the same for variables? If you try it, you’ll know that it’s not the same, that the variable needs to be initialized.

v = tf.Variable(2)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(v, sess.run(v))
Copy the code

At this point you might be wondering, what’s the difference between a variable and a constant? We know that variables should be mutable, so that we can adjust the parameters at any time during the calculation. Here is a code to show how to use it.

v = tf.Variable(2)
# Multiply v by 2
update = tf.assign(v, tf.multiply(v, tf.constant(2)))
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for _ in range(4):
        print("-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --")
        print "Before : ", sess.run(v)
        sess.run(update)
        print "After : ", sess.run(v)

# output:
# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# Before : 2
# After : 4
# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# Before : 4
# After : 8
# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# Before : 8
# After : 16
# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# Before : 16
# After : 32
Copy the code

But if we don’t want to set update-compute update-compute every time… You write the data directly to the computation, and the placeholders come into play. Again, a small example.

c = tf.constant(2)
# Note the same type, in this case tf.int32
p = tf.placeholder(tf.int32)
mul = tf.multiply(c, p)
with tf.Session() as sess:
    # TMP = 2 is equivalent to the initial value of 2 in the previous example
    tmp = 2;
    for _ in range(4) :# Directly populate feed_dict
        tmp = sess.run(mul, feed_dict={p:tmp})
        print tmp

# output:
# 4
# 8
# 16
# 32
Copy the code

The calculation process is summarized as follows:

  • Create data: You can create constants, variables, and placeholders.
  • Build diagram: Build a diagram from the previous data.
  • Initialization: To initialize a variable.
  • Computation: You must start a Session to compute the graph

1.2 visual

Provides a visualization tool —To begin with, let’s introduce how to use it.

Here’s a visualization of the binary tree example above.

# coding: utf-8
import tensorflow as tf

a, b, c, d = tf.constant(1), tf.constant(2), tf.constant(3),tf.constant(4)
add = tf.add(a,b)
mul = tf.multiply(add, c)
sub = tf.subtract(mul, d)
with tf.Session() as sess:
    writer = tf.summary.FileWriter('./graphs', sess.graph)
    print(sess.run(sub))
writer.close()
Copy the code

Then use the command line to storegraphsIs executed in thetensorboard --logdir="./graphs"Command, and then open it in your browser as promptedhttp://localhost:6006If it shows up successfullyThe interface says success.

2. Machine learning using TensorFlow

This is kind of an introduction to machine learning. It’s probably not practical to talk directly about machine learning, and I’m in the learning phase, so let’s give some small examples of how machine learning works.

2.1 Linear regression

Here we use the most familiar linear regression to experience the process of machine learning:

2.1.1 Obtaining Data

It’s very simple, it’s a linear regression model, so we’re going to go ahead and write up some data to see if the prediction is consistent with what we think it is.

train_X = numpy.asarray([1.1.1.8.3.2.4.7.5.9.6.7])
train_Y = numpy.asarray([1.2.2.1.3.1.4.6.5.5.6.9])
Copy the code

2.1.2 Model construction

We use the form of placeholders for the calculation, when the operation can directly import data. In this case, because we are using linear regression, the target function is the shapeOf the form of phi. In other words, we use the points we give to fit a line that matches the distribution of those points.

X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

W = tf.Variable(1., name="weight")
b = tf.Variable(1., name="bias")

# linear model 
# activation = X*W + b
activation = tf.add(tf.multiply(X, W), b)
Copy the code

2.1.3 Parameter evaluation

We take the difference between the y-coordinate given at each point and the y-coordinate calculated by the linear modelThe sum of the squares of theAs a loss function, the gradient descent algorithm is used to minimize the sum in training, and the interest rate is selected. I’m not going to talk about the math here, but I’ll write about machine learning algorithms in a future article. Generally, it is a big difficulty to select loss function and update weight through some optimization means. If you want to know the principle, you need to learn a lot of basic mathematical knowledge (probability theory, linear algebra, calculus…). .

learning_rate = 0.01

cost = tf.reduce_sum(tf.pow(activation - Y, 2))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
Copy the code

2.1.4 Training data

In this case, the data is taken and fed to the input nodes in the graph, and then the model will optimize itself. The data can be iterated many times to make the fitting function better adapt to these data points.

training_epochs = 2000
display_step = 100

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for epoch in range(training_epochs):
        for (x, y) in zip(train_X, train_Y):
            sess.run(optimizer, feed_dict={X: x, Y: y})
        if epoch % display_step == 0:
            print("Epoch:".'%04d' % (epoch + 1), "cost="."{:.9f}".format(sess.run(cost, feed_dict={X: train_X, Y: train_Y})), "W=", sess.run(W), "b=",
                  sess.run(b))
    print("Optimization Finished!")
    print("cost=", sess.run(cost, feed_dict={X: train_X, Y: train_Y}), "W=", sess.run(W), "b=", sess.run(b))
Copy the code

2.1.5 visualization

You can just draw a two-dimensional graph and see what happens. For those unfamiliar, see the Matplotlib tutorial

    writer = tf.summary.FileWriter('./graphs', sess.graph)

    plt.scatter(train_X, train_Y, color='red', label='Original data')
    plt.plot(train_X, sess.run(W) * train_X + sess.run(b), color='blue', label='Fitted line')
    plt.show()
writer.close()
Copy the code

The two-dimensional graph:

2.1.6 summary

In fact, the whole process is easy to understand if you don’t dig into the principle, which is nothing more than providing data – selecting fitting function – building graph – selecting loss function – optimizing – training data (updating weight) – drawing a conclusion. This process is in line with what we expect from the basic idea of solving linear regression. Of course, I think this is just the beginning, to further, learning the necessary mathematical knowledge is the only way to machine learning.

Here’s how to get started with TensorFlow:

# coding: utf-8
from __future__ import print_function
import tensorflow as tf
import numpy
import matplotlib.pyplot as plt

train_X = numpy.asarray([1.1.1.8.3.2.4.7.5.9.6.7])
train_Y = numpy.asarray([1.2.2.1.3.1.4.6.5.5.6.9])

X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

W = tf.Variable(1., name="weight")
b = tf.Variable(1., name="bias")

activation = tf.add(tf.multiply(X, W), b)

learning_rate = 0.01

cost = tf.reduce_sum(tf.pow(activation - Y, 2))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

training_epochs = 2000
display_step = 100

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for epoch in range(training_epochs):
        for (x, y) in zip(train_X, train_Y):
            sess.run(optimizer, feed_dict={X: x, Y: y})
        if epoch % display_step == 0:
            print("Epoch:".'%04d' % (epoch + 1), "cost="."{:.9f}".format(sess.run(cost, feed_dict={X: train_X, Y: train_Y})), "W=", sess.run(W), "b=",
                  sess.run(b))
    print("Optimization Finished!")
    print("cost=", sess.run(cost, feed_dict={X: train_X, Y: train_Y}), "W=", sess.run(W), "b=", sess.run(b))
    
    writer = tf.summary.FileWriter('./graphs', sess.graph)

    plt.scatter(train_X, train_Y, color='red', label='Original data')
    plt.plot(train_X, sess.run(W) * train_X + sess.run(b), color='blue', label='Fitted line')
    plt.show()
writer.close()

# output:
# Epoch: 0001 cost= 0.785177052 W= 1.07263 b= -0.448403
# Epoch: 0101 cost= 0.440001398 W= 1.02555b = -0.0137608 (Epoch: 0101 cost= 0.440001398 W= 1.02555b = -0.0137608
# Epoch: 0201 cost= 0.437495589 W= 1.02078 b= 0.0176154
# Epoch: 0301 cost= 0.437433660 W= 1.02043 b= 0.0199056
# Epoch: 0401 cost= 0.437430561 W= 1.02041 b= 0.0200727
# Epoch: 0501 cost= 0.437429130 W= 1.0204 b= 0.0200851
# Epoch: 0601 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 0701 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 0801 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 0901 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 100,000,000 cost= 0.437429696 W= 1.0204b = 0.0200854
# Epoch: 1101 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1201 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1301 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1401 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1501 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1601 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1701 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1801 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Epoch: 1901 cost= 0.437429696 W= 1.0204 b= 0.0200854
# Optimization Finished!
# cost= 0.43743 W= 1.0204 b= 0.0200854
# You can see that the data becomes stable after about 500 iterations.
Copy the code

3 summary

In fact, this is just the beginning. There is much more to learn. More and more, I feel the importance of the foundation, not only the computer foundation, but also the mathematical foundation is equally important, especially with the trend of the Internet of Things in the future, it may be that the professional coding is becoming more and more downplayed and just used as a tool/skill for some professionals. Facing graduation right away, can only chew these things slowly…

4 Reference Materials

  • 0. Website
  • 1. English :** College translation
  • 2. The Matplotlib tutorial
  • 3. TensorFlow primer