What is a Pytorch?

  1. Pytorch can be used as an alternative to Numpy, doing scientific calculations and being more powerful.
  2. Do AI task modeling, training, deployment, etc. The only thing that comes close is Google’s Tensorflow.

This article is a very brief introduction to Pytorch, as follows:

  1. Tensor creation and manipulation, CUDA speed up calculation.
  2. Pytorch’s automatic differentiation function.

Tensor creates and operates

The creation of a Tensor

  1. Create an uninitialized empty tensor of the shape (5,3).

Input:

x = torch.empty(5, 3)
print(x)
Copy the code

Output:

Tensor ([[1.0102E-38, 9.0919E-39, 1.0102e-38], [8.9082e-39, 8.4489e-39, 9.6429e-39], [8.449090E-39, 9.6429e-39, 9.2755 e-39], [e-39 e-38 1.0286, 9.0919, 8.9082 e-39], [9.2755 e-39, 8.4490 e-39, 1.0194 e-38]])Copy the code





  1. Create a randomly initialized tensor of the shape 5,3.

Input:

x = torch.rand(5, 3)
print(x)
Copy the code

Output:

Tensor ([[0.4133, 0.0885, 0.0503], [0.6771, 0.5543, 0.8236], [0.3047, 0.1217, 0.4441], [0.6269, 0.6820, 0.4217]. [0.5631, 0.8517, 0.8708]])Copy the code





  1. Create a tensor with a 0 fill of the form 5,3.

Input:

x = torch.zeros(5, 3, dtype=torch.long)
print(x)
Copy the code

Output:

tensor([[0, 0, 0],
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]])
Copy the code





Create a tensor directly from the list:

X = Torch. Tensor ([5.5, 3]) print(x)Copy the code

Output:

Tensor ([5.5000, 3.0000])Copy the code


Create a full tensor from your existing tensor. By default, your new tensor will inherit your existing tensor’s properties like shapes, data types, etc. You can also specify that manually. :

x = x.new_ones(5, 3, dtype=torch.double)
print(x)
Copy the code

Output:

tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=torch.float64)
Copy the code


Generates a random tensor with the same shape as the existing tensor with the data type overwritten by torch. Float:

x = torch.randn_like(x, dtype=torch.float)
print(x)
Copy the code

Output:

Tensor ([[0.9629, 0.0349, 0.5597], [2.1172, 1.1874, 0.1596], [0.6841, 0.6172, 0.4732], [0.0468, 0.3634, 1.1014]. [0.6064, 0.1740, 0.2344]])Copy the code


  1. Look at the tensor shape

Input:

print(x.size())
Copy the code

Output:

torch.Size([5, 3])
Copy the code

Tensor. Size returns a tuple object that can perform operations on tuples.

The basic operations of tensor

Please refer to the official documentation for details

1. Create

  • Torch. Rand * Class randomly generated tensor methods.
torch.rand()
torch.rand_like()
torch.randn()
torch.randn_like()
torch.randint()
torch.randint_like()
torch.randperm()
torch.empty()
Copy the code
  • Created from other data sources
torch.tensor()
torch.from_numpy()
torch.full()
torch.range()
torch.linspace()
torch.eye()
Copy the code


2. Index, slice, merge and other conversion

The tensor is divided into several pieces of torch.chunk(). The tensor is divided into several pieces of torch.chunk(). The tensor is divided into three dimensions. Then you go to 3-dimensional tensor and then you put it together. Torch.dstack () # Combine multiple tensor at dimension 1 Torch.hstack () # index at dimension 1 torch.index_select() # Mask at dimension 1 at BoolTensor Torch. Masked_select () # Replace the tensor dimension torch. Movedim () # Return the index torch. Split () remove all the dimensions of the tensor that have size 1. You can also specify which dimension. Torch.squeeze () # combine multiple tensor torch.stack() # tensor transposes torch.t() # extract a new tensor from the existing tensor with the given index. Transpose () # insert insert into the tensor dimension at the required place Torch. Unsqueeze () # insert insert into the tensor dimension torch. You just need to put together torch. Vstack () # For each element in the tensor how to return torch. Where ()Copy the code

3. Set parameters of random sampling machine

Torch. Seed # Torch. Seed # torch. Torch. Get_rng_state # Set torch. Set_rng_state for the random number generatorCopy the code

4. Serialization and deserialization

Save and read torch. Save () torch. Load ()Copy the code

5. Parallel computing

Torch. Get_num_threads Torch. Set_num_threads Torch torch.set_num_interop_threadsCopy the code

6. Gradient control

There are multiple ways that you can control whether the tensor calculates the gradient

Input:

x = torch.zeros(1, requires_grad=True)
with torch.no_grad():
    y = x * 2
print(y.requires_grad)


is_train = False
with torch.set_grad_enabled(is_train):
    y = x * 2
print(y.requires_grad)


torch.set_grad_enabled(True)
y = x * 2
print(y.requires_grad)
Copy the code

Output:

False
False
True
Copy the code

7. Mathematical manipulation

Pytorch supports most of the common mathematical operations that are not detailed here, as shown in the official documentation

< div torch. Exp torch. Pow torch. Log torch Argmin torch. Max torch. Dist torch. Mean torch torch.allclose torch.argsort torch.eq torch.equal torch.ge torch.gt torch.isinf torch.isfinite torch.isnan torch.isreal Torch. Isneginf torch. Sort torchCopy the code





Pytorch’s automatic differentiation function

Pytorch has automatic differentiation.

  1. requires_grad

This parameter tells us whether this tensor needs to calculate the gradient.

x = torch.ones(2, 2, requires_grad=True)
print(x)
Copy the code

Output:

tensor([[1., 1.],
        [1., 1.]], requires_grad=True)
Copy the code


  1. grad_fn

That means what function does this tensor come from, and that’s what you’re going to do when you take the chain derivative. Generally speaking, except for the tensor created by the user, the tensor generated by pyTorch’s built-in functions will have grad_fn.

Input:

y = x + 2
print(y)
print(y.grad_fn)

z = y * y * 3
out = z.mean()
print(z, out)
Copy the code

Output:

tensor([[3., 3.],
        [3., 3.]], grad_fn=<AddBackward0>)
<AddBackward0 object at 0x000001CEE32753C8>
tensor([[27., 27.],
        [27., 27.]], grad_fn=<MulBackward0>) tensor(27., grad_fn=<MeanBackward0>)
Copy the code


  1. backward & grad

Backward method is used to calculate the gradient, and the gradient calculated by the chain derivative rule is in the tensor’s grad attribute.

Input:

out.backward()
print(x.grad)
Copy the code

Output:

Tensor ([[4.5000, 4.5000], [4.5000, 4.5000]])Copy the code


  1. with torch.no_grad

In this way, the gradient calculation in the calculation can be avoided

Input:

print(x.requires_grad)
print((x ** 2).requires_grad)
with torch.no_grad():
    print((x ** 2).requires_grad)
Copy the code

Output:

True
True
False
Copy the code


  1. detach

Detach method to avoid gradient calculation

Input:

print(x.requires_grad)
y = x.detach()
print(y.requires_grad)
print(x.eq(y).all())
Copy the code

Output:

True
False
tensor(True)
Copy the code


  1. requires_grad_

You can directly change the tensor gradient calculation configuration by setting the REQUIres_grad_ attribute.

Input:

a = torch.randn(2, 2)
a = ((a * 3) / (a - 1))
print(a.requires_grad)
a.requires_grad_(True)
print(a.requires_grad)
b = (a * a).sum()
print(b.grad_fn)
Copy the code

Output:

False
True
<SumBackward0 object at 0x000001CEE32753C8>
Copy the code





The end of the

Here you will learn how to create a tensor in Pytorch. You will also learn about the automatic differential mechanism of Pytorch. Kind of a primer.

In the next article, we’ll look at using Pytorch to construct datasets, build neural networks, train models, monitor logs, and more.

reference

  1. Pytorch.org/tutorials/b…
  2. Pytorch.org/docs/stable…
  3. Pytorch.org/tutorials/b…