This is the fourth day of my participation in the August More text Challenge. For details, see: August More Text Challenge

About the definition and understanding of tensors, I will talk to you based on my own knowledge, personal knowledge is limited, for tensors only stay on the surface. Tensor is really complicated, and I’m just explaining tensors from the perspective of deep learning and why we need to introduce tensors in deep learning.

What is a Tensor

In deep learning frameworks, most of the data is in the format of something like a Tensor, and you might wonder what a Tensor is. Let me first give you the definition of a tensor, a quantity that is transformed in a different frame of reference according to certain rules, is a tensor. Tensor’s benefit is that the laws of physics don’t change with the change of reference frame, so in deep learning, Tensor and the data structure like Tensor can represent most things in the world, so deep learning uses Tensor as the basic data structure.

In deep learning, we need a kind of structured data to describe things. For example, the temperature of the weather, which has only a numerical magnitude, is a scalar, and a scalar can be thought of as a zero-dimensional tensor. In fact, the so-called scalar is a point in space. If we choose different coordinates, its value will be different. X ‘=f(x)x^{\prime} =f(x)x’ =f(x)

Velocity is a quantity that has both magnitude and direction, and we usually express it in terms of a vector, a vector can be decomposed into components in x and y, a vector can represent a linear combination of two two components, a vector is a 1-dimensional tensor, maybe because every component is 1-dimensional, so a vector is a tensor of order 1

Now let’s talk about matrices, a three dimensional matrix can be represented by a combination of the components xy, zy, yx,zx, zy,zx, if we have forces in a plane that are perpendicular to the plane and along the plane, then we need to represent them in terms of multi-dimensional tensors.

Environment set up

  • Anaconda must be installed first
  • Install Pytorch

Anaconda and Pytorch installation will not be covered here,

import torch
import numpy as np
Copy the code
# create 2 rows and 3 columns,
a = torch.randn(2.3)
Copy the code

Get the type of Tensor

There are two ways to determine the tensor type in PyTorch. One is to call the type method on the tensor object. The other is to use isInstance to determine the tensor type. Getting type information is not as detailed as using type or isinstance. It is recommended to use the tensor.type() method or isinstance to determine the type of tensor.

a.type(a)#'torch.FloatTensor'
Copy the code

Notice here that if you use type to print type A, you only supply a as a torch.Tensor, and you don’t supply a more detailed, accurate type. Note that a.type() is a method return

type(a)#torch.Tensor
Copy the code
isinstance(a,torch.FloatTensor)#True
Copy the code
isinstance(a,torch.cuda.DoubleTensor)#False
Copy the code

`

Zero dimensional Tensor

A 0-dimensional Tensor can be seen as a scalar, and a 0-dimensional Tensor in PyTorch has a shape that is usually zero. Call const_2’s shape to see the scalar const_2 which is of type torch.size ([]). In this case, the tensor of torch.Size([]) is a dimension type, not an empty tensor of [].

const_1 = torch.tensor(1.)
const_1 #tensor(1.)
Copy the code
const_2 = torch.tensor(1.3)
const_2# tensor (1.3000)
Copy the code
const_2.shape#torch.Size([])
Copy the code

You can see the type of const_2 by calling Shape and dim

len(const_2.shape) # 0
Copy the code
const_2.dim() # 0
Copy the code
const_2.size()#torch.Size([])
Copy the code

1 Dimensional Tensor

In PyTorch, a 1 dimensional Tensor is a vector, but in the neural net, where do we use vectors? In fact, every hidden layer is represented by weight and bias.

Create a tensor
vec_1 = torch.tensor([1.2])
vec_1 # tensor ([1.2000])
Copy the code
vec_2 = torch.tensor([1.2.1.3])
vec_2 # tensor ([1.2000, 1.3000])
Copy the code
vec_3 = torch.FloatTensor(1)
vec_3 #tensor([0.])
Copy the code
vec_5 = torch.FloatTensor(2)
vec_5 # tensor ([1.4569 e-19, 6.4069 e+02])
Copy the code
data = np.ones(2)
data #array([1., 1.])
Copy the code
vec_6 = torch.from_numpy(data)
vec_6 #tensor([1., 1.], dtype=torch.float64)
Copy the code
# ones create 2
a = torch.ones(2)
a #tensor([1., 1.])
Copy the code
b = torch.Size([2])
b #torch.Size([2])
Copy the code

2-dimensional Tensor

In PyTorch, a 2-dimensional matrix is used to represent the image. In computer, we use a 2-dimensional matrix to represent the position of each pixel in the image, and then represent the color and transparency information of the pixel through a channel dimension. The mnIST data set is used as an example to illustrate the use of 2-dimensional matrix in neural network. The input image is 28×2828 \times 2828×28, and then the 2-dimensional image is flattened to 1 dimension 784. Then if the input image is longer at one time, = 32×78432 \times 78432×784

a = torch.rand(2.3)
a
Copy the code
Tensor ([[0.5607, 0.9660, 0.3631], [0.9655, 0.8591, 0.2379]])Copy the code
a.shape #torch.Size([2, 3])
Copy the code
a.size(0)# 2
Copy the code
a.size(1)# 3
Copy the code
a.shape[0] # 2
Copy the code

How do I use Tensor to represent text

With the 3-dimensional Tensor let’s just say, actually, in the RNN network, we use the 3-dimensional Tensor, and then let’s see, the input is the 3-dimensional Tensor, [10,20,100] and let’s see the 3-dimensional Tensor, 10 represents 10 words per sentence, 20 represents 20 sentences per batch, and 100 represents a word with a 100-dimensional vector

a = torch.rand(1.2.3)
a
Copy the code
Tensor ([[[0.0202, 0.9752, 0.4535], [0.7427, 0.5266, 0.2373]]])Copy the code
a.shape #torch.Size([1, 2, 3])
Copy the code
list(a.shape)# [1, 2, 3]
Copy the code
a[0]
Copy the code
Tensor ([[0.0202, 0.9752, 0.4535], [0.7427, 0.5266, 0.2373]])Copy the code
a[0] [0] # tensor ([0.0202, 0.9752, 0.4535])


Copy the code
a[0] [0] [0] # tensor (0.0202)
Copy the code

Tensor how to use an image

For example, we have two images to input into the network. For convolutional neural network, if we want to retain the image location information, the image location information is required to use two dimensions to represent the position of each pixel of the image, namely 28 and 28. Finally, 1 dimension is used to represent the number of channels of each pixel, for example, 3 channels can represent RGB. You can also use 1 for grayscale information.

a = torch.rand(2.3.28.28)
Copy the code
# a
a.shape #torch.Size([2, 3, 28, 28])
Copy the code

​

a.numel() # 4704
Copy the code