Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

All that multiplication, and to be honest I haven’t really gone through it yet, so I’m just going to do it.

We start by declaring a vector and a two-dimensional matrix

import torch
vec = torch.arange(4)
mtx = torch.arange(12).reshape(4.3)
print(vec, mtx,sep='\n')
Copy the code

Output result:

>> 
tensor([0, 1, 2, 3])
tensor([[ 0,  1,  2],
        [ 3,  4,  5],
        [ 6,  7,  8],
        [ 9, 10, 11]])
Copy the code

According to the position *

This * is multiplied by position in PyTorch, and there is a broadcast mechanism.

import torch
vec = torch.arange(4)
mtx = torch.arange(12).reshape(4.3)
print(vec*vec)
print(mtx*mtx)
Copy the code
>>
tensor([0, 1, 4, 9])
tensor([[  0,   1,   4],
        [  9,  16,  25],
        [ 36,  49,  64],
        [ 81, 100, 121]])
Copy the code

But one thing to note is that while many places say that vectors are column vectors by default, there is no such thing in Pytorch for one-dimensional tensors. Even if you multiply a 3 by 4 tensor by a 4 by 1 tensor, the result should be a 3 by 1 tensor, but since it is a one-dimensional tensor, it will become the default 3 (not 1 by 3).

Print (MTX *vec) is the same as print(vec* MTX) in the executable state.

In the example above, an error is reported if print(MTX *vec) or print(vec* MTX) is executed. By default, when one-dimensional tensors and matrices perform * operations, the number of elements in the one-dimensional tensor must be the same as the number of columns in the two-dimensional matrix, otherwise the broadcast function fails.

You can also add a dimension by using reshap(). But there are some dimensional rules to follow when adding dimensions.

import torch
vec = torch.arange(4).reshape(4.1) # add dimensions
mtx = torch.arange(12).reshape(4.3)
print(vec*mtx)
print(mtx*vec)
Copy the code
>>
tensor([[ 0,  0,  0],
        [ 3,  4,  5],
        [12, 14, 16],
        [27, 30, 33]])
tensor([[ 0,  0,  0],
        [ 3,  4,  5],
        [12, 14, 16],
        [27, 30, 33]])
Copy the code

Let’s say the top matrix is 4 by 3.

The second line of code you can use

  • Vec = torch. Arange (4). Reshape (4, 1)
  • Vec = torch. Arange (3). Reshape (1, 3)

That means you have to have the same number of elements in your row or column.

The number by the torch. The mul

torch.mul(input, value, out=None)
Copy the code

Multiply each element of the input input by the scalar value value and return a new result tensor. It’s just a scalar multiplication of tensors.

import torch
vec = torch.arange(4)
mtx = torch.arange(12).reshape(3.4)
print(torch.mul(vec,2))
print(torch.mul(mtx,2))
Copy the code
>>
tensor([0, 2, 4, 6])
tensor([[ 0,  2,  4,  6],
        [ 8, 10, 12, 14],
        [16, 18, 20, 22]])
Copy the code

The matrix vector is multiplied by torch. Mv

Torch. Mv (Mat, vec, Out =None) → TensorCopy the code

Let’s multiply the matrix mat and the vector vec. If mat is an n×m tensor and vec is an m-dimensional tensor, an n-dimensional tensor will be output.

You have to have a matrix in front and a vector in the back, and the dimensions have to conform to matrix multiplication. What comes out is a one-dimensional tensor.

import torch
vec = torch.arange(4)
mtx = torch.arange(12).reshape(3.4)
print(torch.mv(mtx,vec))
Copy the code
>>
tensor([14, 38, 62])
Copy the code

Matrix multiplication torch. Mm

Torch. Mm (mat1, mat2, out=None) → TensorCopy the code

Let’s multiply the matrices MAT1 and mat2. If MAT1 is an n by m tensor and mat2 is an m by P tensor, it will output an n by P tensor out.

The matrix multiplication that we learned from line algebra, the dimensions have to correspond correctly.

import torch
mtx = torch.arange(12)
m1 = mtx.reshape(3.4)
m2 = mtx.reshape(4.3)
print(torch.mm(m1, m2))
Copy the code
>>
tensor([[ 42,  48,  54],
        [114, 136, 158],
        [186, 224, 262]])
Copy the code

Dot product of the torch. The dot

Torch. Dot (tensor1 tensor2) to floatCopy the code

Compute the dot product (inner product) of two tensors, both of which are one-dimensional vectors.

import torch
vec = torch.arange(4)
print(torch.dot(vec, vec))
Copy the code
>>
tensor(14)
Copy the code

Black technology @

There is also a black art, @, which also strictly requires that the number of columns in the first argument be equal to the number of rows in the second argument.

import torch
vec = torch.arange(4)
mtx = torch.arange(12)
m1 = mtx.reshape(4.3)
m2 = mtx.reshape(3.4)

print(vec @ vec)
print(vec @ m1)
print(m2 @ vec)
print(m1 @ m2)
Copy the code
>> tensor(14) tensor([42, 48, 54]) tensor([14, 38, 62]) tensor([[ 20, 23, 26, 29], [ 56, 68, 80, 92], [ 92, 113, 134, [128, 158, 188, 218]])Copy the code

The results above may not be intuitive, but take a look:

import torch
vec = torch.arange(4)
mtx = torch.arange(12)
m1 = mtx.reshape(4.3)
m2 = mtx.reshape(3.4)

print(vec @ vec==torch.dot(vec,vec))
print(vec @ m1) This sentence cannot be executed using torch. Mv () directly.
print(m2 @ vec==torch.mv(m2,vec))
print(m1 @ m2==torch.mm(m1,m2))
Copy the code

Use a single @ instead of the above three functions.

  • An at sign operation on a one-dimensional tensor is dot
  • Performing operations on one and two dimensional tensors is mv
  • The at sign operation on a two-dimensional tensor is mm
>>
tensor(True)
tensor([42, 48, 54])
tensor([True, True, True])
tensor([[True, True, True, True],
        [True, True, True, True],
        [True, True, True, True],
        [True, True, True, True]])
Copy the code

What about the second one that can’t be replaced? To satisfy ocD, do this:

import torch
vec = torch.arange(4)
mtx = torch.arange(12).reshape(4.3)

print(vec @ mtx) This sentence cannot be executed using torch. Mv () directly.
print(torch.mm(vec.reshape(1.4),mtx))

print(vec @ mtx==torch.mm(vec.reshape(1.4),mtx))
Copy the code
>>
tensor([42, 48, 54])
tensor([[42, 48, 54]])
tensor([[True, True, True]])
Copy the code