Participate in the 22nd day of The November Gwen Challenge, see the details of the event: 2021 last Gwen Challenge
Only see dry code certainly can’t understand what I am doing, suggest to cooperate with the theory to eat.
Hands-on learning deep learning 6.1 why need convolution layer | convolution formula – the nuggets (juejin. Cn)
Two-dimensional cross
import torch
from torch import nn
from d2l import torch as d2l
Copy the code# Implement two dimensional cross operation
def corr2d(x,k) :
h,w = k.shape
Y = torch.zeros((x.shape[0]-h+1,x.shape[1]-w+1))
for i in range(Y.shape[0) :for j in range(Y.shape[1]):
Y[i,j] = (x[i:i+h,j:j+w]*k).sum(a)return Y
Copy the codeThis formula is implemented:
# Verify our two dimensional crossover
X = torch.arange(40).reshape(5.8)
K = torch.ones(3.3)
K[:,1] = -1
print(X)
print(K)
Copy the code>>
tensor([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29, 30, 31],
[32, 33, 34, 35, 36, 37, 38, 39]])
tensor([[ 1., -1., 1.],
[ 1., -1., 1.],
[ 1., -1., 1.]])
Copy the codeUsing the same data as in the previous article, manually generate a test data for calculation. Take a look at the picture and the convolution kernel that we generated.
# Verify our two dimensional crossover
X = torch.arange(40).reshape(5.8)
K = torch.ones(3.3)
K[:,1] = -1
print(X)
print(K)
Copy the code>>
tensor([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29, 30, 31],
[32, 33, 34, 35, 36, 37, 38, 39]])
tensor([[ 1., -1., 1.],
[ 1., -1., 1.],
[ 1., -1., 1.]])
Copy the code>>
tensor([[27., 30., 33., 36., 39., 42.],
[51., 54., 57., 60., 63., 66.],
[75., 78., 81., 84., 87., 90.]])
Copy the codeVerify the results.
convolution
# Implement a convolution
class Conv2D(nn.Module) :
def __init__(self,kernel_size) :
super().__init__()
self.weight = nn.Parameter(torch.rand(kernel_size))
self.bias = nn.Parameter(torch.zeros(1))
def forward(self,x) :
return corr2d(x,self.weight) + self.bias
Copy the codeGenerate a test data:
X = torch.ones(5.8)
X[:,2:6] =0
print(X)
Copy the code>> tensor([[1., 1., 0., 0., 0., 0., 1., 1.], [1., 1., 0., 0., 0., 0., 1., 1.], [1., 1., 0., 0., 0., 0., 1., 1.], [1., . 1, 0, 0), 0. 0. 1, 1], [1, 1, 0., 0., 0., 0., 1, 1]])Copy the codeAfter output X looks like this, pretend it is a picture:
K = torch.tensor([[1, -1]])
# Detect vertical edge in image:
Y = corr2d(X,K)
print(Y)
Copy the code>> tensor([[ 0., 1., 0., 0., 0., -1., 0.], [ 0., 1., 0., 0., 0., -1., 0.], [ 0., 1., 0., 0., 0., -1., 0.], [ 0., 1., .. 0, 0, 0., - 1. 0.], [. 0. 1, 0, 0, and 0., - 1, 0.]])Copy the codeAnd if you look at the output you can see what this convolution kernel K is for, which is vertical edge detection.
It can be seen from the result that there was a change from the second column to the third column of the original graph, and a change from the penultimate column to the penultimate column.
Why is that?
Since the convolution kernel is 1*2, the second and penultimate columns of the computed result are generated when rolled to the two positions in the figure below. And these two positions are the second to third columns and the third to the last column of the original graph.
Now let’s test the convolution layer
Code 1
This is li Mu teacher book source code:
Use pyTorch's convolutional layer:
conv2d = nn.Conv2d(1.1,kernel_size=(1.2),bias=False)
X = X.reshape((1.1.5.8))
Y = Y.reshape((1.1.5.7))
epoch = 20
learning_rate = 0.03
for i in range(epoch):
Y_hat = conv2d(X)
l = ((Y_hat-Y)**2).sum()
conv2d.zero_grad()
l.backward()
conv2d.weight.data[:] -= learning_rate * conv2d.weight.grad
if (i+1) %5= =0:
print(f"batch {i+1},loss = {l.sum() :. 5f}")
print(conv2d.weight.data.reshape(1.2))
Copy the code>>
batch 5,loss = 0.30810
batch 10,loss = 0.00828
batch 15,loss = 0.00023
batch 20,loss = 0.00001
Copy the codeIt can be seen that our loss is already very low at this point.
The above code is implemented using PyTorch’s own two-dimensional convolution.
The 0 0 is 0 0 First our test data is 0 0 The first one represents a number of channels, the second one represents a lot, and the third and fourth dimensions are 0 0
And then you write a gradient descent.
print(conv2d.weight.data.reshape(1.2))
Copy the code>>
tensor([[ 0.9994, -0.9994]])
Copy the codeIf you look at the results after training, it’s pretty close to [1,-1].
Code 2
Use pyTorch's convolutional layer:
conv2d = nn.Conv2d(1.1,kernel_size=(1.2),bias=False)
X = X.reshape((1.1.5.8))
Y = Y.reshape((1.1.5.7))
loss = nn.MSELoss(reduction='sum')
trainer = torch.optim.SGD(conv2d.parameters(),learning_rate)
epoch = 20
learning_rate = 0.03
for i in range(epoch):
Y_hat = conv2d(X)
l = loss(Y_hat,Y)
trainer.zero_grad()
l.backward()
# conv2d.weight.data[:] -= learning_rate*conv2d.weight.grad
trainer.step()
if (i+1) %5= =0:
print(f"batch {i+1},loss = {l:. 5f}")
print(conv2d.weight.data.reshape(1.2))
Copy the codeSince we are using their convolution layer, there is no need to write gradient descent, just use their own.
Loss = nn.MSELoss(reduction=’sum’) The reduction parameter should be set to sum, because the default value of the reduction parameter is return mean. If you use mean, the learning rete should be changed to 0.8 to achieve the same effect in 20 steps.
Code 3
# Let's use our own convolutional layer
conv2d = Conv2D(kernel_size=(1.2))
epoch = 20
learning_rate = 0.03
for i in range(epoch):
Y_hat = conv2d(X)
l = (Y_hat-Y)**2
conv2d.zero_grad()
l.sum().backward()
conv2d.weight.data[:] -= learning_rate * conv2d.weight.grad
if (i+1) %5= =0:
print(f"batch {i+1},loss = {l.sum() :. 5f}")
print(conv2d.weight.data.reshape(1.2))
Copy the code>> Batch 5,loss = 0.66233 Batch 10,loss = 0.01583 Batch 15,loss = 0.00044 batch 20,loss = 0.00001 tensor([[0.9992, 0.9992]])Copy the codeOf course, just because we wrote our own convolution layer, we’re going to use it here. It’s actually faster to use someone else’s.
-
More on hands-on Deep Learning series can be seen here: Juejin. Cn
-
Github address: DeepLearningNotes/d2l(github.com)
Still in update …………