This article has participated in the call for good writing activities, click to view: back end, big front end double track submission, 20,000 yuan prize pool waiting for you to challenge!

Python OpenCV 365 day learning plan, enter the graphics realm with eraser.

Foundation of basic knowledge

Learning image pyramid, found more information online, retrieval is easier.

An image pyramid is a multi-scale representation of an image, or can be understood as a display of different resolutions of an image.

The lower the pyramid is, the higher the pixel is, the higher the pixel is, the lower the resolution is.

Gauss pyramid

We still don’t explain the concept too much, the first time to learn the application, application, after all, 365 days cycle, long time, supplement theoretical knowledge later.

The Gaussian pyramid is used for downsampling and is the most basic image tower.

In the principle of Internet search, the simplest description is as follows:

Gaussian pyramid at the bottom of the image (0), such as gaussian kernel (5 x5) on the convolution operation, the convolution here mainly to get rid of is an even number of rows and columns, and then obtain a layer of the pyramid image (i.e., gaussian pyramid level 1), repeated convolution in the image, get level 2, executed repeatedly, You get the Gauss pyramid.

After each operation, the M×N image is transformed into an M/2 ×N /2 image, which is reduced by half.

Also found in the actual measurement, the need to use the same width and height of the image of the image, and the width and height of the power of 2, for example, 8 pixels, 16 pixels, 32 pixels and so on, later you can also test the actual.

The image pyramid is applied to functions such as cv2.pyrDown() and cv2.pyRUP ().

Cv2. pyrDown and cv2.pyrUp function prototype

The function prototype is obtained by the help function as follows:

pyrDown(src[, dst[, dstsize[, borderType]]]) -> dst
pyrUp(src[, dst[, dstsize[, borderType]]]) -> dst
Copy the code

The parameters of the two function prototypes are consistent, and are described as follows:

  • src: Input image;
  • dst: Output image;
  • dstsize: Output image size, default value according to((src.cols+1)/2, (src.rows+1)/2)Calculation.

Additional notes on two functions:

  • cv2.pyrDownA pyramid is constructed from a large image with relatively high resolution. The result after operation is that the image becomes smaller and the resolution decreases (downsampling).
  • cv2.pyrUpIt is an upsampling process, although the relative size increases, the resolution does not increase, the image will become more blurred.

The test code is as follows:

import cv2 as cv

src = cv.imread("./testimg.jpeg")
print(src.shape[:2])
cv.imshow("src", src)
# downsampling
dst = cv.pyrDown(src)
print(dst.shape[:2])
cv.imshow("dst", dst)

# sample down again
dst1 = cv.pyrDown(dst)
print(dst1.shape[:2])
cv.imshow("dst1", dst1)

cv.waitKey()
Copy the code

After running the code, you get three images, decreasing in size and resolution.

The minimum figure is obtained by the above operation, by performing the sampling up, image will be blurred, it also suggests that the sampling and sampling is under nonlinear processing, they are irreversible damage to processing, so the sample after the image is unable to restore, even if will also enlarge images blur (behind the study to the Laplacian pyramid can solve the problem).

# upward sampling
dst2 = cv.pyrUp(dst1)
print(dst2.shape[:2])
cv.imshow("dst2", dst2)
Copy the code

To summarize the steps of up-sampling and down-sampling:

  1. Upsampling: usecv2.pyrUpFunction, first enlarge the image twice as the original in each direction, fill the newly added rows and columns with 0, and then use the same kernel previously convolved with the enlarged image to obtain the approximate value of the newly added pixels;
  2. Downsampling: usecv2.pyrDownFunction, the image is first convolved with gaussian kernel, and then all even rows and columns are removed.

Laplacian Pyramid (LP)

Laplacian Pyramid is mainly used for image reconstruction. As we know from the above, up-sampling and down-sampling will cause the loss of image details when using Gaussian pyramid.

Laplace is designed to predict residual error when enlarges the image. What is residual error, that is, when enlarges a small image, some pixel values need to be inserted, and 0 is directly inserted above. Laplace pyramid algorithm can predict according to the surrounding pixels, so as to achieve the maximum restoration of the image.

The learning principle is as follows: Take each layer of the Gauss pyramid image, subtract the prediction image after sampling and Gaussian convolution of the image of the next layer, and get a series of difference images, which are LP decomposition images (LP is the Image of The Laplace pyramid).

There is also a formula (written for the first time in this series) for Laplace, where L is a Laplace pyramid image and G is a Gauss pyramid image


L n = G n – P y r U p ( P y r D o w n ( G n ) ) L_n = G_n-PyrUp(PyrDown(G_n))

Use the following code to test.

import cv2 as cv

src = cv.imread("./testimg.jpeg")
print(src.shape[:2])
cv.imshow("src", src)
# sample down once
dst = cv.pyrDown(src)
print(dst.shape[:2])
cv.imshow("dst", dst)

# sample up once
dst1 = cv.pyrUp(dst)
print(dst1.shape[:2])
cv.imshow("dst1", dst1)

# Calculate the Image of Laplacian pyramid
# Original image - the image sampled upward once
laplace = cv.subtract(src, dst1)
cv.imshow("laplace", laplace)
cv.waitKey()
Copy the code

The result is as follows, and the relevant image is rendered, focusing on the image on the far right.

Subtract (SRC, dst1) after cV.subtract (SRC, dst1) is used to set up the same subtract equation.

# cv.subtract(src, dst1)
laplace = src - dst1
Copy the code

The code runs like this.

In the process of learning, it is found that the image size had better be the whole power of 2, such as 256,512, etc., otherwise the image size will be different in the process of the pyramid upward, which will lead to errors in the Laplace pyramid processing due to the subtraction of different size matrices.

I found this to be true when I actually measured it. For example, the image used in the case will change its size when it is downsampled for 2 times. The test code is as follows:

import cv2 as cv

src = cv.imread("./testimg.jpeg")
print(src.shape[:2])
cv.imshow("src", src)
# Downsample once
dst1 = cv.pyrDown(src)
print(dst1.shape[:2])
cv.imshow("dst", dst1)
# Sample down 2 times
dst2 = cv.pyrDown(dst1)
print(dst1.shape[:2])
cv.imshow("dst2", dst2)

# Sample up once
up_dst1 = cv.pyrUp(dst2)
print(up_dst1.shape[:2])
cv.imshow("up_dst1", up_dst1)

# Calculate the Image of Laplacian pyramid
# Sample 1 - Sample the graph up once
laplace = dst1 - up_dst1
cv.imshow("laplace", laplace)
cv.waitKey()
Copy the code

Shape (up_dst1.shape[:2]) :

(355, 200) (355, 200) (356, 200)Copy the code

If you use the Laplacian image pyramid on this basis, the following error occurs

Sizes of input arguments do not match
Copy the code

In summing up the execution process of the Laplace image pyramid:

  • Downsampling: subtract I +1 from the I layer of the Gauss pyramid to get the image of the I layer of the Laplace;
  • Up-sampling: The original image of the i-layer is obtained by up-sampling the I +1 layer of the Gauss pyramid and adding the i-layer of Laplace.

The code above has been implemented, but the Laplace up sampling has not been implemented, to improve the code as follows, for code clarity, we change the variable name.

import cv2 as cv

src = cv.imread("./testimg_rect.jpeg")
print(src.shape[:2])
cv.imshow("src", src)
# Level 0 of the Gauss Pyramid
gus0 = src  # artwork
# Gauss pyramid level 1
gus1 = cv.pyrDown(gus0)
# Gauss layer 2
gus2 = cv.pyrDown(gus1)

# Layer 0 of the Laplace pyramid
lap0 = gus0 - cv.pyrUp(gus1)
# 1 Laplacian pyramid
lap1 = gus1 - cv.pyrUp(gus2)

# Display the first layer of Laplace code
cv.imshow("laplace", lap1)
cv.waitKey()
Copy the code

Now use the modified code to complete the operation of restoring the picture.

import cv2 as cv

src = cv.imread("./testimg_rect.jpeg")
print(src.shape[:2])
cv.imshow("src", src)
# Level 0 of the Gauss Pyramid
gus0 = src  # artwork
# Gauss pyramid level 1
gus1 = cv.pyrDown(gus0)
# Gauss layer 2
gus2 = cv.pyrDown(gus1)


# Layer 0 of the Laplace pyramid
lap0 = gus0 - cv.pyrUp(gus1)
# 1 Laplacian pyramid
lap1 = gus1 - cv.pyrUp(gus2)

rep = lap0 + cv.pyrUp(lap1 + cv.pyrUp(gus2))
gus_rep = cv.pyrUp(cv.pyrUp(gus2))

cv.imshow("rep", rep)
cv.imshow("gus_rep", gus_rep)
cv.waitKey()
Copy the code

The most important parts of the above code are the following:

rep = lap0 + cv.pyrUp(lap1 + cv.pyrUp(gus2))
gus_rep = cv.pyrUp(cv.pyrUp(gus2))
Copy the code

In the first line of code, LAP1 + CV. PyrUp (gus2) is the translation of the literal formula [the original image of the I layer is obtained by sampling up the I +1 layer of the Gauss pyramid and adding the I layer of Laplace].

The second line of code uses direct up-sampling to get an image of the loss detail.

The result of the above code run is the following, using Laplace to restore the image perfectly.

After studying this example, you can review the code from the beginning of this article and modify it.

Finally, I am learning a skill that can directly present two pictures. The code is as follows:

import cv2 as cv
import numpy as np
src = cv.imread("./testimg_rect.jpeg")
print(src.shape[:2])
cv.imshow("src", src)
# Downsample once
down_dst1 = cv.pyrDown(src)
print(down_dst1.shape[:2])
cv.imshow("dst", down_dst1)


# Sample up once
up_dst1 = cv.pyrUp(down_dst1)
print(up_dst1.shape[:2])
cv.imshow("up_dst1", up_dst1)

res = np.hstack((up_dst1, src))
cv.imshow('res', res)

cv.waitKey()
Copy the code

After running, the two image matrices are combined by the np.hstack((up_dST1, SRC)) function, and the results are as follows:

Eraser section

I hope you got something out of your hour today, and we’ll see you in the next blog

Blogger ID: Dream eraser, hope you like, comment, favorites.