Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.

Python OpenCV 365 day learning plan, enter the graphics realm with eraser. This blog is the 52nd in a series.

Learned in the previous

The Fourier Transform (FT) is the first time we’ve seen it today, and we’re going to apply it, and then we’re going to learn it iteratively.

The Fourier transform is a change in the view of the same thing, not in the time domain, but in the frequency domain. Time domain and frequency domain, just to make a little bit of sense, time domain, how things change over time, frequency domain refers to how often things change, is not very complicated, no problem, first use, first tune API.

I’m going to skip Fourier’s principle, and I’m going to talk about what happens to the picture when you apply it.

  • After using high-pass filter, high frequency information will be retained and image details will be enhanced, such as boundary enhancement.
  • After using the low pass filter, the low frequency information will be retained and the boundary is blurred.

Fourier transform application

Since the principle of first put down, that first apply it, we will respectively learn numpy and OpenCV two ways to achieve the Fourier transform.

Numpy implements the Fourier transform

The function used to realize the Fourier transform through Numpy is Np.fft.fft2, and the function prototype is as follows:

fft2(a, s=None, axes=(-2, -1), norm=None)
Copy the code

The parameters are described as follows:

  • a: Input image;
  • s: integer sequence, output array size;
  • axex: integer sequence used for calculationFFTOptional shaft;
  • norm: Normalized mode.

Some parameters if not used in practice, it is difficult to see the results, of course, the function description, or the official manual most reliable. The code writing process of application layer is as follows: Fourier transform of NP.fft. ffT2 function to obtain the frequency distribution, and then call nP.ffT. fftShift function to transfer the center position to the middle.

The test code is as follows:

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

img = cv.imread('test.jpg'.0)

# Fast Fourier transform algorithm to obtain the frequency distribution, the spatial domain into the frequency domain
f = np.fft.fft2(img)

The default center position is in the upper left corner. Use the following code to move the center to the middle position
# Move the low-frequency part to the center of the image
fshift = np.fft.fftshift(f)

# FFT results in a complex number, and its absolute value results in an amplitude
result = 20*np.log(np.abs(fshift))

plt.subplot(121)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(122)
plt.imshow(result, cmap='gray')
plt.title('result')
plt.axis('off')

plt.show()
Copy the code

This is a case repeatedly mentioned in many places, I would like to add relevant content. Np.fft. ffT2 is a frequency conversion function, its first parameter is the input image, that is, the gray image. The second parameter is optional and determines the size of the output array. If the size of the second parameter is greater than the size of the input image, the input image is filled with zero before FFT calculation; If less than the input image, the input image will be cropped. If no arguments are passed, the output array will be the same size as the input, tested as follows:

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

# Generate an image
src = np.ones((5.5), dtype=np.uint8)*100

print(src)
print(src.shape)

f = np.fft.fft2(src,(7.7))
print(f.shape)
print(f)
Copy the code

After the center point is moved, you can refer to the following operation results, and it can be known that the complex matrix is obtained after the application of the np.fft.fft2 function.

After the frequency transformation, the amplitude spectrum can be constructed. Finally, the range can be compressed by logarithmic transformation. Or it can be understood as converting the complex number into a floating point number for Fourier spectrum display, the supplementary code is as follows:

fimg = np.log(np.abs(fshift))
Copy the code

The final result is as follows. Of course, this is a random graph. If it is changed into a grayscale graph, it can be verified as follows.

The modified code looks like this:

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

# Generate an image
# src = np.ones((5, 5), dtype=np.uint8)*100
src = cv.imread("./test.jpg".0)
print("*"*100)
print(src)
print(src.shape)

f = np.fft.fft2(src)
print("*"*100)
print(f)

fshift = np.fft.fftshift(f)
print("*"*100)
print(fshift)
# Convert complex numbers to floating point numbers for Fourier spectrum display
fimg = 20*np.log(np.abs(fshift))
print(fimg)

# Image display
plt.subplot(121), plt.imshow(src, "gray"), plt.title('origin')
plt.axis('off')
plt.subplot(122), plt.imshow(fimg, "gray"), plt.title('Fourier')
plt.axis('off')
plt.show()
Copy the code

Basic conclusions: The original grayscale image is on the left, and the frequency distribution atlas is on the right. The closer it is to the center, the lower the frequency is. The higher the gray value is, the brighter the brightness is. And then you invert it, you go back, you do the inverse Fourier transform by numpy, which is the inverse of the Fourier transform, you convert the spectrum image to the original image. The core functions and prototypes used are as follows. np.fft.ifft2

# Implement inverse Fourier transform of image, return a complex array
np.fft.ifft2(a, s=None, axes=(-2, -1), norm=None)
Copy the code

np.fft.ifftshift

The inverse of the # fftshit() function, which moves the low-frequency center of the spectral image to the upper-left corner
np.fft.ifftshift(x, axes=None)
Copy the code

Based on the above content, reverse operation of the Fourier transform, the test code is as follows:

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

# Generate an image
# src = np.ones((5, 5), dtype=np.uint8)*100
src = cv.imread("./test.jpg".0)
print("*"*100)
print(src)
print(src.shape)

f = np.fft.fft2(src)
print("*"*100)
print(f)

fshift = np.fft.fftshift(f)
print("*"*100)
print(fshift)

# Convert complex numbers to floating point numbers for Fourier spectrum display
fimg = np.log(np.abs(fshift))
print(fimg)

The inverse Fourier transform
ifshift = np.fft.ifftshift(fshift)
# Convert complex numbers to floating point numbers for Fourier spectrum display
ifimg = np.log(np.abs(ifshift))
if_img = np.fft.ifft2(ifshift)

origin_img = np.abs(if_img)

# Image display
plt.subplot(221), plt.imshow(src, "gray"), plt.title('origin')
plt.axis('off')
plt.subplot(222), plt.imshow(fimg, "gray"), plt.title('fourier_img')
plt.axis('off')
plt.subplot(223), plt.imshow(origin_img, "gray"), plt.title('origin_img')
plt.axis('off')
plt.subplot(224), plt.imshow(ifimg, "gray"), plt.title('ifimg')
plt.axis('off')
plt.show()
Copy the code

The final result is as follows:

If a low pass filter is added to the spectral image after the above Fourier transform, the following results will be obtained.

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

# Low pass filtering of Fourier transform
src = cv.imread("./test.jpg".0)
f = np.fft.fft2(src)
fshift = np.fft.fftshift(f)

rows, cols = src.shape
crow, ccol = rows//2, cols//2

# Make the mask, the same size as the image, np.zeros initialization
mask = np.zeros((rows, cols), np.uint8)
# Set the center position, 30 up, down, left, right and left, to 1
mask[crow-30:crow+30, ccol-30:ccol+30] = 1

The mask is multiplied by the DFT result to preserve only the middle region
fshift = fshift*mask

The inverse Fourier transform
ifshift = np.fft.ifftshift(fshift)
# Convert complex numbers to floating point numbers for Fourier spectrum display
ifimg = np.fft.ifft2(ifshift)
dft_img = np.abs(ifimg)


# Image display
plt.subplot(121), plt.imshow(src, "gray"), plt.title('origin')
plt.axis('off')
plt.subplot(122), plt.imshow(dft_img, "gray"), plt.title('dft_img')
plt.axis('off')
plt.show()
Copy the code

If you want to implement high-pass filtering, you only need to modify the mask data.

mask = np.ones((rows, cols), np.uint8)
mask[crow-30:crow+30, ccol-30:ccol+30] = 0
Copy the code

I have referred to many articles, but one article has been mentioned repeatedly. Here is a note:

Zhuanlan.zhihu.com/p/19763358 Fourier transform in the content of the official manual, of course, also need time to read a little, the website address

OpenCV implements the Fourier transform

OpenCV to achieve the Fourier transform and NUMpy basically the same core differences in the use of the function, the specific differences are as follows: In Opencv, cv2.dif and cv2.idif (Inverse Fourier transform) are mainly used. Before input images, they need to be converted into converting images from NP. uint8 to NP. float32 format. The part of the obtained results with the frequency of 0 will be in the upper left corner. Cv2. dif returns a two-channel (real part, imaginary part) result that needs to be converted to an image format.

The function cv2.dft() has the following prototype

dst = cv.dft(src[, dst[, flags[, nonzeroRows]]])
Copy the code

Parameter Description:

  • src: Input image, requestnp.float32Format;
  • dst: Output image, dual channel (real + imaginary), size and type depend on the third parameterflags;
  • flags: indicates the conversion flag. The default value is 0.
  • nonzeroRows: The default value is 0.

Flags values are as follows:

  • DFT_INVERSE: Replace the default forward transform with one-dimensional or two-dimensional inverse transform;
  • DFT_SCALE: Scale identifier. Average the scaling result according to the number of data elements. If there are N elements, the output result is scaled by 1/NDFT_INVERSEUse in combination;
  • DFT_ROWS: Performs the forward or reverse Fourier transform on each row of the input matrix; This identifier can be used to reduce the overhead of resources when processing a variety of appropriate operations, often complex operations such as three-dimensional or high-dimensional transformations;
  • DFT_COMPLEX_OUTPUT: Perform forward transformation on one-dimensional or two-dimensional arrays of real numbers. Such a result is a complex array, but it has the conjugate symmetry (CCS) of complex numbers, and can be filled with a real array with the same size as the original array.This is the fastest choice and the default method of the function. You might want to get a full-size array of complex numbers (like simple spectral analysis, etc.) by setting the flag bit to make the function generate a full-size array of complex numbers;
  • DFT_REAL_OUTPUT: The inverse transformation of a one-dimensional, two-dimensional complex array usually results in a complex matrix of the same size, but if the input matrix has a complex conjugate symmetry (e.gDFT_COMPLEX_OUTPUTThe result of the positive transformation of the identifier), the real matrix is output.

The above content excerpt network, the original person has been unable to find, embarrassing.

To sum it up:

  • DFT_COMPLEX_OUTPUT: Get a complex matrix;
  • DFT_REAL_OUTPUT: outputs only the real part of the complex number;
  • DFT_INVERSE: Inverse Fourier transform;
  • DFT_SCALE: Whether to divide by MxN (MxN pixels in M rows and N columns);
  • DFT_ROWS: Enter the Fourier transform or inverse transform of each row of the matrix.

Finally, note that the output spectral result is a complex number, which requires a call to the cv2.Magnitude () function to convert the two-channel result of the Fourier transform into a range of 0 to 255.

This function is relatively simple and is prototyped and illustrated below.

Cv2. Magnitude (x, y)

  • X represents the floating-point X coordinate value, the real part
  • Y represents the floating point y coordinate value, the imaginary part

Basic knowledge of a simple, direct case explanation.

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif'] = ['SimHei']

src = cv.imread("test.jpg".0)

# OpneCV Fourier transform function
A float conversion is required for the image
result = cv.dft(np.float32(src), flags=cv.DFT_COMPLEX_OUTPUT)
# Move the low frequency of the spectrum from the upper left to the center position
dft_shift = np.fft.fftshift(result)
# Spectrum image double channel complex conversion to 0-255 range
result1 = 20 * np.log(cv.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))
# Image display
plt.subplot(121), plt.imshow(src, 'gray'), plt.title('Original image')
plt.axis('off')
plt.subplot(122), plt.imshow(result1, 'gray'), plt.title(Fourier transform.)
plt.axis('off')
plt.show()
Copy the code

The result is consistent with numpy.

To apply the above knowledge, use the functions in OpenCV to use low-pass filtering on the image.

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif'] = ['SimHei']

src = cv.imread("test.jpg".0)

# OpneCV Fourier transform function
A float conversion is required for the image
result = cv.dft(np.float32(src), flags=cv.DFT_COMPLEX_OUTPUT)
# Move the low frequency of the spectrum from the upper left to the center position
dft_shift = np.fft.fftshift(result)
# Spectrum image double channel complex conversion to 0-255 range
result = 20 * np.log(cv.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))

rows, cols = src.shape
crow, ccol = rows//2, cols//2

mask = np.zeros((rows, cols, 2), np.uint8)
mask[int(crow-30) :int(crow+30), int(ccol-30) :int(ccol+30)] = 1
# LPF (low pass filtering)
fshift = dft_shift*mask
f_ishift = np.fft.ifftshift(fshift)
img_back = cv.idft(f_ishift)
img_back = cv.magnitude(img_back[:, :, 0], img_back[:, :, 1])

# Image display
plt.subplot(131), plt.imshow(src, 'gray'), plt.title('Original image')
plt.axis('off')
plt.subplot(132), plt.imshow(result, 'gray'), plt.title(Fourier transform.)
plt.axis('off')
plt.subplot(133), plt.imshow(img_back, 'gray'), plt.title('Image after low pass filtering')
plt.axis('off')
plt.show()
Copy the code

High-pass filtering code and the effect of running, it is up to you to achieve.

Eraser section

HPF, LPF hopefully you found something in your hour today. See you in the next blog