This is the 29th day of my participation in the August Wenwen Challenge.More challenges in August

Python OpenCV 365 day learning plan, enter the graphics realm with eraser.

Foundation of basic knowledge

Image gradient is a method to calculate the speed of image change. For the edge part of the image, if the gray value changes greatly, the corresponding gradient value is also large; conversely, for the smooth part of the image, the gray value changes little, and the corresponding gradient value changes little.

With the above content, you can learn image gradient correlation calculation, which will be used to obtain image edge information related technology.

OpenCV offers three different gradient filters, or high-pass filters: Sobel, Scharr, and Laplacian.

In mathematics, Sobel, Scharr is the first or second derivative, Scharr is the optimization of Sobel (when solving gradient angles using small convolution kernels), and Laplacian is the second derivative.

The mathematics part we still do not unfold, after the first general studies, from the perspective of God to supplement.

Sobel operator and Scharr operator

Sobel operator description and use

Sobel operator is a combination of Gaussian smoothing and differential operation, so its anti-noise ability is very good.

In the process of use, you can set the direction of derivation (xorder or yorder), and also set the size of the convolution kernel (ksize).

The function prototype is as follows:

dst = cv2.Sobel(src, ddepth, dx, dy[, dst[, ksize[, scale[, delta[, borderType]]]]])
Copy the code

The parameters are described as follows:

  • src: Input image;
  • ddepth: Image color depth. For different input images, output images have different depth.- 1The image depth is consistent;
  • dx: Derivative order in x direction;
  • dy: Derivative order in y direction;
  • ksize: kernel size, usually an odd number, 1,3,5,7;
  • scale: Zoom size, default is 1;
  • delta: Incremental value, default is 0;
  • borderType: Boundary type, default value isBORDER_DEFAULT.

Note: If ksize=-1, a 3×3 Scharr filter will be used, which is better than a 3×3 Sobel filter (and has the same speed, so Scharr filter should be used as much as possible when using 3×3 filters).

The test code for this function is as follows (note that you select your own image and place it in the same folder) :

import cv2
import numpy as np

img = cv2.imread('./test.jpg')

dst = cv2.Sobel(img, cv2.CV_64F, 1.0, ksize=3)
dst = cv2.convertScaleAbs(dst)
cv2.imshow("dst", dst)
cv2.waitKey()
cv2.destroyAllWindows()
Copy the code

There is a lot to explain and learn from the above code, so let’s step through it.

The second parameter in the Sobel filter, set to cv2.cv_64f, searched the Internet for a common explanation

We generally deal with 8-bit images, and when the calculated gradient is less than zero, it will automatically change to 0, resulting in the loss of boundary information. The reason why the gradient is less than zero is that the boundary derivative from black to white is an integer, while the boundary derivative from white to black is negative. When the image depth is Np.uint8, the negative value will become 0. Based on this reason, it is necessary to set the data type of output image higher, such as Cv2.cv_16s, cv2.cv_64f equivalent.

And the easy way to remember it, of course, is to set it to -1.

After the processing is completed, it is converted back to the original uint8 format by the cv2.convertScaleAbs function. If it is not converted, the image obtained is not correct.

In contrast, conversion and no conversion twice run effect, the right side of the apparent loss of boundary values.

Note that there is one more detail that needs to be explained in the code above

dst = cv2.Sobel(img, cv2.CV_64F, 1.0, ksize=3)
Copy the code
  • dx: Derivative order in x direction;
  • dy: The derivative order in the y direction.

So dx is equal to 1,dy is equal to 0, which means you compute the horizontal direction, not the vertical direction, which direction is equal to 1, which direction you compute.

You can also compare different values of the ddepth parameter in the function to check the difference content. The following code uses the image fusion function Cv.addWeighted

import numpy as np
import cv2 as cv
def sobel_demo1(image) :
    grad_x = cv.Sobel(image, cv.CV_16S, 1.0)
    grad_y = cv.Sobel(image, cv.CV_16S, 0.1)
    gradx = cv.convertScaleAbs(grad_x)
    grady = cv.convertScaleAbs(grad_y)
    cv.imshow('sobel_demo1_gradient_x', gradx)
    cv.imshow('sobel_demo1_gradient_y', grady)
    Merge the x and y gradients
    add_image = cv.addWeighted(gradx, 0.5, grady, 0.5.0)
    cv.imshow('sobel_demo1_addWeighted', add_image)

def sobel_demo2(image) :
    grad_x = cv.Sobel(image, cv.CV_32F, 1.0)
    grad_y = cv.Sobel(image, cv.CV_32F, 0.1)
    gradx = cv.convertScaleAbs(grad_x)
    grady = cv.convertScaleAbs(grad_y)
    cv.imshow('sobel_demo2_gradient_x', gradx)
    cv.imshow('sobel_demo2_gradient_y', grady)
    Merge the x and y gradients
    add_image = cv.addWeighted(gradx, 0.5, grady, 0.5.0)
    cv.imshow('sobel_demo2_addWeighted', add_image)

src = cv.imread('./test.jpg'.1)
sobel_demo1(src)
sobel_demo2(src)
cv.waitKey(0)
cv.destroyAllWindows()
Copy the code

You can calculate the x and y derivatives directly using the Sobel function without using the image fusion function as follows:

def sobel_demo3(image) :
    grad = cv.Sobel(image, cv.CV_16S, 1.1)
    grad = cv.convertScaleAbs(grad)
    cv.imshow('sobel_demo1_gradient', grad)
Copy the code

The result is much worse than the image fusion effect.

This comparison is not easy to see clearly, can be combined with the following code for transformation, the picture together to display.

def sobel_demo1(image) :

    grad_x = cv.Sobel(image, cv.CV_16S, 1.0)
    grad_y = cv.Sobel(image, cv.CV_16S, 0.1)
    gradx = cv.convertScaleAbs(grad_x)
    grady = cv.convertScaleAbs(grad_y)

    Merge the x and y gradients
    dst = cv.addWeighted(gradx, 0.5, grady, 0.5.0)
    imgs1 = np.hstack([cv.cvtColor(image, cv.COLOR_BGR2RGB), gradx])
    imgs2 = np.hstack([grady, dst])
    img_all = np.vstack([imgs1, imgs2])
    plt.figure(figsize=(20.10))
    plt.imshow(img_all, cmap=plt.cm.gray)
    plt.show()
Copy the code

The above code works as follows and is easy to compare.

Sobel operator algorithm has the advantages of simple calculation and fast speed.

However, because only the two-direction template is used, only the horizontal and vertical direction edges can be detected, so the edge detection effect of this algorithm is not very ideal for images with complex texture. The algorithm believes that all pixels with new gray value greater than or equal to the threshold are edge points. This judgment is not very reasonable, will cause the edge of the misjudgment, because many noise points gray value is also very large.

Description and use of Scharr operator

In the Sobel operator algorithm function, if ksize=-1, a 3×3 Scharr filter will be used.

It’s the same principle as the Sobel operator, but the convolution kernel is different, so it’s a little bit more precise.

The prototype of this function is as follows:

# Sobel operator algorithm
dst = cv2.Sobel(src, ddepth, dx, dy[, dst[, ksize[, scale[, delta[, borderType]]]]])
# Scharr operator algorithm
dst = cv2.Scharr(src, ddepth, dx, dy[, dst[, scale[, delta[, borderType]]]])
Copy the code

Its parameters are basically the same as Sobel’s.

The test code is as follows to compare the differences between the two algorithms:

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt

def scharr_demo(image) :
    grad_x = cv.Scharr(image, cv.CV_16S, 1.0)
    grad_y = cv.Scharr(image, cv.CV_16S, 0.1)
    gradx = cv.convertScaleAbs(grad_x)
    grady = cv.convertScaleAbs(grad_y)

    Merge the x and y gradients
    dst = cv.addWeighted(gradx, 0.5, grady, 0.5.0)
    return dst

def sobel_demo1(image) :
    dst1 = scharr_demo(image)
    grad_x = cv.Sobel(image, cv.CV_16S, 1.0)
    grad_y = cv.Sobel(image, cv.CV_16S, 0.1)
    gradx = cv.convertScaleAbs(grad_x)
    grady = cv.convertScaleAbs(grad_y)
    Merge the x and y gradients
    dst = cv.addWeighted(gradx, 0.5, grady, 0.5.0)
    imgs = np.hstack([dst, dst1])
    plt.figure(figsize=(20.10))
    plt.imshow(imgs, cmap=plt.cm.gray)
    plt.show()


def sobel_demo3(image) :
    grad = cv.Sobel(image, cv.CV_16S, 1.1)
    grad = cv.convertScaleAbs(grad)
    cv.imshow('sobel_demo1_gradient', grad)

def sobel_demo2(image) :
    grad_x = cv.Sobel(image, cv.CV_32F, 1.0)
    grad_y = cv.Sobel(image, cv.CV_32F, 0.1)
    gradx = cv.convertScaleAbs(grad_x)
    grady = cv.convertScaleAbs(grad_y)
    cv.imshow('sobel_demo2_gradient_x', gradx)
    cv.imshow('sobel_demo2_gradient_y', grady)
    Merge the x and y gradients
    add_image = cv.addWeighted(gradx, 0.5, grady, 0.5.0)
    cv.imshow('sobel_demo2_addWeighted', add_image)

src = cv.imread('./test.jpg'.1)
sobel_demo1(src)
# sobel_demo2(src)
# sobel_demo3(src)
cv.waitKey(0)
cv.destroyAllWindows()
Copy the code

sobelOperator andscharrOperator differences

  • sobelOperator coefficient: [1 2 1];scharrOperator [3 10 3];
  • scharrOperator thansobelOperators have higher accuracy;
  • scharrThe operator can detect the smaller boundary.

Laplacian operator

The method for realizing Laplace function is: Firstly, Sobel operator is used to calculate the second order x and y derivatives, and then sum them.

At the application level, let’s first look at the prototype of this function:

dst = cv2.Laplacian(src, ddepth[, dst[, ksize[, scale[, delta[, borderType]]]]])
Copy the code

No special parameters, can directly enter the test code learning.

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

image = cv.imread('./test1.png',cv.IMREAD_GRAYSCALE)

laplacian = cv.Laplacian(image, cv.CV_64F)
laplacian = cv.convertScaleAbs(laplacian)

imgs = np.hstack([image, laplacian])

plt.figure(figsize=(20.10))
plt.imshow(imgs, cmap=plt.cm.gray)
plt.show()
Copy the code

I pre-converted the image to grayscale for testing convenience.

Laplace convolution kernel, you can define it yourself. Is used by default domain neighbourhood [[0, 1, 0], [1, 4, 1], [0, 1, 0]], modified to eight neighborhood [[1, 1, 1], [1, 8, 1], [1, 1, 1]].

The code of the custom convolution kernel is as follows, where the 2D convolution operation we learned before is used:

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

image = cv.imread('./test1.png',cv.IMREAD_GRAYSCALE)

Define the convolution sum
kernel = np.array([[1.1.1], [1, -8.1], [1.1.1]])
dst = cv.filter2D(image, cv.CV_32F, kernel=kernel)
lapalian = cv.convertScaleAbs(dst)

imgs = np.hstack([image, lapalian])

plt.figure(figsize=(20.10))
plt.imshow(imgs, cmap=plt.cm.gray)
plt.show()
Copy the code

Laplace is sensitive to noise, produces a bilateral effect, but does not detect the direction of the edge. And it is not directly used for edge detection, but only plays an auxiliary role, detecting whether a pixel is on the bright side or the dark side of the edge, using zero crossing to determine the position of the edge

Gradient is simply derivative, that’s the key derivative, to learn math, this part after the first time, we will learn. Gradient is shown on the image is to extract the edge of the image (whether horizontal, vertical, oblique, etc.), all that is needed is a core template, different template results are different. Based on this, the operator functions mentioned above can be represented by the function cv2.filter2d (). Different methods give different kernel templates and then evolve into different operators.

Eraser section

I hope you found something useful in today’s hour. See you in our next blog

reading

If you have ideas or techniques you’d like to share, feel free to leave them in the comments section.

Blogger ID: Dream eraser, hope you like, comment, favorites.