Original link: tecdat.cn/?p=13173

 

introduce

In this tutorial, we will learn how to perform image processing using the Python language. We are not limited to a single library or framework; However, the Open CV library is the one we will use most often. We’ll discuss some image processing first, and then move on to different applications/scenarios where image processing can be easily used.

 

What is image processing?

It is important to understand exactly what image processing means, and what image processing does in the larger picture before diving into the role of image processing. Image processing is most often referred to as “digital image processing”, and the area of frequent use is “computer vision”. Don’t get confused. Image processing algorithms and computer vision (CV) algorithms both take images as input. However, in image processing, the output is also an image, while in computer vision, the output may be some features/information about the image.

 

Why do we need it?

Most of the data we collect or generate is raw data, which is not suitable for direct use in applications for a number of possible reasons. Therefore, we need to analyze it and perform the necessary preprocessing before we use it.

For example, suppose we are trying to build a CAT classifier. Our program takes an image as input and then tells us whether the image contains a cat. The first step in building the classifier is to collect hundreds of pictures of cats. A common problem is that all the images we grab will not have the same size/dimensions, so we need to adjust/preprocess all the dimensions to standard before feeding them into the model for training.

This is just one of many reasons why image processing is essential to any computer vision application.

 

A prerequisite for

Before we continue, let’s discuss what we need to know to follow this tutorial with ease. First, you should have a basic programming knowledge of any language. Second, you should know the basics of what machine learning is and how it works, because in this article we will use some machine learning algorithms for image processing. Also, it would be helpful if you had any knowledge or basic knowledge of Open CV before continuing with this tutorial. But it’s not necessary.

One thing you must know in order to follow this tutorial is exactly how images are represented in memory. Each image is represented by a set of pixels, a matrix of pixel values. For grayscale images, pixel values range from 0 to 255 and represent the intensity of that pixel. For example, if you have a 20×20 image, it will be represented as a 20×20 matrix (400 pixel values in total).

If you are working with color images, you should know that it will have three channels – red, green, and blue (RGB). Thus, a single image will have three such matrices.

 

The installation

Note: Since we will be using OpenCV through Python, the implicit requirement is that You have Python installed on your workstation (version 3).

windows

$ pip install opencv-python
Copy the code

Apple system

$ brew install opencv3 --with-contrib --with-python3
Copy the code

Linux

$ sudo apt-get install libopencv-dev python-opencv
Copy the code

To check for a successful installation, run the following command from the Python Shell or command prompt:

import cv2
Copy the code

 

Some basics you should know

Before we continue using image processing in our applications, it is important to understand which operations fall into this category and how to do them. These and other actions will be used in future applications.

For this article, we will use the following image:

Note: The image has been scaled for display in this article, but the original size we used was about 1180×786.

You may have noticed that the image is currently colored, which means it is represented by three color channels, namely red, green, and blue. We convert the image to a grayscale image and divide the image into separate channels using the following code.

 

Find image details

After loading the image using the imread() function, we can retrieve some simple properties about the image, such as the number of pixels and size:


print("Image Properties")
print("- Number of Pixels: " + str(img.size))
print("- Shape/Dimensions: " + str(img.shape))
Copy the code

Output:

Image Properties
- Number of Pixels: 2782440
- Shape/Dimensions: (1180, 786, 3)
Copy the code

 

Divide the image into separate channels

Now we will use OpenCV to divide the image into red, green, and blue components and display them:

Cv2_imshow (red) # cv2_imshow(blue) # cv2_imshow(green) # cv2_imshow(img_gs) # show the grey versionCopy the code

For simplicity, we will display only grayscale images.

 

Grayscale image:

 

Image threshold

The concept of thresholds is very simple. As discussed above in image representation, pixel values can be anything from 0 to 255. Suppose we want to convert an image to a binary image, that is, assign a value of 0 or 1 to the pixels. To do this, we can perform thresholding. For example, if the threshold (T) value is 125, all pixels with a value greater than 125 will be assigned a value of 1, and all pixels with a value less than or equal to that value will be assigned a value of 0. Get a better understanding through the code.

Images for thresholds:

import cv2

cv2_imshow(threshold)
Copy the code

As you can see, in the generated image, two regions have been established, namely the black region (pixel value 0) and the white region (pixel value 1). As it turns out, the threshold we set is right in the middle of the image, which is why we split the black and white values here.

 

Application field

# 1: Remove noise in the image

Now that you have a basic understanding of what image processing is and what it does, let’s move on to some of its specific applications.

In most cases, there is noise in the raw data we collect, even though the images are hard to perceive undesirable features. Although these images can be directly used for feature extraction, the accuracy of the algorithm will be greatly affected. This is why it is important to image process the image before passing it on to the algorithm for better accuracy.

There are many different types of noise, such as Gaussian noise, pepper noise and so on. We can apply filters to remove noise from the image, or to minimize it, or at least to minimize its impact. There are also many options for filters, each with a different strength and therefore the best choice for a particular type of noise.

To understand this properly, we will add “salt and pepper” noise to the grayscale version of the rose image considered above, then try to remove that noise from the noisy image using different filters, and then see which is best – suitable for that type.

import numpy as np

cv2.imwrite('sp_05.jpg', sp_05)
Copy the code

Ok, we added noise to the rose image and it now looks like this:

Noisy images:

Now let us apply different filters to it and note our observation that each filter reduces the noise.

 

Arithmetic filter with sharpened kernel

Sharpened_img = cv2.filter2d (SP_05, -1, kernel_sharpening) cv2_imshow(sharpened_img)Copy the code

By applying an arithmetic filter to the image with noise, the resulting image is shown below. When compared with the original grayscale image, we can see that it makes the image too bright and fails to highlight the bright spots on the rose. Therefore, it can be concluded that the arithmetic filter cannot remove noise.

Arithmetic filter output:

 

Midpoint filter

Print ("\n\n-- Effects on S&P Noise Image with Probability 0.5-- \n\n") midpoint(sp_05) print("\n\n-- Effects on S&P Noise Image with Probability 0.5-- \n\n") midpoint(sp_05)Copy the code

The result of applying a midpoint filter to a noisy image is shown below. After comparing with the original grayscale image, we can see that, just like the kernel method above, the image brightness is too high. However, it can highlight the rose. Therefore, it can be said that it is a better choice than arithmetic filter, but still cannot completely restore the original image.

 

Harmonic harmonic mean filter

Note: Implementations of these filters can be easily found online, and how they work is beyond the scope of this tutorial. We will examine applications at an abstract/higher level.

Print ("\n\n-- Effects on S&P Noise Image with Probability 0.5 --\n\n") cv2_imshow(contraharmonic_mean(sp_05, (3,3)) 0.5))Copy the code

The image obtained by applying Contraharmonic Mean Filter to the image under noise is shown below. After comparing with the original grayscale image, we can see that it has copied almost the same image as the original image. It has the same intensity/brightness level and also highlights the rose. Therefore, we can conclude that the harmonic mean filter is very effective in dealing with salt and pepper noise.

Now that we have found the best filter to recover the original image from the noisy image, we can move on to the next application.

 

2: Edge detection using Canny Edge Detector

So far, the rose image we’ve been using has a constant background, black, so for this application, we’ll use a different image to better show the function of the algorithm. The reason is that if the background is constant, the edge detection task becomes very simple, and we don’t want to do that.

We discussed cat classifiers earlier in this tutorial, so let’s look ahead to this example and see how image processing plays an integral role.

In the classification algorithm, the image is first scanned for “objects”, that is, when you enter an image, the algorithm finds all objects in that image and then compares them to the characteristics of the object you are looking for. If it is a cat classifier, it will compare all objects found in the image with the features of the cat image, and if a match is found, it will tell us that the input image contains cats.

Since we are using the CAT classifier as an example, it is fair to use cat images fairly. Here is the image we will use:

Images for edge detection:

 

Import cv2 import numpy as NP from matplotlib import pyplot as PLTCopy the code

Edge detection output:

 

As you can see, the part of the image that contains the object (in this case the cat) has been separated by the edge detection point to /. Now you must know, what is Canny Edge Detector and how is it implemented? Now let’s talk about it.

To understand the above, three key steps need to be discussed. First, it performs noise reduction on the image in a similar way to what we discussed earlier. Second, it uses the first derivative at each pixel to find edges. The logic behind this is that a sudden change in intensity at a point with an edge causes a spike in the value of the first derivative, making the pixel an “edge pixel”.

Finally, it enforces hysteresis thresholds; We said above that there is a peak in the value of the first derivative of the edge, but we didn’t say how high the peak needs to be to classify it as an edge – this is called a threshold!

Earlier in this tutorial, we discussed simple thresholding. Hysteresis thresholds are an improvement on this by using two thresholds instead of one. The reason behind this is that if the threshold is too high, we may miss some actual edges (true negative values), while if the threshold is too low, we get a lot of points that are classified as edges that are not actually edges (false positive values). . Set one threshold to high and one threshold to low. All points above the “high threshold” are identified as edges, and all points above the low threshold but below the high threshold are evaluated; Points near or adjacent to points that are identified as edges are also identified as edges, and the rest is discarded.

These are the basic concepts/methods used by the Canny Edge Detector algorithm to identify image edges.

 

conclusion

In this article, you learned how to install OpenCV (the most popular library for Python image processing) on different platforms such as Windows, MacOS, and Linux, and how to verify that the installation was successful.

We went on to discuss what image processing is and its use in the field of computer vision for machine learning. We discussed some common types of noise and how to use different filters to remove it from an image before using it in your application.

In addition, we learned how image processing can play an integral role in high-end applications such as “object detection” or “classification.” Please note that this article is just the tip of the iceberg and cannot be covered in a single tutorial.