This is the 26th day of my participation in Gwen Challenge

concept

Feature points

Both feature point and corner point are points with local features in the image, but feature point has a unique descriptor to describe the feature of pixel point. Feature points are composed of key points and descriptors, for example, ORB feature points. We need to use FAST algorithm to obtain key points and then calculate BRIEF descriptors to form ORB feature points. The so-called KeyPoint, literally understood as the key pixel in the image, the corresponding type in OpenCV is KeyPoint, which contains the coordinate, diameter, direction, intensity and other useful information of the pixel point; Descriptor, on the other hand, is a string of numbers that are used to uniquely describe key points. Descriptors can be used to distinguish two different key points, and can also be used to match the same key points. Therefore, it is important to construct descriptors in an appropriate way.

BRIEF describe the son

BRIEF(Binary Robust Independent Elementary Features) is a Binary string feature point descriptor proposed by Michael Calonder et al in ECCV2010. Compared with SIFT and SURF, It is simpler and has less storage space. Hamming Distance is used for comparison, and the matching speed is faster. BRIEF is only a feature descriptor. Therefore, it must be used together with feature point detection algorithms, such as FAST and Harris.

Basic process:

First, the image is smoothed, and then a Patch is selected around the feature points. In this Patch, NDN_DND point pairs are selected by a selected method. Then for each point pair (p,q)(p,q)(p,q) (p,q), we compare the pixel values of the two points. If I(p)

The advantages and disadvantages:

It abandons the traditional feature extraction method using gray histogram or gradient histogram of image local neighborhood, and uses random response detection instead, which greatly speeds up the construction of descriptors. The generated binary feature descriptors are convenient for high-speed matching (calculating hamming distance only requires xOR operation and counting the number of “1” in binary code, which can be achieved by underlying operations) and easy to implement on hardware. BRIEF has the advantages of speed, but has the disadvantages of no rotation, no deformation, no noise sensitivity, and no scale invariance.

The ORB feature points

Oriented BRIEF, ORB feature points are composed of FAST corner points and BRIEF descriptors. Key points are determined through FAST corner points detection, and then BRIEF descriptors of each key point are calculated to determine the ORB feature points. FAST (Features from Accelerated Segment Test) algorithm concept: If a pixel differs greatly from enough pixels in its surrounding neighborhood, the pixel may be a corner point. Corner points of FAST do not have dimensional invariance and rotation invariance, so the ORB feature point detection is based on FAST algorithm to do follow-up processing for dimensional invariance and rotation invariance.

API

OpenCV Feature2D module, literally a module that deals with two-dimensional features, has an API focused on feature point detection and matching.

Draw key points

public static void drawKeypoints(Mat image, MatOfKeyPoint keypoints, Mat outImage, Scalar color, int flags) 
Copy the code
  • Parameter 1: image, original image;
  • Parameter 2: keypoints, the keypoints of the original image;
  • Parameter 3: outImage, output image after drawing key points;
  • Parameter 4: color, draw the color of key points;
  • Parameter 5: flags, draw function flag bit.
// C++: enum DrawMatchesFlags
public static final int
        DrawMatchesFlags_DEFAULT = 0.// Create the output matrix, draw the center of the circle, the circle around the key point, and the size and direction of the key point
        DrawMatchesFlags_DRAW_OVER_OUTIMG = 1.// Without creating the output matrix, draw the key points directly in the original image
        DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS = 2.// Do not draw a single key point
        DrawMatchesFlags_DRAW_RICH_KEYPOINTS = 4;      // Draw circles at key points to reflect the size and direction of key points
Copy the code

Key points of calculation

Single image detection and multiple image detection

public void detect(Mat image, MatOfKeyPoint keypoints, Mat mask)
Copy the code
  • Parameter 1: image, input image of key point to be calculated;
  • Parameter 2: keypoints, the keypoints obtained by calculation;
  • Parameter 3: mask, mask matrix, size must be the same as the input image and the type is CV_8U.
public void detect(List<Mat> images, List<MatOfKeyPoint> keypoints, List<Mat> masks)
Copy the code
  • Parameter 1: images, input images “we” of key points to be calculated;
  • Parameter 2: keypoints, calculated keypoints “we”;
  • Parameter 3: Masks, masks matrix “s”.

Computational descriptor

public void compute(Mat image, MatOfKeyPoint keypoints, Mat descriptors)
Copy the code
  • Parameter 1: image, input image corresponding to key point;
  • Parameter 2: keypoints, calculated keypoints;
  • Parameter 3: Descriptors, the corresponding descriptor for each key point.
public void compute(List<Mat> images, List<MatOfKeyPoint> keypoints, List<Mat> descriptors) 
Copy the code
  • Parameter 1: images, input image “we” corresponding to the key point;
  • Parameter 2: keypoints, the calculated keypoints “are”;
  • Parameter 3: Descriptors, the corresponding descriptor of the key “ones”.

Compute key points and descriptors

public void detectAndCompute(Mat image, Mat mask, MatOfKeyPoint keypoints, Mat descriptors, boolean useProvidedKeypoints)
Copy the code
  • Parameter 1: image, input image of key point to be calculated;
  • Parameter 2: mask, mask matrix, size must be the same as the input image and type is CV_8U;
  • Parameter 3: keypoints, used as input to represent the calculated keypoints, used as output to store the keypoints calculated by this method. It depends on the fifth parameter;
  • Parameter 4: Descriptors, the corresponding descriptor for each key;
  • Parameter 5: useProvidedKeypoints, whether to use the existing key points identifier. The default value is false, indicating that the method calculates key points by itself, and the third parameter is not used as input. If set to true, the third argument is input to indicate the calculated key point, and the function of this method is equal tocomputeAgain, only the descriptors are counted.

ORB

public static ORB create(int nfeatures, float scaleFactor, int nlevels, int edgeThreshold, int firstLevel, int WTA_K, int scoreType, int patchSize, int fastThreshold)
Copy the code
  • Parameter 1: Nfeatures, maximum number of feature points detected;
  • Parameter 2: scaleFactor, pyramid size scaling ratio, must be greater than 1. If scaleFactor=2, which means that the width and height of the next level of the classical pyramid is reduced by half, i.e. the next level has four times fewer pixels than the previous level, such a large scale will significantly reduce the feature matching score. On the other hand, if scaleFactor and 1 are too close to each other, scaling to a given size requires more pyramid levels, and speed will suffer;
  • Parameter 3: Nlevels, number of pyramid layers. The minimum linear size is equal to (input_image_linear_size)/ POW (scaleFactor,nlevelsāˆ’firstLevel)(input\_image\_linear\_size)/ POW (scaleFactor, (nlevels – firstLevel) (input_image_linear_size)/pow scaleFactor, nlevels – firstLevel);
  • Parameter 4: edgeThreshold, edgeThreshold, which should roughly match patchSize parameter;
  • Parameter 5: firstLevel, the level to put the original image into the pyramid. If there are layers in front, zoom in on the image to fill them.
  • Parameter 6: WTA_K, the number of pixels needed to generate each descriptor. If WTA_K is the default value 2, it means that a BRIEF descriptor of 0/1 result is obtained by randomly selecting a pair of points and comparing pixel values. If WTA_K=3, it means that the descriptor is obtained by comparing the pixel values of three random points. If WTA_K=4, it means that the descriptor is obtained by comparing the pixel values of 4 random points.
  • Parameter 7: scoreType, critical point evaluation method, default value isHARRIS_SCORE;
// C++: enum ScoreType
public static final int
        HARRIS_SCORE = 0,
        FAST_SCORE = 1;
Copy the code
  • Parameter 8: patchSize, the size of the neighborhood around the key point when the description sub-is generated;
  • Parameter 9: fastThreshold, FAST Corner calculation threshold.

operation

/** * key words: yidong * 2021-03-28 */
class ORBActivity : AppCompatActivity() {

    private val bgr by lazy {
        Utils.loadResource(this, R.drawable.lena)
    }
    private val rgb by lazy { bgr.toRgb() }
    
    private val mBinding: ActivityOrbBinding by lazy {
        ActivityOrbBinding.inflate(layoutInflater)
    }

    override fun onCreate(savedInstanceState: Bundle?). {
        super.onCreate(savedInstanceState)
        setContentView(mBinding.root)
        wrapCoroutine({ showLoading() }, { doORB() }, { hideLoading() })
    }


    private fun showLoading(a) {
        mBinding.isLoading = true
    }

    private fun hideLoading(a) {
        mBinding.isLoading = false
    }

    private fun doORB(a) {
        val orbDetector = ORB.create(
            500.1.2 f.8.31.0.2,
            ORB.HARRIS_SCORE,
            31.20
        )
        val points = MatOfKeyPoint()
        orbDetector.detect(rgb, points)
        val description = Mat()
        orbDetector.compute(rgb, points, description)
        val result = Mat()
        rgb.copyTo(result)
        drawKeypoints(rgb, points, result, Scalar.all(255.0), DrawMatchesFlags_DRAW_RICH_KEYPOINTS)
        GlobalScope.launch(Dispatchers.Main) {
            mBinding.ivResult.showMat(result)
        }
    }

    override fun onDestroy(a) {
        bgr.release()
        rgb.release()
        super.onDestroy()
    }
}
Copy the code

The effect

The source code

Github.com/onlyloveyd/…

Reference Documents:

Docs.opencv.org/master/d0/d…

Docs.opencv.org/master/db/d…

Blog.csdn.net/songzitea/a…

www.cnblogs.com/zyly/p/9615…