A, above

The industry believes that it is the affine invariant region with the best performance. MSER is the most Stable region obtained when the image is binarization with different gray threshold values. Characteristics: 1. The affine change of the image gray level has invariance 2. Stability, the support set of the region is stable relative to the change of gray level 3. Regions can be detected with varying degrees of fineness

2. Feature extraction process

  1. A series of gray threshold values are used to binarization the image
  2. For each threshold worthy binary image, the corresponding black region and white region are obtained
  3. The region with stable shape within a relatively wide range of gray threshold is MSERs
  4. Evaluation criteria:dA/dt.A: binary image area, T: gray scale

3. Interface layout

  • aLabel
  • twoButton
  • threePicture

Iv. Functional realization

4.1 Opening pictures

 private void openFileBtn_Click(object sender, EventArgs e)
 {
     OpenFileDialog openfiledialog = new OpenFileDialog();
     openfiledialog.Filter = "PNG Files (*.png)|*.png|JPG Files (*.jpg)|*.jpg|GIF Files (*.gif)|*.gif";
     openfiledialog.RestoreDirectory = true;
     if (openfiledialog.ShowDialog() == DialogResult.OK)
     {
         Console.WriteLine(openfiledialog.FileName);
         fileName = openfiledialog.FileName;

         //Mat src = new Mat("foo.png", LoadMode.Color);
         Mat src = new Mat(fileName);
         //Mat src = new Mat(fileName, ImreadModes.Color);
         var frameBitmap = BitmapConverter.ToBitmap(src);

         pictureBox1.Image?.Dispose();
         pictureBox1.Image = frameBitmap;
     }
 }
Copy the code

4.2 Feature extraction – source code

private void mserBtn_Click(object sender, EventArgs e)
{
    if (String.IsNullOrEmpty(fileName))
    {
        MessageBox.Show("Please open a picture first.");
        return;
    }
     

    Mat src = new Mat(fileName);
    Mat gray = new Mat();
    Mat dst = src.Clone();

    Cv2.CvtColor(src, gray, ColorConversionCodes.BGR2GRAY);

    int delta = 5;
    int minArea = 60;
    int maxArea = 14400;
    //int maxArea = 100000;
    double maxVariation = 0.25;

    MSER mser = MSER.Create(delta, minArea, maxArea, maxVariation);

    DateTime currentTime = DateTime.Now;
    KeyPoint[] contours = mser.Detect(gray, null); 
    resultLabel.Text = "Detect time:" + (System.DateTime.Now - currentTime).TotalMilliseconds + "ms";
    Console.WriteLine("contours len="+ contours.Length);

    currentTime = DateTime.Now;
    Point[][] outPoint;
    Rect[] bboxes;
    InputArray input = InputArray.Create(gray);
    mser.DetectRegions(input, out outPoint, out bboxes);
    resultLabel.Text += ", DetectRegions time:" + (System.DateTime.Now - currentTime).TotalMilliseconds + "ms";
    Console.WriteLine("bboxes len=" + bboxes.Length);

    gray = src.Clone();
    foreach (KeyPoint pts in contours)
    {
        gray.Circle((int)pts.Pt.X, (int)pts.Pt.Y, (int)pts.Size, Scalar.Black);
        gray.Circle((int)pts.Pt.X, (int)pts.Pt.Y, (int)1, Scalar.Black, 2);
        Console.WriteLine("("+pts.Pt.X+","+pts.Pt.Y+") Size="+pts.Size+", Angle="+pts.Angle+", Response="+pts.Response + ", ClassId=" + pts.ClassId);
    }

    foreach (Point[] pts in outPoint)
    {
        foreach (Point p in pts)
        {
            dst.Circle(p, 1, Scalar.Black); }}foreach (Rect b in bboxes)
    {
        dst.Rectangle(b, Scalar.Black, 2);
    }

    srcPictureBox.Image = BitmapConverter.ToBitmap(src); 
    grayPictureBox.Image = BitmapConverter.ToBitmap(gray); 
    dstPictureBox.Image = BitmapConverter.ToBitmap(dst);
}
Copy the code

4.3 Feature Extraction — Parameter explanation

MSER does not need to pass special parameters when detecting but only when creating

//
/ / in this paper:
// Creates MSER parameters
//
/ / parameters:
// delta:
// delta, in the code, it compares (size_{i}-size_{i-delta})/size_{i-delta}
//
// minArea:
// prune the area which smaller than min_area
//
// maxArea:
// prune the area which bigger than max_area
//
// maxVariation:
// prune the area have simliar size to its children
//
// minDiversity:
// trace back to cut off mser with diversity < min_diversity
//
// maxEvolution:
// for color image, the evolution steps
//
// areaThreshold:
// the area threshold to cause re-initialize
//
// minMargin:
// ignore too small margin
//
// edgeBlurSize:
// the aperture size for edge blur
public static MSER Create(int delta = 5.int minArea = 60.int maxArea = 14400.double maxVariation = 0.25.double minDiversity = 0.2.int maxEvolution = 200.double areaThreshold = 1.01.double minMargin = 0.003.int edgeBlurSize = 5);
Copy the code
  • delta, the gray difference of the region
  • minArea, the lower limit of the detection area (number of pixels)
  • maxArea, the upper limit of the detection area (number of pixels)
  • maxVariation, the upper limit of the deviation between the region and the subregion

The rest of the parameters are MSCR, and this C is Color C so if you input gray image, you use MSER and if you input Color image, you use MSCR

  • minDiversity, For color image, trace back to cut off mser with diversity less than min_diversity
  • maxEvolution, For color image, the Evolution steps
  • areaThreshold, For color image, the area threshold to cause re-initialize
  • minMargin, For color image, ignore too small margin
  • edgeBlurSize, For color image, the aperture size For edge blur

4.4 Feature Extraction – Detect and DetectRegions

Two detection methods, Detect and DetectRegions, exist in the OpenCV library

Detect returns an array of keypoints containing:

  • centerPoint2f
  • Feature sizeSize
  • Characteristics of the AngleAngle
  • The intensity of characteristicResponse
  • Octave,ClassIdEtc.

DetectRegions returns the two-dimensional array Point and the one-bit array Rect

  • Two arraysPointThere are N regions, and all the points in each regionPoint
  • An array ofRectThere are N regions, each regionRect

Five, the operation effect diagram

  • From left to right
  • The first one is the original
  • The second isDetectResults the figure
    • The middle of the detected results will be shaded, according to the resultssizeDraw a circle
  • The third isDetectRegionsResults the figure
    • All detected areas will be blackened
    • Box the Rect area found by the test

6. Identify and solve problems

The question is why the green rectangle in the upper right corner is not recognized

After thinking about it, I took a bold guess because the area is too large.

Because, you can’t think of any other factor, the small square green is recognized, why the bigger rectangle green is not recognized. Color out, that’s just the area.

Changed the code

    //int maxArea = 14400;
    int maxArea = 100000;
Copy the code

Run:

perfect

7. Other Issues

There are other issues, of course

  • The last two pictures, they’re the same picture, but one isjpg, one ispng, but the effect of extraction is completely different
  • The guess is that the data conversion process is different
  • Just guess and then take the time to verify. If someone sees it and knows why, please comment

If you think it’s good, you can connect it with three clicks (like + favorites + follow).