Android OpenCV (41) : Image Segmentation (Flood fill method)

This is the 12th day of my participation in Gwen Challenge

Image segmentation

Image segmentation is the technology and process of dividing an image into several specific and unique regions and proposing objects of interest. It is the key step from image processing to image analysis. The existing image segmentation methods can be divided into the following categories: thresholding based segmentation method, region based segmentation method, edge based segmentation method and specific theory based segmentation method. From a mathematical point of view, image segmentation is the process of dividing a digital image into non-intersecting areas. The process of image segmentation is also a marking process, that is, the pixels belonging to the same region are assigned the same number.

Flood filling method

Flooding filling algorithm is based on the difference between pixel gray values to find the same area to achieve segmentation. We can understand the gray value of the image as the height of pixels. Such an image can be regarded as rugged ground or mountainous area. A certain amount of water is poured into a low-lying place on the ground, and the water will cover up the area below a certain height. Flooding filling method is the use of such a principle, its form is similar to water injection, so it is called the image of “flooding”.

Consistent with water injection into the ground, flooding filling method also needs to select a water injection pixel in the image, which is called the seed point. The seed point continuously diffuses outward according to certain rules to form an independent region with similar characteristics, thus achieving image segmentation. Flooding filling segmentation method is mainly divided into the following three steps:

Select seed point (x, Y);
With the seed point as the center, judge the difference between the pixel value of neighborhood 4 or neighborhood 8 and the pixel value of the seed point, and add the pixel point whose difference is less than the threshold into the region.
Use the newly added pixels as new seed points and repeat the second step until no new pixels have been added to the area.

Calculation method:

In case of a grayscale image and floating range
$\texttt{src} (x’,y’)- \texttt{loDiff} \leq \texttt{src} (x,y) \leq \texttt{src} (x’,y’)+ \texttt{upDiff}$
In case of a grayscale image and fixed range
$\texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)- \texttt{loDiff} \leq \texttt{src} (x,y) \leq \texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)+ \texttt{upDiff}$
In case of a color image and floating range
$\texttt{src} (x’,y’)_r- \texttt{loDiff} _r \leq \texttt{src} (x,y)_r \leq \texttt{src} (x’,y’)_r+ \texttt{upDiff} _r,$ $\texttt{src} (x’,y’)_g- \texttt{loDiff} _g \leq \texttt{src} (x,y)_g \leq \texttt{src} (x’,y’)_g+ \texttt{upDiff} _g$
and
$\texttt{src} (x’,y’)_b- \texttt{loDiff} _b \leq \texttt{src} (x,y)_b \leq \texttt{src} (x’,y’)_b+ \texttt{upDiff} _b$
In case of a color image and fixed range
$\texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)_r- \texttt{loDiff} _r \leq \texttt{src} (x,y)_r \leq \texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)_r+ \texttt{upDiff} _r,$ $\texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)_g- \texttt{loDiff} _g \leq \texttt{src} (x,y)_g \leq \texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)_g+ \texttt{upDiff} _g$
and
$\texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)_b- \texttt{loDiff} _b \leq \texttt{src} (x,y)_b \leq \texttt{src} ( \texttt{seedPoint} .x, \texttt{seedPoint} .y)_b+ \texttt{upDiff} _b$

In the above formula, SRC (x ‘,y ‘) represents the known values of adjacent pixels in this region. In short, when is a floating range, only the difference between it and the neighborhood that already belongs to a certain region is small enough (meeting the formula range), can it be selected to enter the region; When it is a fixed range, only the difference between the pixels and the seed is small enough to be selected into the region.

API

public static int floodFill(Mat image, Mat mask, Point seedPoint, Scalar newVal, Rect rect, Scalar loDiff, Scalar upDiff, int flags) 
Copy the code

Return value: Number of filled pixels.
Parameter 1: image, input and output image, image can be CV_8U or CV_32F type of single-channel or three-channel image. When the last parameter is set to FLOODFILL_MASK_ONLY flag, the original image is not changed.
Parameter 2: Mask, operation mask, single-channel 8-bit image, 2 pixels wider and 2 pixels higher than the input image. Because the mask is both an input parameter and an output parameter, it must be initialized. Flooding does not fill the non-zero region in the mask. For example, the output of edge detection can be used as an operational mask to prevent flooding from filling edges.
Parameter 3: seedPoint
Parameter 4: newVal, the new value of the redrawn field pixel.
Parameter 5: Rect, default 0, sets the minimum bounding rectangle region that the floodFill function will redraw, i.e., if the floodFill region is < RECt, it will not be filled.
Parameter 6: loDiff, the lower bound difference value added to the seed point region condition. Represents the maximum negative difference in brightness or color between the currently observed pixel value and its neighbor pixel value or the seed pixel value to be added.
Parameter 7: upDiff, the upper bound difference value added to the condition of the seed point region. Represents the maximum positive difference in brightness or color between the currently observed pixel value and its neighbor pixel value or the seed pixel value to be added.
Parameter 8: flags, the operation flag of the flood filling method. The logo consists of three parts. The first part indicates the type of neighborhood, 4 neighborhood or 8 neighborhood. The second part represents the new pixel value of the filled pixel point in the mask matrix. The third part is the rule mark of the filling algorithm. Int Specifies the operation identifier. The default value is 4, consisting of 23 bits.
- Lower octet (0 to 7) : Used to control the connectivity of the algorithm, 4 (default) or 8. If set to 4, it means that the filling algorithm only considers the adjacent points in the current pixel level or disposal direction; if set to 8, in addition to the above adjacent points, adjacent points in the diagonal direction will also be included.
- Middle octet (8 to 15) : Specifies the value of the image to fill the mask. If the middle octet is 0, the mask will be filled with 1.
- High eight bits (16-23) : The value can be 0 or a combination of the following two identifiers.
  
  FLOODFILL_FIXED_RANGE: If set to this identifier, the difference between the current pixel and the seed is considered, otherwise the difference between the current pixel and its neighbors is considered.
  
  FLOODFILL_MASK_ONLY. If set to FLOODFILL_MASK_ONLY, the function does not fill the original image, but fills the mask image. That is, the third argument, newVal, is ignored.
```
// C++: enum FloodFillFlags
public static final int
        FLOODFILL_FIXED_RANGE = 1 << 16,
        FLOODFILL_MASK_ONLY = 1 << 17;
Copy the code
```
  So, can use bitwise or flag, namely the ‘|’. For example, if you want to fill a 4 neighborhood with a fixed pixel range, fill the mask instead of the original image, and set the fill value to 250, the input parameters are
```
4 or (250 shl 8) or Imgproc.FLOODFILL_FIXED_RANGE or Imgproc.FLOODFILL_MASK_ONLY 
Copy the code
```

operation

* author: yidong * 2020/11/7 */
class FloodFillActivity : AppCompatActivity() {
    private val mBinding by lazy { ActivityFloodFillBinding.inflate(layoutInflater) }
    private lateinit var mMenuDialog: BottomSheetDialog
    private lateinit var mMenuDialogBinding: LayoutFloodFillMenuBinding

    private var mConnectionType = 4
    private var mFloodFillFlag = 0
    private var mScalarNumber = 250 shl 8

    private lateinit var mRgb: Mat
    private var loDiff = 0.0
        set(value) {
            field = value
            doFloodFill()
        }
    private var upDiff = 0.0
        set(value) {
            field = value
            doFloodFill()
        }

    override fun onCreate(savedInstanceState: Bundle?). {
        super.onCreate(savedInstanceState)
        setContentView(mBinding.root)
        val bgr = Utils.loadResource(this, R.drawable.wedding)
        mRgb = Mat()
        Imgproc.cvtColor(bgr, mRgb, Imgproc.COLOR_BGR2RGB)
        mBinding.ivLena.showMat(mRgb)
        mBinding.sbLow.setOnSeekBarChangeListener(object : SeekBar.OnSeekBarChangeListener {
            override fun onProgressChanged(p0: SeekBar? , p1:Int, p2: Boolean) {
                mBinding.tvLoDiff.text = p1.toString()
                loDiff = p1.toDouble()
            }

            override fun onStartTrackingTouch(p0: SeekBar?).{}override fun onStopTrackingTouch(p0: SeekBar?). {
            }

        })
        mBinding.sbUp.setOnSeekBarChangeListener(object : SeekBar.OnSeekBarChangeListener {
            override fun onProgressChanged(p0: SeekBar? , p1:Int, p2: Boolean) {
                mBinding.tvUpDiff.text = p1.toString()
                upDiff = p1.toDouble()
            }

            override fun onStartTrackingTouch(p0: SeekBar?).{}override fun onStopTrackingTouch(p0: SeekBar?). {
            }

        })
        mBinding.btFlag.setOnClickListener {
            showMenuDialog()
        }
        doFloodFill()
    }

    private fun doFloodFill(a) {
        val tmp = mRgb.clone()
        val maskers = Mat(mRgb.rows() + 2, mRgb.cols() + 2, CV_8UC1, Scalar.all(0.0))
        Imgproc.floodFill(
            tmp,
            maskers,
            Point(7.0.7.0),
            Scalar(65.0.105.0.225.0),
            Rect(),
            Scalar.all(loDiff),
            Scalar.all(upDiff),
            mConnectionType or mFloodFillFlag or mScalarNumber
        )
        if (mFloodFillFlag and Imgproc.FLOODFILL_MASK_ONLY == Imgproc.FLOODFILL_MASK_ONLY) {
            mBinding.ivResult.showMat(maskers)
        } else {
            mBinding.ivResult.showMat(tmp)
        }

        tmp.release()
        maskers.release()
    }

    private fun showMenuDialog(a) {
        if (!this::mMenuDialog.isInitialized) {
            mMenuDialog = BottomSheetDialog(this)
            mMenuDialogBinding = LayoutFloodFillMenuBinding.inflate(layoutInflater)
            mMenuDialog.setContentView(mMenuDialogBinding.root)
            mMenuDialog.setOnDismissListener {
                mConnectionType =
                    if (mMenuDialogBinding.rgFirst.checkedRadioButtonId == R.id.rb_8) {
                        8
                    } else {
                        4
                    }
                mFloodFillFlag = if (mMenuDialogBinding.cbFixed.isChecked) {
                    mFloodFillFlag or Imgproc.FLOODFILL_FIXED_RANGE
                } else {
                    mFloodFillFlag and Imgproc.FLOODFILL_FIXED_RANGE.inv()
                }
                mFloodFillFlag = if (mMenuDialogBinding.cbMaskOnly.isChecked) {
                    mFloodFillFlag or Imgproc.FLOODFILL_MASK_ONLY
                } else {
                    mFloodFillFlag and Imgproc.FLOODFILL_MASK_ONLY.inv()
                }
                try {
                    mScalarNumber = mMenuDialogBinding.etScalar.text.toString().toInt(10) shl 8
                } catch (e: NumberFormatException) {
                    e.printStackTrace()
                }
                doFloodFill()
            }
        }
        mMenuDialog.show()
    }

    override fun onDestroy(a) {
        mRgb.release()
        super.onDestroy()
    }
}
Copy the code

The results of

As shown in the following figure, when the FLAG is set to FLOODFILL_MASK_ONLY, the MASK MASK image is automatically displayed.

The source code

Github.com/onlyloveyd/…

Android OpenCV (41) : Image Segmentation (Flood fill method)

Image segmentation

Flood filling method

API

operation

The results of

The source code

Related Posts

ThinkPHP project summary

Use Python to beautify text, add Spaces between Chinese and English, numbers, and punctuation

Google Play is on the market, and you fill your own holes