This is the 10th day of my participation in Gwen Challenge

What is template matching?

Template matching is a method for searching and locating template images in larger images. OpenCV provides the matchTemplate() method for template matching. The template matching result returns a grayscale image, where each pixel represents the degree of matching between the neighborhood of the pixel and the template. Assuming the size of the input image (W * H) and the size of the template image (W * H), the size of the output image will be (W-W + 1, H-H + 1). Once you have the result, you can use the minMaxLoc() method to find the maximum/minimum position as the upper left corner of the rectangle, and (w, h) as the width and height of the rectangle to determine the region to which the template matches.

Principle of Template Matching

On the image to be detected, traverse the image from left to right and top to bottom, and calculate the pixel matching degree between the template and the overlapping sub-image from top to bottom. If the matching degree is greater, it means that the possibility of the same is greater. It’s just the calculation of the match.

API

public static void matchTemplate(Mat image, Mat templ, Mat result, int method, Mat mask) 
Copy the code

  • Parameter 1: image: indicates the image to be matched. Must be an 8 – bit or 32 – bit floating – point image.

  • Parameter two: templ, a template image of the same type as the input image and no larger than the source image.

  • Parameter 3: result, output result, must be a single-channel 32-bit floating point number, assuming the source image W*H, template image W*H, the result must be the size of (W-W +1) * (h-H +1).

  • Parameter 4: Method: indicates the flag of the matching mode. If it is TM_SQDIFF or TM_SQDIFF_NORMED, the smaller the calculated value, the higher the matching degree, and the larger the calculated value of the remaining flag bits, the higher the matching degree.

    // C++: enum TemplateMatchModes
public static final int
        TM_SQDIFF = 0,
        TM_SQDIFF_NORMED = 1,
        TM_CCORR = 2,
        TM_CCORR_NORMED = 3,
        TM_CCOEFF = 4,
        TM_CCOEFF_NORMED = 5;
Copy the code

  • Parameter 5: mask, optional mask. It must have the same size as the templ parameter, either with the same number of channels as templ or with a single channel. If the data type is # CV_8U, the mask is interpreted as a binary mask, indicating that only elements with a non-zero mask are used and that the weight is independent of the actual mask value (always equal to 1). If the data type is # CV_32F, the mask value will be used as the weight.

Flag bit

set


R ( x . y ) Is the result matrix, T ( x . . y . ) Is the template matrix, I ( x . y ) Is the source image matrix R(x,y) is the result matrix, T(x^,,y^,) is the template matrix, and I(x,y) is the source image matrix

TM_SQDIFF


R ( x . y ) = ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) – I ( x + x ‘ . y + y ‘ ) ) 2 R(x,y)= \sum _{x’,y’} (T(x’,y’)-I(x+x’,y+y’))^2

with mask:


R ( x . y ) = ∑ x ‘ . y ‘ ( ( T ( x ‘ . y ‘ ) – I ( x + x ‘ . y + y ‘ ) ) ⋅ M ( x ‘ . y ‘ ) ) 2 R(x,y)= \sum _{x’,y’} \left( (T(x’,y’)-I(x+x’,y+y’)) \cdot M(x’,y’) \right)^2

TM_SQDIFF_NORMED


R ( x . y ) = ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) – I ( x + x ‘ . y + y ‘ ) ) 2 ∑ x ‘ . y ‘ T ( x ‘ . y ‘ ) 2 ⋅ ∑ x ‘ . y ‘ I ( x + x ‘ . y + y ‘ ) 2 R(x,y)= \frac{\sum_{x’,y’} (T(x’,y’)-I(x+x’,y+y’))^2}{\sqrt{\sum_{ x’,y’}T(x’,y’)^2 \cdot \sum_{x’,y’} I(x+x’,y+y’)^2}}

with mask:


R ( x . y ) = ∑ x ‘ . y ‘ ( ( T ( x ‘ . y ‘ ) – I ( x + x ‘ . y + y ‘ ) ) ⋅ M ( x ‘ . y ‘ ) ) 2 ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) ⋅ M ( x ‘ . y ‘ ) ) 2 ⋅ ∑ x ‘ . y ‘ ( I ( x + x ‘ . y + y ‘ ) ⋅ M ( x ‘ . y ‘ ) ) 2 R(x,y)= \frac{\sum _{x’,y’} \left( (T(x’,y’)-I(x+x’,y+y’)) \cdot M(x’,y’) \right)^2}{\sqrt{\sum_{x’,y’} \left( T(x’,y’) \cdot M(x’,y’) \right)^2 \cdot \sum_{x’,y’} \left( I(x+x’,y+y’) \cdot M(x’,y’) \right)^2}}

TM_CCORR


R ( x . y ) = ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) ⋅ I ( x + x ‘ . y + y ‘ ) ) R(x,y)= \sum _{x’,y’} (T(x’,y’) \cdot I(x+x’,y+y’))

with mask:


R ( x . y ) = ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) ⋅ I ( x + x ‘ . y + y ‘ ) ⋅ M ( x ‘ . y ‘ ) 2 ) R(x,y)= \sum _{x’,y’} (T(x’,y’) \cdot I(x+x’,y+y’) \cdot M(x’,y’) ^2)

TM_CCORR_NORMED


R ( x . y ) = ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) ⋅ I ( x + x ‘ . y + y ‘ ) ) ∑ x ‘ . y ‘ T ( x ‘ . y ‘ ) 2 ⋅ ∑ x ‘ . y ‘ I ( x + x ‘ . y + y ‘ ) 2 R(x,y)= \frac{\sum_{x’,y’} (T(x’,y’) \cdot I(x+x’,y+y’))}{\sqrt{ \sum_{x’,y’}T(x’,y’)^2 \cdot \sum_{x’,y’} I(x+x’,y+y’)^2}}

with mask:


R ( x . y ) = ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) ⋅ I ( x + x ‘ . y + y ‘ ) ⋅ M ( x ‘ . y ‘ ) 2 ) ∑ x ‘ . y ‘ ( T ( x ‘ . y ‘ ) ⋅ M ( x ‘ . y ‘ ) ) 2 ⋅ ∑ x ‘ . y ‘ ( I ( x + x ‘ . y + y ‘ ) ⋅ M ( x ‘ . y ‘ ) ) 2 R(x,y)= \frac{\sum_{x’,y’} (T(x’,y’) \cdot I(x+x’,y+y’) \cdot M(x’,y’)^2)}{\sqrt{\sum_{x’,y’} \left( T(x’,y’) \cdot M(x’,y’) \right)^2 \cdot \sum_{x’,y’} \left( I(x+x’,y+y’) \cdot M(x’,y’) \right)^2}}

TM_CCOEFF


R ( x . y ) = ∑ x ‘ . y ‘ ( T ‘ ( x ‘ . y ‘ ) ⋅ I ‘ ( x + x ‘ . y + y ‘ ) ) R(x,y)= \sum _{x’,y’} (T'(x’,y’) \cdot I'(x+x’,y+y’))

where


T ‘ ( x ‘ . y ‘ ) = T ( x ‘ . y ‘ ) – 1 / ( w ⋅ h ) ⋅ ∑ x ‘ ‘ . y ‘ ‘ T ( x ‘ ‘ . y ‘ ‘ ) I ‘ ( x + x ‘ . y + y ‘ ) = I ( x + x ‘ . y + y ‘ ) – 1 / ( w ⋅ h ) ⋅ ∑ x ‘ ‘ . y ‘ ‘ I ( x + x ‘ ‘ . y + y ‘ ‘ ) \begin{array}{l} T'(x’,y’)=T(x’,y’) – 1/(w \cdot h) \cdot \sum _{ x”,y”} T(x”,y”) \\ I'(x+x’,y+y’)=I(x+x’,y+y’) – 1/(w \cdot h) \cdot \sum _{x”,y”} I(x+x”,y+y”) \end{array}

with mask:


T ‘ ( x ‘ . y ‘ ) = M ( x ‘ . y ‘ ) ⋅ ( T ( x ‘ . y ‘ ) – 1 ∑ x ‘ ‘ . y ‘ ‘ M ( x ‘ ‘ . y ‘ ‘ ) ⋅ ∑ x ‘ ‘ . y ‘ ‘ ( T ( x ‘ ‘ . y ‘ ‘ ) ⋅ M ( x ‘ ‘ . y ‘ ‘ ) ) ) I ‘ ( x + x ‘ . y + y ‘ ) = M ( x ‘ . y ‘ ) ⋅ ( I ( x + x ‘ . y + y ‘ ) – 1 ∑ x ‘ ‘ . y ‘ ‘ M ( x ‘ ‘ . y ‘ ‘ ) ⋅ ∑ x ‘ ‘ . y ‘ ‘ ( I ( x + x ‘ ‘ . y + y ‘ ‘ ) ⋅ M ( x ‘ ‘ . y ‘ ‘ ) ) ) \begin{array}{l} T'(x’,y’)=M(x’,y’) \cdot \left( T(x’,y’) – \frac{1}{\sum _{x”,y”} M(x”,y”)} \cdot \sum _{x”,y”} (T(x”,y”) \cdot M(x”,y”)) \right) \\ I'(x+x’,y+y’)=M(x’,y’) \cdot \left( I(x+x’,y+y’) – \frac{1}{\sum _{x”,y”} M(x”,y”)} \cdot \sum _{x”,y”} (I(x+x”,y+y”) \cdot M(x”,y”)) \right) \end{array}

TM_CCOEFF_NORMED


R ( x . y ) = ∑ x ‘ . y ‘ ( T ‘ ( x ‘ . y ‘ ) ⋅ I ‘ ( x + x ‘ . y + y ‘ ) ) ∑ x ‘ . y ‘ T ‘ ( x ‘ . y ‘ ) 2 ⋅ ∑ x ‘ . y ‘ I ‘ ( x + x ‘ . y + y ‘ ) 2 R(x,y)= \frac{ \sum_{x’,y’} (T'(x’,y’) \cdot I'(x+x’,y+y’)) }{ \sqrt{\sum_{x’,y’}T'(x’,y’)^2 \cdot \sum_{x’,y’} I'(x+x’,y+y’)^2} }

operation

/** ** template matching * author: yidong * 2020/10/23 */
class MatchTemplateActivity : AppCompatActivity() {

    private lateinit var mBinding: ActivityMatchTemplateBinding
    private lateinit var mRgb: Mat
    private lateinit var mTemplate: Mat
    private var method = Imgproc.TM_SQDIFF
        set(value) {
            field = value
            doMatch(field)
        }

    override fun onCreate(savedInstanceState: Bundle?). {
        super.onCreate(savedInstanceState)
        mBinding = DataBindingUtil.setContentView(this, R.layout.activity_match_template)
        title = "TM_SQDIFF"
        val bgr = Utils.loadResource(this, R.drawable.kobe)
        mRgb = Mat()
        mTemplate = Mat()
        Imgproc.cvtColor(bgr, mRgb, Imgproc.COLOR_BGR2RGB)
        val templateBgr = Utils.loadResource(this, R.drawable.kobe_template)
        Imgproc.cvtColor(templateBgr, mTemplate, Imgproc.COLOR_BGR2RGB)
        mBinding.ivLena.showMat(mTemplate)
        doMatch(Imgproc.TM_CCOEFF)
    }

    private fun doMatch(method: Int) {
        val tmp = mRgb.clone()
        val result = Mat()
        Imgproc.matchTemplate(mRgb, mTemplate, result, method)
        val minMaxLoc = Core.minMaxLoc(result)
        val topLeft = if (method == Imgproc.TM_SQDIFF || method == Imgproc.TM_SQDIFF_NORMED) {
            minMaxLoc.minLoc;
        } else {
            minMaxLoc.maxLoc;
        }
        val rect = Rect(topLeft, Size(mTemplate.cols().toDouble(), mTemplate.rows().toDouble()))
        Imgproc.rectangle(tmp, rect, Scalar(255.0.0.0.0.0), 4, Imgproc.LINE_8)
        mBinding.ivResult.showMat(mRgb)
        tmp.release()
    }

    override fun onCreateOptionsMenu(menu: Menu?).: Boolean {
        menuInflater.inflate(R.menu.menu_match_template, menu)
        return true
    }

    override fun onOptionsItemSelected(item: MenuItem): Boolean {
        when (item.itemId) {
            R.id.match_tm_sqdiff -> {
                method = Imgproc.TM_SQDIFF
                title = "TM_SQDIFF"
            }
            R.id.match_tm_sqdiff_normed -> {
                method = Imgproc.TM_SQDIFF_NORMED
                title = "TM_SQDIFF_NORMED"
            }
            R.id.match_tm_ccoeff -> {
                method = Imgproc.TM_CCOEFF
                title = "TM_CCOEFF"
            }

            R.id.match_tm_ccoeff_normed -> {
                method = Imgproc.TM_CCOEFF_NORMED
                title = "TM_CCOEFF_NORMED"
            }
            R.id.match_tm_ccorr -> {
                method = Imgproc.TM_CCORR
                title = "TM_CCORR"
            }
            R.id.match_tm_ccorr_normed -> {
                method = Imgproc.TM_CCORR_NORMED
                title = "TM_CCORR_NORMED"}}return true
    }

    override fun onDestroy(a) {
        mTemplate.release()
        mRgb.release()
        super.onDestroy()
    }
}
Copy the code

The results of

The source code

Github.com/onlyloveyd/…