“This is the first day of my participation in the Gwen Challenge in November. Check out the details: The last Gwen Challenge in 2021”

Tooth uncle tutorial is easy to understand

The statement

This tutorial teaches you how to find transparent images. The logic for finding transparent images is similar and is not specific to any app.

The effect

Objective: Find the double head in the picture

Three resolutions were used in the test

  • 1600×720
  • 2310×1080
  • 2400×1080

Five for each resolution,

It can be seen that the head in the upper left corner of the figure is a black and white image, which is obtained by graying and binarization a large image of 2400×1080 and then cutting the two-person area. All the resolution images are found by using the same two-person head small image.

Is it necessary to read on

  • Do you have an idea for multi-resolution map finding
  • Would you look for a transparent graph
  • You’re looking for transparencies in multiple resolutions, right?
  • Do you want to read dozens of tutorials to learn SIFT
  • Will you change the Java version of SIFT to autoJS
  • Do you want to spend two weeks working on something

If you have no problem with any of the above, there’s no need to read on. Paddle away.

The environment

Mi 11 Pro Android version: 11 Autojs version: 9.0.10

note

Autojs8 comes with opencV version 3, which does not include SIFT. Autojs9 comes with OpencV version 4, which includes SIFT.

Multi – resolution transparent map ideas

Sift is used for multi-resolution image search, and the transparent image is binarized first, and then the image is found

Binary image

We need to select an appropriate threshold for binarization, so we need to visually adjust the threshold. The threshold selected in this tutorial is 246. Under this threshold, only the double head and the star in the lower right corner are left on the binarization image, which can help us eliminate the noise in the image to the maximum extent.

The following is the corresponding binary graph when the threshold is 246

Binary code

This code is very good, even at the UI interface refresh rate of 16ms, there are no memory leaks, worthy of careful study for beginners

"ui";
engines.all().map((ScriptEngine) = > {
  if (engines.myEngine().toString() !== ScriptEngine.toString()) {
    ScriptEngine.forceStop();
  }
});
importClass(org.opencv.imgproc.Imgproc);
importClass(org.opencv.core.Mat);
ui.layout(
  <vertical>
    <img id="Original"></img>
    <horizontal>
      <text>Threshold:</text>
      <text id="threshold">0</text>
      <seekbar id="seekbar" margin="9" w="*"></seekbar>
    </horizontal>
    <img id="Binarization"></img>
  </vertical>
);
let imgPath = ". / screenshot 1. PNG";
imgPath = files.path(imgPath);
let img = images.read(imgPath);
letbitmap = img.getBitmap(); log(bitmap); The UI. The original image. SetImageBitmap (bitmap); ui.seekbar.setMax(255);

let thresholdBitmap;
ui.seekbar.setOnSeekBarChangeListener({
  onProgressChanged: function (seekBar, progress, fromUser) {
    if (fromUser) {
      let lastBitmap = thresholdBitmap;
      ui.threshold.setText(String(progress));
      let threshold = parseInt(progress);
      log("Img =" before binarization + img);
      letThresholdImg = binary (img, threshold); thresholdBitmap = thresholdImg.getBitmap(); SetImageBitmap (thresholdBitmap); ui.post(function () {
        thresholdImg.recycle();
      });
      if (lastBitmap) {
        log("lastBitmap = "); log(lastBitmap); lastBitmap.recycle(); }}}}); events.on("exit".function () {
  bitmap.recycle();
  img.recycle();
  thresholdBitmap.recycle();
});

/ * -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- a custom function -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - * /

functionBinary (img, threshold) {
  log("threshold = " + threshold);
  let trainImage = img.mat;
  let trainImage_gray = new Mat();
  Imgproc.cvtColor(trainImage, trainImage_gray, Imgproc.COLOR_BGR2GRAY);
  let binary = new Mat();
  // Imgproc.threshold(trainImage_gray, binary, threshold, 255, Imgproc.THRESH_BINARY | Imgproc.THRESH_OTSU);
  Imgproc.threshold(trainImage_gray, binary, threshold, 255, Imgproc.THRESH_BINARY);
  trainImage_gray.release();
  var newImg = com.stardust.autojs.core.image.ImageWrapper.ofMat(binary);
  return newImg;
}
Copy the code

Cut out pictures

On the computer, I used PS to crop the picture. On the mobile phone, you can use an app with cropping function. If you can’t find a suitable app, you can reply on the official accountPhoto editorThis app has the cutting function. Below is the double head we cut out from the binary graph

Save pictures to SD card

Opencv image format is Mat, save Mat to SD card, the code is as follows

function viewMat(mat) { let mat2 = mat.clone(); Imgproc.cvtColor(mat, mat2, Imgproc.COLOR_BGRA2RGBA); Let tempFilePath = files.join(files.getsdcardPath (), "script "," mate.png "); let tempFilePath = files.join(files.getsdcardPath (), "script "," mate.png "); Imgcodecs.imwrite(tempFilePath, mat2); mat2.release(); app.viewFile(tempFilePath); }Copy the code

There is a line of color conversion code in the middle, because MAT uses BGR by default, instead of RGB. Without color conversion, the color of the picture will look abnormal.

You should have these by now

  • The original image
  • Binarization of the original
  • Cropped out of the head of two small drawing

Sift comes in next

Sift for image effect

The point at both ends of the color line connecting the double heads in the picture is the feature point with the highest similarity found by SIFT in the two pictures. There is also a feature point on the star in the lower right corner, so there is a purple line connected to it. The blue rectangle in the picture is used to wrap the similar feature points.

Let’s look at another picture

This picture is very good, not affected by the stars in the lower right corner, look at other pictures

As can be seen from the picture, the stars in the lower right corner affect our search. Therefore, when we search for the picture, we can first cut out the pictures on the right side that are useless for our search. For example, we can cut out one-third of the right side of the picture, so as to avoid the influence of the stars in the lower right corner on the search. All the useless areas can be trimmed down in advance and then go to the map

Therefore, my custom method has a region parameter that only keeps the parts that are useful to us, and the rest of the image is cropped out

/ * * *@description:
 * @param {img} bigImg
 * @param {img} smallImg
 * @param {Array} rect [left, top, right, bottom]
 * @return {Array} [left, top, right, bottom]
 */
function findImageSift(bigImg, smallImg, rect) {... }Copy the code

conclusion

These are the basic steps for finding transparent graphs. The theory of this method is feasible, but the actual effect is not known until we test it.

Does this approach fit any scenario? Of course, it is impossible. We will encounter different scenes, and we need to adopt appropriate methods for each scene. I don’t think there is a universal method for finding pictures, especially for mobile phone resolutions, but this scheme is the most appropriate scheme for this scene at present.

Learn this, your ability to find pictures is a big step forward, your ability to find pictures is about to be great, give yourself a thumbs-up!

Quotes.

Ideas are the most important, other Baidu, Bing, StackOverflow, Github, Android docs, AutoJS docs, and last but not least, ask in the group

The statement

This tutorial is intended for learning purposes only and is not intended for any other use