“This is the first day of my participation in the First Challenge 2022. For details: First Challenge 2022.”

An overview of the

OpenCV is a cross-platform computer vision and machine learning software library distributed under the Apache2.0 license (open source) and can run on Windows, Android and other operating systems. Can realize the image processing and other aspects of the general algorithm. address

OpeCV is a resource plug-in that can convert Texture2D and OpenCV’s Mat into each other in Unity. It can realize real-time image processing and face recognition. The website address

This article describes how to use OpeCV for Unity to get the color of a pixel in an image. OpenCV for Unity plugin version 2.3.3 (found in Baidu share resources, need resources can leave a comment below (including the source code of this article).

Results the preview

Notice the color bar effect 1 on the right.

Results 2

Function implementation

Functional analysis

This article aims to achieve the following functions: In the Unity scene, there is an Image Image, click the mouse to the corresponding position of the Image to obtain the color of the pixel. It’s actually reading the local image file, not reading the file in the scene, and the file in the scene is just used as a display to let us know where to click and what color is needed.

Scenario building

The first step is to set up the scene, which is also very simple. Create an Image Image in Unity and put the Image needed to be identified into the scene. This article only needs a picture in the scene for mouse click to obtain the color, as shown below:

The code

One thing to note when calculating pixels in OpenCV is that their positions are different from those in Unity. It starts at the top left corner (0,0) and ends at the bottom right corner, as shown in figure 1 below. The dimensions used are 1920*1080.

Note: do not use the dot in the lower right corner (1920*1080). Use the dot in the lower right corner (nineteen nineteen,1079) by one pixel, otherwise the program will be empty (see figure 2 below).

Figure 1

Figure 2

Since the pixel calculation is different, input. mousePosition in Unity is from the bottom left to the top right, so the pixel conversion takes place first.

// </param name="vector"></param> // <returns></returns> public static Vector2 Vector3ToPixelLocation(Vector3 vector) { Debug.Log(vector); float x = vector.x; float y = Screen.height-vector.y; if (x >= Screen.width) x = Screen.width - 1; if (y >= Screen.height) y = Screen.height - 1; return new Vector2(x, y); }Copy the code

The next step is to get the color of the corresponding pixels through OpenCV

// </summary> void CheckColorForOpencv() {Mat SRC =Imgcodecs.imread(Application.dataPath + "/Texture/TestTexture.png"); Mat dst = new Mat(src.cols(), src.rows(), CvType.CV_8UC4); Imgproc.cvtColor(src,dst,Imgproc.COLOR_BGR2BGRA); double b = dst.get((int) point.y, (int) point.x)[0]; double g = dst.get((int) point.y, (int) point.x)[1]; double r = dst.get((int) point.y, (int) point.x)[2]; double a = dst.get((int) point.y, (int) point.x)[3]; color = new Color((float) r / 255, (float) g / 255, (float) b / 255, (float) a / 255); }Copy the code

In the above code we notice that there is a Mat class, which is one of the most important classes in OpenCV. When we use OpenCV to process image-related operations, we almost always use the Mat class. It is the basic container for storing image data in OpenCV. It can store vectors and matrices of real/complex numbers, monochromatic or color images, etc. So how do you create Mat? The following is used

Mat Mat = new Mat(row number, column number, type);Copy the code

In this way, you can basically get the color of the corresponding pixel of the image.

Write in the last

All the shared content is the author used in the daily development process of a variety of small function points, sharing is also a way to review, if there is a bad place to write, please give more advice. Welcome to learn from each other and make progress. This piece article is written here first, hope to be able to help you