OpenCV topic 1 - AndroidStudio's JNI project and references OpenCV

A sharp blade, not used properly, can hurt you all over. If you use it well, you will be invincible. Between good and bad, it is experience.

After many twists and turns, a new door has been opened with OpenCV integration and grayscale image implementation. At this point I have almost all the pieces of skill I need. In this article you include:

[1].OpenCV integration in AndroidStudio [2]. Analysis of the first JNI project [3]. The use of Bitmap class in Android in JNI [4]. A grayscale example opens the world of OpenCVCopy the code

NDK series:

[-NDK Guide -] Things you should know before NDK development
OpenCV topic 1 – AndroidStudio’s JNI project and references OpenCV
OpenCV Thematic 2 – Face detection + automatic sizing

1. Create a project

1.1: Download the SDK of OpenCV

First go to the official website to download OpenCV Android package

SDK -> native -> libs c++ code: SDK -> native -> jni -> include -> opencv2Copy the code

1.2: Create an Android`Native c++`The project of

The project structure is as follows

1.3: Run the first project

The result looks like this, with a line in the middle :”Hello from C++”

2.JNI initial project analysis

2.1: MainActivity analysis

Native lib is loaded in a static code block using the System.loadLibrary method

The native method stringFromJNI() returns a String and sets it to the TextView

---->[src/main/java/com/toly1994/rec/MainActivity.java]---- public class MainActivity extends AppCompatActivity { static  { System.loadLibrary("native-lib"); } @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); TextView tv = findViewById(R.id.sample_text); tv.setText(stringFromJNI()); } /** * A native method that is implemented by the 'native-lib' native library, * which is packaged with this application. */ public native String stringFromJNI(); }Copy the code

2.2:`native-lib.cpp`Analysis of the

Introduced jNI and string headers, a Java_com_toly1994_rec_MainActivity_stringFromJNI function

An sring variable is defined in the function body and a string is created via the env pointer and returned

#include <jni.h>
#include <string>

extern "C" JNIEXPORT jstring JNICALL
Java_com_toly1994_rec_MainActivity_stringFromJNI(
        JNIEnv *env,
        jobject /* this */) {
    std::string hello = "Hello from C++";
    return env->NewStringUTF(hello.c_str());
}

Copy the code

2.3: CMakeLists. TXT

Cmake_minimum_required (VERSION 3.4.1) # Generate native-lib add_library(native-lib SHARED) using native-lib. CPP Log-lib find_library(log-lib log) # set target to link library target_link_libraries(native lib) ${log-lib} )Copy the code

3. The integrated OpenCV

3.1: Library import and reference

Copy the required libraries and so packages into the project, along with the configuration of cmakelists.txt

# Specify the minimum VERSION of cmake cmake_minimum_required(VERSION 3.4.1) include_directories(include)# Import folder # compile header file # define global my_source_path variable file(GLOB my_source_path ${CMAKE_SOURCE_DIR}/*.cpp ${CMAKE_SOURCE_DIR}/*.c) add_library(tolyCV SHARED ${my_source_path}) Add_library (lib_opencv SHARED IMPORTED) set_target_properties(lib_opencV PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/.. ${ANDROID_ABI}/ ${ANDROID_ABI}/libopencv_java4.so Find_library (jnigrapics -lib jnigraphics) # set target to link to library target_link_libraries( tolyCV lib_opencv ${jnigraphics-lib} ${log-lib})Copy the code

3.2: The bug that almost ended my NDK career

dlopen failed: library "libc++_shared.so" not found

This bug was so nightmarish that I almost gave up on the NDK until five days later, I finally got the solution:

---->[app/build.gradle]---- android { defaultConfig { externalNativeBuild { cmake { cppFlags "" arguments "-dandroid_stl =c++ _shared.so}}Copy the code

3.3: Utility class for creating bitmap

Android Bitmap class cannot be operated directly in C++, so it needs to be converted to pixel matrix processing, which is written as a header file first.

In #include

, build –> Refresh Linked C++ Projects

---->[cpp/bitmap_utils.h]---- #ifndef REC_UTILS_H #define REC_UTILS_H #include <android/bitmap.h> #include <opencv2/opencv.hpp> using namespace cv; //Mat extern "C" {/** * Bitmap * @param env JNI * @param Bitmap * @param Mat image matrix * @param Void bitmap2Mat(JNIEnv *env, jobject bitmap, Mat, bool needPremultiplyAlpha = false); /** * matrix to Bitmap * @param env JNI * @Param mat image matrix * @param Bitmap Bitmap object * @param needPremultiplyAlpha whether to precede transparency */ void mat2Bitmap(JNIEnv *env, Mat mat, jobject bitmap, bool needPremultiplyAlpha = false); /** ** Create Bitmap * @param env JNI * @param SRC matrix * @param config Bitmap configuration * @return Bitmap object */ jobject createBitmap(JNIEnv *env, Mat src, jobject config); } #endif //REC_UTILS_HCopy the code

4.OpenCV to achieve grayscale pictures

4.1: The following is the concrete implementation of the three methods

Bitmap2Mat obtains the pixel matrix through bitmap and puts it into mat, so that mat can operate in C++

Mat2Bitmap is the opposite of the above, by putting the pixel information of the matrix into the MAT matrix, createBitmap obtains the object through reflection in Android createBitmap method, and then puts the information into mat2Bitmap.

#include "bitmap_utils.h" void bitmap2Mat(JNIEnv *env, jobject bitmap, Mat *mat, bool needPremultiplyAlpha) { AndroidBitmapInfo info; void *pixels = 0; Mat &dst = *mat; CV_Assert(AndroidBitmap_getInfo(env, bitmap, &info) >= 0); / / Bitmap information CV_Assert (info. The format = = ANDROID_BITMAP_FORMAT_RGBA_8888 RGBA_8888 / / picture format or RGB_565 | | info. The format = = ANDROID_BITMAP_FORMAT_RGB_565); CV_Assert(AndroidBitmap_lockPixels(env, bitmap, &pixels) >= 0); CV_Assert(pixels); dst.create(info.height, info.width, CV_8UC4); if (info.format == ANDROID_BITMAP_FORMAT_RGBA_8888) { Mat tmp(info.height, info.width, CV_8UC4, pixels); if (needPremultiplyAlpha) { cvtColor(tmp, dst, COLOR_mRGBA2RGBA); } else { tmp.copyTo(dst); } } else { Mat tmp(info.height, info.width, CV_8UC2, pixels); cvtColor(tmp, dst, COLOR_BGR5652RGBA); } AndroidBitmap_unlockPixels(env, bitmap); } void mat2Bitmap(JNIEnv *env, Mat mat, jobject bitmap,bool needPremultiplyAlpha) { AndroidBitmapInfo info; void *pixels = 0; CV_Assert(AndroidBitmap_getInfo(env, bitmap, &info) >= 0); / / Bitmap information CV_Assert (info. The format = = ANDROID_BITMAP_FORMAT_RGBA_8888 RGBA_8888 / / picture format or RGB_565 | | info. The format = = ANDROID_BITMAP_FORMAT_RGB_565); CV_Assert(mat.dims==2&&info.height==(uint32_t)mat.rows && info.width==(uint32_t)mat.cols); CV_Assert(mat.type()==CV_8UC1||mat.type()==CV_8UC3||mat.type()==CV_8UC4); CV_Assert(AndroidBitmap_lockPixels(env, bitmap, &pixels) >= 0); CV_Assert(pixels); if (info.format == ANDROID_BITMAP_FORMAT_RGBA_8888) { Mat tmp(info.height, info.width, CV_8UC4, pixels); switch (mat.type()){ case CV_8UC1: cvtColor(mat,tmp,COLOR_GRAY2RGBA); break; case CV_8UC3: cvtColor(mat,tmp,COLOR_RGB2RGBA); break; case CV_8UC4: cvtColor(mat,tmp,COLOR_RGBA2mRGBA); if (needPremultiplyAlpha) { cvtColor(mat, tmp, COLOR_RGBA2mRGBA); } else { mat.copyTo(tmp); } break; default:break; } } else { Mat tmp(info.height, info.width, CV_8UC2, pixels); switch (mat.type()){ case CV_8UC1: cvtColor(mat,tmp,COLOR_GRAY2BGR565); break; case CV_8UC3: cvtColor(mat,tmp,COLOR_RGB2BGR565); break; case CV_8UC4: cvtColor(mat,tmp,COLOR_RGBA2BGR565); break; default:break; } } AndroidBitmap_unlockPixels(env, bitmap); } jobject createBitmap(JNIEnv *env, Mat src, jobject config){ jclass java_bitmap_class=(jclass)env->FindClass("android/graphics/Bitmap"); / / the name of the class jmethodID mid = env - > GetStaticMethodID (java_bitmap_class, "createBitmap", / / access method "(IILandroid/graphics/Bitmap $Config) Landroid/graphics/Bitmap;" ); jobject bitmap=env->CallStaticObjectMethod(java_bitmap_class,mid,src.cols,src.rows,config); mat2Bitmap(env,src,bitmap, false); return bitmap; }Copy the code

4.2: Operations in MainActivity:

The layout is pretty simple, I’m not going to paste it, an iv_photo ImageView.

An opBitmap native method is called when clicked to grayscale the image.

public class MainActivity extends AppCompatActivity {
    static {
        System.loadLibrary("tolyCV");
    }
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        ImageView iv = findViewById(R.id.iv_photo);
        iv.setOnClickListener(v -> {
            tv.setText(stringFromJNI());
            Bitmap bitmap = BitmapFactory.decodeResource(getResources(), R.mipmap.wy_300x200);
            iv.setImageBitmap(opBitmap(bitmap,Bitmap.Config.ARGB_8888));
        });
    }
    public native Bitmap opBitmap(Bitmap bitmap, Bitmap.Config argb8888);
}
Copy the code

4.4: processing in C++ files

Put the pixel information of the picture in dstMat, and then use dstMat to create a Bitmap object. At this point, a logic will be smooth

---->[cpp/native-lib.cpp]--- #include <jni.h> #include <string> #include <opencv2/imgproc/types_c.h> #include "bitmap_utils.h" extern "C" JNIEXPORT jobject JNICALL Java_com_toly1994_rec_MainActivity_opBitmap(JNIEnv *env, jobject instance, jobject bitmap, jobject argb8888) { Mat srcMat; Mat dstMat; bitmap2Mat(env, bitmap, &srcMat); cvtColor(srcMat, dstMat, CV_BGR2GRAY); Return createBitmap(env,dstMat,argb8888); // Create a Bitmap with dstMat}Copy the code

With that, the end of this piece. Simplicity has a simple cost, complexity has a complex value.

OpenCV topic 1 – AndroidStudio’s JNI project and references OpenCV