• The time it might take to read this article

    15 minutes

  • Basic use of CMake 2. Android NDK development/use 3. JNI layer manipulation of Java objects

  • Android uses C/C++ to achieve the frosted glass effect of pictures.

  • Note:

    1. The research object of this paper is Android JNI/NDK development, not image algorithm, so it will not explain the frosted glass algorithm.

    2. My ability is limited, if there is anything wrong, please point out.

preface

After a few days of thinking during the National Holiday, I have established my own advanced direction. I plan to learn about computer vision technology, namely OpencV. The integration of OpencV in Android is bound to master the development of JNI/NDK, so I wrote this article, one is to share my learning experience with you, the other is to consolidate my knowledge of JNI/NDK development and ABANDONED C/C++ for a long time.

CMake

CMake is a project-building tool that generates makefiles by writing straightforward instructions in cmakelists. TXT. It is simply a makefile generator.

To install CMake in Android Studio, go to Tools->Android->SDK Manager, select the SDK Tools TAB, check CMake, LLDB, NDK, and OK. LLDB allows us to debug C/C++ applications in Android Studio. The NDK is a native development kit and is essential.

Why do JNI/NDK development

As we all know, Java/Android programs run in the JVM/Dalvik VM, so Java programs are far less high-performance than C/C++ programs, especially in cpu-intensive operations, so the Java platform provides JNI (Java Native Interface). So dynamic link library written by C/C++ can be called through JNI. Note: Google completely replaced Dalvik VM with ART after Android L, but ART is still a virtual machine in essence and supports all Dalvik VM instruction sets. Almost all hardware-related methods in the Java API are native, such as I/O operations, network access, cell phone sensors, serial port reading and writing, etc. The image processing covered in this article is a CPU-intensive task that is best suited for Android development using native methods.

How to use CMake to do JNI/NDK development

1 New Construction project


2 configure C++ support


3 know CMakeLists. TXT

After the project is created, Android Studio will generate the cmakelists. TXT file in the app directory. Cmakelists. TXT is a configuration file for CMake, used to indicate versions, dependencies, etc.

Cmake_minimum_required (VERSION 3.4.1) add_library(native-lib SHARED SRC /main/ CPP /native-lib.cpp) find_library(log-lib log)

target_link_libraries(native-lib ${log-lib})Copy the code
  • Cmake_minimum_required (VERSION 3.4.1) The minimum VERSION of CMake uses 3.4.1.
  • add_library()

    Configure so library information (add library for current current script file)
    • Native-lib This is the name of the so library that is declared to reference. In your project, if you need to use the so file, this is the name of the reference. It is worth noting that the name of the generated so file is actually libnative-lib.
    • SHARED

      This parameter indicates that the so library file will be shared in the directory during Run or build projectsintermediates\transforms\mergeJniLibs\debug\folders\2000\1f\mainGenerate the so file.
    • SRC /main/ CPP /native-lib. CPP build so library source file.
  • find_library()

    Find a library file
    • Log-lib specifies that each type of library is stored in a specific location in the NDK library, and the log store is stored in log-lib
    • Log Specifies the use of the log library
  • target_link_libraries()

    If your native-lib wants to call the log library’s methods, then you need to configure this property, which means to associate the NDK library with the local library.
    • Native-lib Specifies the name of the library to be associated with
    • ${log-lib} specifies the name of the library to associate with, wrapped in curly braces and preceded by a $symbol to reference it.

Understand the C/C++ specification of JNI

The data type

JNI contains two types of data types, basic type and reference type. The mapping between them and data types in Java is shown in the following two tables.

Basic data types
JNI type Java type describe
jboolean boolean Unsigned 8-bit integer
jbyte byte Unsigned 8-bit integer
jchar char Unsigned 16-bit integer
jshort short Signed 16-bit integer type
jint int A 32-bit integer
jlong long A 64 – bit integer
jfloat float 32-bit floating-point type
jdouble double 64-bit floating point type
void void No type
Reference data type
JNI type Java type describe
jobject Object The Object type
jclass Class The Class type
jstring String Type String
jobjectArray Object[] An array of objects
jbooleanArray boolean[] Boolean array
jbyteArray byte[] Byte array
jcharArray char[] Char array
jshortArray short[] Short array
jintArray int[] An array of int
jlongArray long[] Long array
jfloatArray float[] A float array
jdoubleArray double[] A double array
jthrowable Throwable Throwable

JNI type signature

JNI’s type signature identifies a specific Java type, which can be either a class, method, or data type.

  • Class signature is relatively simple, it uses L+ package name + type +; In the form of, just put the. Replace it with /. For example, java.lang.String, whose signature is Ljava/lang/String; , pay attention to the end; It’s part of the signature.
  • The signature of the basic data type is represented by a series of uppercase letters, as shown in the following table
The signature of the base data type
Java type The signature Java type The signature Java type The signature
boolean Z byte B char C
short S int I long J
float F double D void V

JNI C/C++ function preparation

Let’s start with an example Android Studio has generated for us

JNIEXPORT jstring JNICALL
Java_com_glee_myapplication_MainActivity_stringFromJNI(
        JNIEnv* env,
        jobject /* this */) {
    std::string hello = "Hello from C++";
    return env->NewStringUTF(hello.c_str());
}Copy the code
  • The two macros JNIEXPORT and JNICALL (defined in jni.h) ensure that this function is visible outside the local library and that the compiler calls the conversion correctly.
  • The function specification

    In JNI C/C++ function names are required by the specification and are concatenated by the following sections
    • Java_ prefix
    • Fully qualified class name, delimited by an underscore "_"
    • The first parameter JNIEnv* env
    • The second argument jobject or jclass
    • Other parameters are mapped by type
    • Returns parameter mapping by type

The JNI layer operates on Bitmap objects

The principle of

The JNI layer in Android typically handles bitmaps in one of two ways

  • The Byte array in the Bitmap is obtained and passed to the Native method. The JNI layer returns a new byte array after processing the byte array. The Java layer reconstructs the Bitmap object. (Not recommended)
  • The Java layer directly passes a reference to the Bitmap to the JNI layer, which gets the address of the image data of the Bitmap object and directly modifies the Byte array of the Bitmap.

Having read many blogs, many developers adopt the first approach, which I highly recommend. This method recreates a byte array in memory, resulting in memory waste and poor performance. The second method is the best performance. JNI layer makes full use of the C/C++ pointer feature to directly obtain the byte array address in memory in Bitmap and modify image data directly through the pointer, so Android /bitmap.h in NDK is used.

android/bitmap.h

The android/bitmap.h header is used to manipulate bitmap objects at the JNI layer. It is included in the jnigraphics library, so add -ljnigraphics to target_link_libraries in cmakelists.txt, as shown below

target_link_libraries(native-lib -ljnigraphics ${log-lib})Copy the code

Three common functions

  • AndroidBitmap_getInfo() gets information from bitmap handles (width, height, pixel format)
  • AndroidBitmap_lockPixels() Locks the pixel cache, that is, gets a pointer to that cache.
  • AndroidBitmap_unlockPixels unlock ()

JNI interface functions

Please see comments

JNIEXPORT void JNICALL Java_com_glee_ndkroad1006_MainActivity_gaussBlur(JNIEnv *env, jobject /* this */, jobject bmp) { AndroidBitmapInfo info = {0}; Int *data=NULL; AndroidBitmap_getInfo(env, BMP, &info); AndroidBitmap_getInfo(env, BMP, &info); AndroidBitmap_lockPixels(env, bmp, (void **) &data); / / lock Bitmap, and gain a pointer / * * * * * * * * * * a gaussian blur algorithm against an int array processing * * * * * * * * * * * / / / call gaussBlur function, The image data pointer, picture width and radius of fuzzy incoming gaussBlur (data and info. Width, info. Height, 80); /****************************************************/ AndroidBitmap_unlockPixels(env,bmp); / / unlock}Copy the code

The gaussBlur function code used here is listed at the end of this article. The gaussBlur function code used here is listed at the end of this article. The gaussBlur function code used here is listed at the end of this article.

Java layer code

Please see comments

Public class MainActivity extends appactivity {static {// Load the so library system.loadLibrary ("native-lib");
    }

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main); ImageView iv1 = (ImageView) findViewById(R.i.img1); ImageView iv2 = (ImageView) findViewById(R.id.img2); Iv1. setImageResource(r.rawable.test); / / generated bitmap object bitmap bitmap. = BitmapFactory decodeResource (getResources (), R.d rawable. Test); // Call the native method, pass in the Bitmap object, and perform gaussBlur(Bitmap) on the Bitmap. // Set the Bitmap object to iv2 iv2.setimageBitmap (Bitmap); } //native method declaration public native void gaussBlur(Bitmap Bitmap); }Copy the code

Running effect

The upper ImageView is not Gaussian blurred, and the lower ImageView calls the JNI method for Gaussian blur.

Gaussian fuzzy algorithm

void gaussBlur1(int* pix, int w, int h, int radius)
{
    float sigma = (float) (1.0 * radius / 2.57);
    float deno  = (float) (1.0 / (sigma * sqrt(2.0 * PI)));
    float nume  = (float) (-1.0 / (2.0 * sigma * sigma));
    float* gaussMatrix = (float*)malloc(sizeof(float)* (radius + radius + 1));
    floatGaussSum = 0.0;for (int i = 0, x = -radius; x <= radius; ++x, ++i)
    {
        float g = (float) (deno * exp(1.0 * nume * x * x));
        gaussMatrix[i] = g;
        gaussSum += g;
    }
    int len = radius + radius + 1;
    for (int i = 0; i < len; ++i)
        gaussMatrix[i] /= gaussSum;
    int* rowData  = (int*)malloc(w * sizeof(int));
    int* listData = (int*)malloc(h * sizeof(int));
    for (int y = 0; y < h; ++y)
    {
        memcpy(rowData, pix + y * w, sizeof(int) * w);
        for (int x = 0; x < w; ++x)
        {
            float r = 0, g = 0, b = 0;
            gaussSum = 0;
            for (int i = -radius; i <= radius; ++i)
            {
                int k = x + i;
                if(0 <= k && k <= w) {// Get the RGB value of the pixel int color = rowData[k]; int cr = (color & 0x00ff0000) >> 16; int cg = (color & 0x0000ff00) >> 8; int cb = (color & 0x000000ff); r += cr * gaussMatrix[i + radius]; g += cg * gaussMatrix[i + radius]; b += cb * gaussMatrix[i + radius]; gaussSum += gaussMatrix[i + radius]; } } int cr = (int)(r / gaussSum); int cg = (int)(g / gaussSum); int cb = (int)(b / gaussSum); pix[y * w + x] = cr << 16 | cg << 8 | cb | 0xff000000; }}for (int x = 0; x < w; ++x)
    {
        for (int y = 0; y < h; ++y)
            listData[y] = pix[y * w + x];
        for (int y = 0; y < h; ++y)
        {
            float r = 0, g = 0, b = 0;
            gaussSum = 0;
            for (int j = -radius; j <= radius; ++j)
            {
                int k = y + j;
                if (0 <= k && k <= h)
                {
                    int color = listData[k];
                    int cr = (color & 0x00ff0000) >> 16;
                    int cg = (color & 0x0000ff00) >> 8;
                    int cb = (color & 0x000000ff);
                    r += cr * gaussMatrix[j + radius];
                    g += cg * gaussMatrix[j + radius];
                    b += cb * gaussMatrix[j + radius];
                    gaussSum += gaussMatrix[j + radius];
                }
            }
            int cr = (int)(r / gaussSum);
            int cg = (int)(g / gaussSum);
            int cb = (int)(b / gaussSum);
            pix[y * w + x] = cr << 16 | cg << 8 | cb | 0xff000000;
        }
    }
    free(gaussMatrix);
    free(rowData);
    free(listData);
}Copy the code

After the