1

What is JNI ?

JNI (Java Native Interface) is a channel that the JVM provides for upper-layer Java applications to invoke Native modules. We know that the Java language is a managed language that relies on parsing execution by the JVM virtual machine rather than running directly on the operating system, where the virtual machine is just a normal process.

JNI is an important mechanism for the JVM. It not only provides Java code development with the opportunity to call native modules, but also some important implementations of virtual machines themselves rely on JNI calls. Here the author has a look at the dependency library of the Java process in the system. It can be found that some important modules are also sealed into the dynamic link library through the JNI mechanism:

So why do Java processes use JNI technology? This question is the same as why processes need to use dynamic link libraries. There are two main reasons:

  1. From the perspective of software development, some modules are put into the dynamic link library, which facilitates the modular development and management of Java virtual machine, making the realization of virtual machine not so complicated and bloated, which conforms to the decoupling principle of software engineering. In the recent release of Java 9, one of the most important features is modularity. Although the author did not go into the implementation mechanism, it is assumed that JNI support is also necessary.

  2. Dynamically linked libraries help save memory and can make processes consume fewer memory resources. Dynamic link libraries (in the case of Windows Dynamic link Library DLLS) are loaded only once (or more accurately, once allocated via the VirtualAlloc API) if two processes on the system rely on the same dynamic link library. This greatly increases the memory usage of the system. If some key modules in Java are implemented directly in the VIRTUAL machine, the same modules will be loaded repeatedly, resulting in a waste of memory. Of course, PE and ELF involve a lot of content, not the focus of this article, want to learn more friends, you can see “Programmer self-cultivation” book.

In the Java development of traditional projects, JNI technology is rarely used. Basically, Java layer can do all things, while JNI technology is generally used in security reinforcement. For example, many APKS on the market put their encryption and decryption functions, and even the encapsulation of protocol packages, into the dynamic link library, which improves the difficulty of cracking.

2

Use and development of JNI

The development of JNI is divided into two parts, one is the Java layer for the definition of the call interface, and another important part is the C/C++ dynamic link library for the implementation of the interface (interface here does not refer to the Java interface, but actually declare the class). Native keyword declarations are required for Java parts of the interface declaration, and must be declared as static methods:

After that, you can use the Javah tool to generate the C/C++ project reference function definition header file based on the declaration class as follows:

Javah generates a function definition header based on the package name, class name, and method name defined by the Java interface, and this function is declared as an export function of the dynamic link library:

Once the header file is generated, you can reference it directly in your C/C++ project to develop a dynamic link library.

3

Dynamic link library for JNI implementation

Because dynamically linked libraries are developed in C/C++, they do not have cross-platform features. In Windows, dynamic link libraries are DLL files in PE file format, while in Linux (Android), they are ELF so files. The compilation of dynamic link library under Windows platform, need to specify /DLL switch for compiler during compilation, also can use Visual Stuido directly create a DLL project for development, dynamic link library project attribute configuration is as follows:

The specific development process is not discussed in this paper. What functions are developed are determined by project requirements. Here the author lists some problems that need to be paid attention to in the development process of dynamic link library:

1. The platform version of Dll is faulty. On 32-bit operating systems, 32-bit processes (x86) can use no more than 4GB of memory space, and the higher 2GB memory address is still occupied by the operating system kernel (default: 1:1), so 4GB memory addresses are often insufficient for today’s large systems. In order for processes to use more than 4GB of memory, modern cpus support 64-bit operating systems, but operating systems are generally compatible with previous 32-bit software. 64-bit versions of Windows provide support for 32-bit processes through Wow64 technology. However, for the development of dynamic link libraries, programmers still need to distinguish between the platform version of the dynamic link library, because the 64-bit process cannot load the 32-bit dynamic link library, so the Java process must be consistent with the PLATFORM of the JNI dynamic link library. However, there is no need for programmers to do additional development work (unless inline assembly is used) to distinguish between 32-bit and 64-bit, just set up the compiler platform type support, such as Visual Studio only need to add support for the corresponding platform.

2. Dynamic LINK and static LINK of DLL. We develop DLL is actually made up of Java processes dynamically linked into the memory space of function modules, the difference between dynamic and static link lies in the fact that the target code (instruction) is directly included in the executable file, it contains only the interface with Java development, and put the jar package in Java process load directory to realize the library calls, Or directly package the library JAR package into the generated package. DLL DLL (msvcrXXx. DLL) may be missing from the Java process under different versions of Windows. This is equivalent to Java ClassNotFound. So the safe and sound approach is to generate DLLS directly using static links. But if you’re really sure about the runtime library version of the Java runtime platform, and you want to keep the runtime memory to a minimum, you can build dynamically linked.

3. Cross-platform development issues. JNI dynamically linked libraries need to be aware of cross-platform development issues, and it would be nice to have a set of code that supports multiple platforms. So how do we do that? First, try to avoid using platform-dependent system apis. Functions that can use the C++ standard runtime library should be implemented using the runtime library. If you must use platform-specific system apis, you can use the _WIN32 macro to do code-specific compilation. Here the author gives an example. In jni_Md. h, JNIEXPORT and JNIIMPORT function export macros have different declarations on different platforms, but the declarations are not supported by cross-platform compilation. Therefore, the author has modified jni_Md. h in the project. The Linux and Windows platform function export declarations are put together, and the _WIN32 macro is used to differentiate the platform version compilation, as follows:

4

Principle of JNI loading in VMS

After the dynamic link library has been developed and successfully compiled to generate an executable, you can introduce a Java project and make calls through JNI.

If you are familiar with C/C++ development, you must know that when you use a dynamic link library in a process, you need to load the dynamic link library into the memory space of the process through the LoadLibrary() API, and call the function through the function pointer and the function address. In Java, the link library is loaded by the system.loadLibrary () static method. After loading, the link library can be called through the previously declared native interface.

The key implementation of JNI loading is in the system.loadLibrary () function, which is a native implementation of the JVM by OS ::dll_load() method, which is also the System API to load dynamically linked libraries. It’s just that different platforms call different system apis, but the principles are similar (by the way, the JVM source code is really a good example for cross-platform C++ projects) :

So, JNI is a system API that the JVM encapsulates for the Java language for process calls to dynamically linked libraries. In section 2, we explained the naming of functions for C/C++ headers generated by Javah, but there is one other thing that needs to be noted: functions are declared as exported functions through the JNIEXPORT macro. Why are exported functions declared?

From the perspective of a Java programmer, here is what the PE(ELF) file export function is. Exported functions are equivalent to public methods in Java packages and can be invoked by other Java applications by reference. In the PE(ELF) file, the names and addresses of functions declared as exported functions are filled out in the PE(ELF) file’s exported table at compile time, while other processes can only call functions in the exported table when referencing the DLL (unless using hacks). The dynamic link library export table is as follows:

You can see that the function names in these exported tables are exactly the same as those in the header files generated by the Javah tool. The last issue that needs to be discussed is the invocation of native methods, which is actually implemented by JVM. From the perspective of process, method invocation is actually a jump (JMP) of address register (ESI) executed by CPU in instruction memory space. When Java calls native methods, Note that the address in the exported table is not the memory address of the function, but the offset address of the file. After loading the function into memory, you also need to add the mirror base address to find the memory address of the function. Of course, the operating system has provided a ready-made API to find the function address, which will not be explained in detail here. The specific implementation on different platforms is shown in the figure below:

5
Debugging of JNI

JNI debugging is actually source debugging technology, but because the caller is a Java process, in Idea, eclipse seems to have no way to directly from the Java JNI interface into the C++ implementation function for debugging (Android studio NDK can debug, But compared to other C++ source debuggers), so we really can’t debug? Of course not. In fact, when using VS compilation to generate binary files, the compiler generates a symbol file (*.pdb file). As long as the debugger loads the PDB file during debugging, the debugger matches the symbol file with the source code, thus implementing source debugging. IDEA + Visual Studio for JNI project debugging

  1. First make sure that the PDB file is generated in the C/C++ project compilation configuration:

  2. Turn off compiler optimization options for debugging:

  3. After compiling, place the resulting XXX.dll file in the JNI load path and run the Java caller (preferably next breakpoint in the Java caller first). Then use the ProcessHacker tool to find the CALLING process PID of JNI. Note that this PID must be found correctly. Some Java processes in the system are not the calling process.

  4. Open Visual Studio, and open the DLL source project, and use Debug -> Attach Process to Debug:

  5. Find the Java process whose PID is the same as the Java calling process’S PID in the process list and click Attache:

  6. After loading DLL and running to the breakpoint set in C++, the Java process will be broken, and then you can debug the source code, as shown below:

C++ source debugging, must pay attention to the PDB file and the generated dynamic link library version, otherwise Visual studio will report to find the symbol file problem, of course, does not mean that no symbol file you can not do binary debugging, you can use assembly debugging, ha ha!

6

Cross-platform JNI encryption library sharing and open source

After consultation with YrZX404, the author of this article decided to open source a set of cross-platform JNI encryption and decryption projects used in the project, which is a real JNI DEMO project in the product. In Windows, Android, Ubuntu platform actual test stable running, thread safety. Interested partners can learn, if need be free to change. The encryption and decryption function in the project is transplanted from the open source OpenSSL project by the author, so there is no need to worry about the cross-platform encryption algorithm. At present, the DEMO project only transplants AES encryption algorithm. If partners need other encryption algorithms or interfaces, they can write their own, or submit an issue to us on Github. If there is time, the author will definitely meet the needs of partners. In addition, the project reade. ME is also the essence, recording the author in the cross-platform JNI development encountered some pits and other platform development and compilation methods, I suggest partners can browse. The final project comes with a Windows PE file format picture!

Finally, thanks to those of you who kept reading this, many Java developers don’t think it’s necessary to think about such low-level stuff. Indeed, the Java ecosystem is so powerful that today’s Java developers can focus on the upper layers, and in practice, if JNI technology is required, it must be implemented by professional C/C++ programmers. However, I always believe that all systems are the same. If you have enough energy, you will have more perspectives and ideas to look at problems. Even if we read ten thousand books, we still need to travel thousands of miles!

Tip: we provide the corresponding code example for small partners, you can read the original to download.