One, foreword

Question: What is a DEX file? What is DEX file optimization?

What problems will the dex file optimization bring to the project and how to solve these problems?

1.1 APK compilation and packaging process

1. Pack resource file RES with aapt and generate R.java, Resources.arsc and RES files (binary and non-binary code remain the same)

2. Process AIDL file and generate Java interface file (if there is no AIDL, ignore it)

3. Compile R.java and Java interface files with Java Compile to generate corresponding.class files (Java Compiler)

4. Use the dex command to convert.class files and.class files in the third party SDK library into classes.dex files

5. Use APKBuilder to pack CompiledResources and other resource files generated by APT and classes.dex files to generate APK

6. Again, you can debug or release the apk above using Jarsigner tool

The APK compilation and packaging flow chart is as follows:

In the actual project development, the requirements can be met through the two steps of 5 and 6, which can be directly generated with the release package of Jenkins platform.

1.2 Application scenarios of DEX file

The most popular scenarios are: APK thinning, hot fixes, pluginization, application hardening, Android reverse engineering, 64 K method limit.

Take the limit of method number as an example. In Step 4 above, when the class file is converted into a dex file, only one dex file will be generated by default. The number of methods in a single dex file shall not exceed 65536, otherwise the compilation will report an error. To fix this problem, use multiple DEXs instead of one. Gradle adds one line: multidexEnabled True.

There are many application scenarios of DEX file introduced on the Internet, but this paper does not introduce them. Instead, I analyzed the actual problems encountered in the project, so as to have a further understanding of DEX optimization.

II. DEX to VDEX and ODEX

2.1 ART pre-optimization

What is ART (Android Runtime)? It is a virtual machine, operating Java code, APP used. In reference to Android history, Android 5.0 uses ART as the default virtual machine, instead of the DVM (Dalvik VM) used in Android4.4 and before.

To review the relationship between DVM and ART and Android, let’s take a look at how several virtual machines running Java work: (1) JVM: JVM virtual machines operate in Java bytecode. Java-> Java Bytecode (class) -> Java Bytecode (JAR) -> Java Bytecode (JAR)

DVM: The DVM virtual machine parses the DEX bytecode executed. Java -> Java bytecode(class) -> dalvik bytecode(dex)

ART: The ART virtual machine executes local machine code. Java -> Java Bytecode (Class) -> Dalvik Bytecode (DEX) -> Optimized Android Runtime Machine Code (OAT)

It can be seen that the evolution from DVM to ART is actually a transition from Java files to virtual machine execution code. Comparatively speaking, ART is more than the process Of OAT. ART is compiled by AOT (Ahead Of Time), and when the application is installed for the first Time, bytecode is pre-compiled into machine code stored locally. DVM is compiled using JIT (just-in-time), and every Time the application is run, the bytecode needs to be converted to machine code by the compiler on the fly In order to continue execution. Compared with DVM, ART eliminates the need to parse bytecode every time, thus reducing the memory occupied at runtime and improving the running efficiency of the application.

2.2 Operation mode of ART

Art In the Android5.0 era, it was claimed that AOT could run the system on 512M machines. Starting with Android 7.0 (N for short), ART combines AOT, just-in-time (JIT) compilation, and profile-bootstrap compilation.

These modes can be configured in combination. For example, the Pixel device of Google is configured with the following compilation process:

1) No AOT compilation is done when the application is initially installed. The first few times the application is run, it is deciphered and the frequently executed methods are JIT compiled.

2) When the device is idle and charged, the compile daemon runs to AOT compile common code from configuration files generated during the previous runs of the application.

The next time the application is restarted, it will boot the code using the configuration file and avoid JIT compilation of already-compiled methods at run time. Methods that are JIT-compiled during subsequent runs of the application will be added to the configuration file, and the compilation daemon will then AOT compile these methods.

ART includes a compiler (the dex2oat tool) and a runtime (libart.so) loaded to start Zygote. The dex2oat tool takes an APK file and generates one or more compiled files, which are then loaded by the runtime. The number of files, extensions, and names will vary by version, but in Android O, the following files will be generated:

VDEX: This contains the uncompressed DEX code for APK, plus some metadata designed to speed up validation.

Odex: This contains AOT-compiled method code in APK.

ART (Optional) : This contains the ART internal representation of some of the strings and classes listed in APK to speed up application startup.

2.3 Functions of VDEX and ODEX

Unzip an APK package (using the vendor’s system application package as an example) and you can see the following structure without any dex files

If you look at the directory structure of the app on the phone, the vdex, odex files contain all the code for the apk and normally include classes.dex files as well. Because VDEX and ODEX are machine codes, they cannot be directly converted to the secondary code which can be viewed (it may be that the tools I use are wrong).

2.4 Relationship between VDEX, ODEX and classes.dex

It may be a bug in system compilation, or it may be a secondary processing of ODEX and VDEX files after generating ART files. The phenomenon is like this: try to get the DEX file in ODEX, but the prompt does not contain the DEX file.

In order to reconfirm whether the ODEX file really contains the DEX file, use 010Editor to reconfirm, you can see that there is still no DEX file under the recent Files.

Attempts to get the dex file in Vdex are also not available.

So it is incorrect to say that Odex (or Vdex) contains classes.

What is Arouter

Ali a routing component, many functions, my actual use of the scene is the page jump. For specific functions, you can refer to the introduction of AROUTER on Ali Summit.

Drawing on one of the things that came out of the summit, that’s what we’re going to talk about. “The last thing I want to share is the future development plans of Arouter. In the future, AROUTER will support pluginization and generate mapping documents, because pluginization is the technical solution used in many large apps. Many DEX and functions are dynamically sent underground to the APP, and in this case, it is impossible to find all the DEX files, that is, for the DEX files that have not been loaded. The mapping relationship is not transferable, so once the location of the Dex file changes, the normal solution will not be able to find the Dex or implement the mapping file initialization, this part will be supported in the later version.”

The AROUTER path that Ali can recognize is as follows:

In other words, arouter may not be found because of a change in the location of the dex file or a change in the path.

Fourth, hit the pit

4.1 the phenomenon

In 2.4, we mentioned that the ODEX file does not contain DEX, and the AROUTER search path follows the rules of grouping load on demand. In the end, it is actually a search of class files, as shown in the figure below:

The class file is recorded in the dex file, so there is a classNotFound Exception when using the AROUTER page to jump.

4.2 Solutions

To find a solution, you need to know how to make the ODEX not affect the AROUTER path. In this regard, it may be difficult to find a solution without relevant experience, but you can only look for it bit by bit. By searching the working principle of ART, I found the article “Configuring ART”, which mentioned:

If you configure the LOCAL\_DEX\_PREOPT attribute to prevent Odex optimization, set LOCAL\_DEX\_PREOPT := noustripping in Android.mk where the attribute is set.

Do dex optimizations at compile time (generate Odex files) without stripping dex from the APK. Here is the path comparison after apk generation, and then look at the path where dex is not stripped. Below is the classes.dex file.

When you open the classes.dex file with jadx, you find that the path file for arouter is there, so the jump for arouter is normal and the exception is no longer present.

Five, the summary

Odex optimizing this system do, often can appear some unexpected results, if you are in charge of the manufacturer’s application, often need to built projects, this time should pay attention to, when your application is contained in a third party framework, pay attention to the path, resource references are no problem, although under normal circumstances, odex file not to interfere with your path, However, Odex can also make mistakes, because for Odex, the resources in it do not need to be saved, and the generated ART file can be run. The proper use of the configuration of ART can help solve many problems.

Author: Vivo Internet Client Team -Xu Jie