Android D8 compiler and R8 tools

This is the 13th day of my participation in Gwen Challenge

The Android installation Package suffix is.apk, which stands for Android Package. In the process of APK packaging and compilation, it involves the Javac tool to compile.java files into.class files, and then the.class files are desugred and packaged into.dex files by the dex tool.

javac: used to.javaThe file is compiled to.classFile;
Desugar: For adapting Java 8 features to the Android platform;
ProGuard: Used to bring up useless Java code and do some optimizations;
DX: Convert all Java code to DEX format.

After Android Studio 3.X, Google introduced the D8 compiler and R8 tools as the new DEX compiler and obfuscating compression tools, respectively.

1. Development history of D8 and R8

Android Studio version	Android Gradle Plugin version	change
3.1	3.0.1	The introduction of the D8
3.2	3.2.0	Introduction of R8, D8 desugarization is enabled by default
3.4	3.4.0	R8 is enabled by default

1.1 D8 compiler

Google introduced the D8 compiler in Android Studio 3.1 as the default DEX bytecode file compiler. Enable the D8 compiler by adding android.enableD8=true to gradle.properties.

The D8 compiler features:

Compiling is faster and takes less time;
DEXLess footprint at compile time;
.dexSmaller file sizes;
D8Compilation of.dexFiles with equal or better runtime performance;

According to theGoogle AndroidThe team usedDex 与 D8Compilers test comparison data:

1.2 the R8 tools

Google introduced R8 in Android Studio 3.2 as an alternative to ProGuard for shrinking and obfuscation code. Enable the R8 tool by adding android.enableR8 = true to gradle.properties.

# Disables R8 for Android Library modules only.
android.enableR8.libraries = false
# Disables R8 for all modules.
android.enableR8 = false
Copy the code

1.3 Android Studio version 3.4 D8 R8 changed

In Android Studio version 3.4, R8 combines desugaring, Shrinking, Obfuscating, Optimizing, and dexing into one step. Before Android Studio 3.4, the compilation process is as follows:

The compilation process is as follows:

Note that if we set useProguard = false in build.gradle, R8 will be used to compress the code regardless of whether R8 compilation is enabled.

2. Take off the candy

Android Studio provides built-in support for using some of the Java 8 language features and third-party libraries that take advantage of them. The default toolchain performs bytecode conversion (called desugar) on the output of the JavAC compiler to implement the new language functionality.

Desugaring refers to converting syntactically unsupported features of the underlying bytecode into basic bytecode structures at compile time (for example, the generic type on List is actually Object after desugaring at bytecode level). The Android toolchain is a rich process of desugaring the syntax features of Java8, but their ultimate goal is the same: to make the new syntax work on all devices.

As D8 and R8 have learned above, D8 already supports deicing, allowing Java 8 features (such as lambdas) to be converted to Java 7 features. Integrating the desugar step into D8 affects all development tools that read or write.class bytecodes, because it uses the Java 8 format. Deicing can be disabled in gradle.properties.

android.enableIncrementalDesugaring=false.
android.enableDesugar=false
Copy the code

2.1 Lambda Expression

One major change in Java 8 is the introduction of Lambda expressions.

public class Lambda {
    public static void main(String[] args) {
        logDebug(msg-> System.out.println(msg), "HelloWorld");
    }

    static void logDebug(Logger logger, String msg) {
        logger.log(msg);
    }

    @FunctionalInterface
    interface Logger {
        void log(String msg); }}Copy the code

Lambda.java above is converted to bytecode via the Javac directive.

$javac Lambda.java
Copy the code

Next, view the bytecode details using the Javap -v directive:

dengshiweideMacBook-Pro:Desktop dengshiwei$ javap -v Lambda.class
Classfile /Users/dengshiwei/Desktop/Lambda.class
  Last modified 2019-5-12; size 1120 bytes
  MD5 checksum 67301305720e60d4ef1ff286769768e6
  Compiled from "Lambda.java"
public class Lambda
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Methodref          #9.#25         // java/lang/Object."<init>":()V
   #2 = InvokeDynamic      #0:#30         // #0:log:()LLambda$Logger;
   #3 = String             #31            // HelloWorld
   #4 = Methodref          #8.#32         // Lambda.logDebug:(LLambda$Logger;Ljava/lang/String;)V
   #5 = InterfaceMethodref #10.#33        // Lambda$Logger.log:(Ljava/lang/String;)V
   #6 = Fieldref           #34.#35        // java/lang/System.out:Ljava/io/PrintStream;
   #7 = Methodref          #36.#37        // java/io/PrintStream.println:(Ljava/lang/String;)V
   #8 = Class              #38            // Lambda
   #9 = Class              #39            // java/lang/Object
  #10 = Class              #40            // Lambda$Logger
  #11 = Utf8               Logger
  #12 = Utf8               InnerClasses
  #13 = Utf8               <init>
  #14 = Utf8               ()V
  #15 = Utf8               Code
  #16 = Utf8               LineNumberTable
  #17 = Utf8               main
  #18 = Utf8               ([Ljava/lang/String;)V
  #19 = Utf8               logDebug
  #20 = Utf8               (LLambda$Logger;Ljava/lang/String;)V
  #21 = Utf8               lambda$main$0
  #22 = Utf8               (Ljava/lang/String;)V
  #23 = Utf8               SourceFile
  #24 = Utf8               Lambda.java
  #25 = NameAndType        #13:#14        // "<init>":()V
  #26 = Utf8               BootstrapMethods
  #27 = MethodHandle       #6:#41         // invokestatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
  #28 = MethodType         #22            //  (Ljava/lang/String;)V
  #29 = MethodHandle       #6:#42         // invokestatic Lambda.lambda$main$0:(Ljava/lang/String;)V
  #30 = NameAndType        #43:#44        // log:()LLambda$Logger;
  #31 = Utf8               HelloWorld
  #32 = NameAndType        #19:#20        // logDebug:(LLambda$Logger;Ljava/lang/String;)V
  #33 = NameAndType        #43:#22        // log:(Ljava/lang/String;)V
  #34 = Class              #45            // java/lang/System
  #35 = NameAndType        #46:#47        // out:Ljava/io/PrintStream;
  #36 = Class              #48            // java/io/PrintStream
  #37 = NameAndType        #49:#22        // println:(Ljava/lang/String;)V
  #38 = Utf8               Lambda
  #39 = Utf8               java/lang/Object
  #40 = Utf8               Lambda$Logger
  #41 = Methodref          #50.#51        // java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
  #42 = Methodref          #8.#52         // Lambda.lambda$main$0:(Ljava/lang/String;)V
  #43 = Utf8               log
  #44 = Utf8               ()LLambda$Logger;
  #45 = Utf8               java/lang/System
  #46 = Utf8               out
  #47 = Utf8               Ljava/io/PrintStream;
  #48 = Utf8               java/io/PrintStream
  #49 = Utf8               println
  #50 = Class              #53            // java/lang/invoke/LambdaMetafactory
  #51 = NameAndType        #54:#57        // metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
  #52 = NameAndType        #21:#22        // lambda$main$0:(Ljava/lang/String;)V
  #53 = Utf8               java/lang/invoke/LambdaMetafactory
  #54 = Utf8               metafactory
  #55 = Class              #59            // java/lang/invoke/MethodHandles$Lookup
  #56 = Utf8               Lookup
  #57 = Utf8               (Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
  #58 = Class              #60            // java/lang/invoke/MethodHandles
  #59 = Utf8               java/lang/invoke/MethodHandles$Lookup
  #60 = Utf8               java/lang/invoke/MethodHandles
{
  public Lambda();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 1: 0

  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=1, args_size=1
         0: invokedynamic #2,  0              // InvokeDynamic #0:log:()LLambda$Logger;
         5: ldc           #3                  // String HelloWorld
         7: invokestatic  #4                  // Method logDebug:(LLambda$Logger;Ljava/lang/String;)V
        10: return
      LineNumberTable:
        line 3: 0
        line 4: 10

  static void logDebug(Lambda$Logger, java.lang.String);
    descriptor: (LLambda$Logger;Ljava/lang/String;)V
    flags: ACC_STATIC
    Code:
      stack=2, locals=2, args_size=2
         0: aload_0
         1: aload_1
         2: invokeinterface #5,  2            // InterfaceMethod Lambda$Logger.log:(Ljava/lang/String;)V
         7: return
      LineNumberTable:
        line 7: 0
        line 8: 7
}
SourceFile: "Lambda.java"
InnerClasses:
     static #11= #10 of #8; //Logger=class Lambda$Logger of class Lambda
     public static final #56= #55 of #58; //Lookup=class java/lang/invoke/MethodHandles$Lookup of class java/lang/invoke/MethodHandles
BootstrapMethods:
  0: #27 invokestatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #28 (Ljava/lang/String;)V
      #29 invokestatic Lambda.lambda$main$0:(Ljava/lang/String;)V
      #28 (Ljava/lang/String;)V

Copy the code

In the detailed bytecode information above, there are two important things to note:

major version: 52

Major version 52 corresponds to JDK 1.8.

invokedynamicinstruction

Invokedynamic was introduced in Java7, but actually Javac does not support generating InvokeDynamic. In Java 8, Javac can generate Invokedynamic instructions, such as lambda expressions.

2.2 DX instruction packaging

First we compile the above class file using buildToolsVersion = 22.0.1.

$/ Users/dengshiwei/Library/Android/SDK/build - the tools / 22.0.1 / dx - dex *. ClassCopy the code

Make a mistake!

UNEXPECTED TOP-LEVEL EXCEPTION:
com.android.dx.cf.iface.ParseException: bad class file magic (cafebabe) or version (0034.0000)
	at com.android.dx.cf.direct.DirectClassFile.parse0(DirectClassFile.java:472)
	at com.android.dx.cf.direct.DirectClassFile.parse(DirectClassFile.java:406)
	at com.android.dx.cf.direct.DirectClassFile.parseToInterfacesIfNecessary(DirectClassFile.java:388)
	at com.android.dx.cf.direct.DirectClassFile.getMagic(DirectClassFile.java:251)
	at com.android.dx.command.dexer.Main.processClass(Main.java:704)
	at com.android.dx.command.dexer.Main.processFileBytes(Main.java:673)
	at com.android.dx.command.dexer.Main.accessThe $300(Main.java:83)
	at com.android.dx.command.dexer.MainThe $1.processFileBytes(Main.java:602)
	at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:170)
	at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
	at com.android.dx.command.dexer.Main.processOne(Main.java:632)
	at com.android.dx.command.dexer.Main.processAllFiles(Main.java:510)
	at com.android.dx.command.dexer.Main.runMonoDex(Main.java:280)
	at com.android.dx.command.dexer.Main.run(Main.java:246)
	at com.android.dx.command.dexer.Main.main(Main.java:215)
	at com.android.dx.command.Main.main(Main.java:106)
...while parsing Lambda$Logger.class

Copy the code

Bad class file magic cannot be identified because JDK and DX versions do not match. At the same time, version (0034.0000) is translated to decimal as 52, corresponding to JDK 1.8. Here we change the version to: 28.0.1.

DengshiweideMacBook - Pro: dex dengshiwei $/ Users/dengshiwei/Library/Android/SDK/build - the tools / 28.0.1 / dx - dex *. Class Uncaught translation error: com.android.dx.cf.code.SimException: ERRORin Lambda.main:([Ljava/lang/String;)V: invalid opcode ba - invokedynamic requires --min-sdk-version >= 26 (currently 13)
1 error; aborting
Copy the code

This is because lamda expressions use invokedynamic at the Java bytecode level, whereas Android supports invokeDynamic at the bytecode level until the device SDK version is older than 26. So how does Android implement support for lambda functions for all device API versions? Android does this by desugaring.

2.3 D8 instruction packaging

Also for the above file, compiled by the D8 instruction.

/ Users/dengshiwei/Library/Android/SDK/build - the tools / 28.0.1 / d8 *. ClassCopy the code

3. Summary

D8Compiler as defaultDEXBytecode file compiler, with better performance;
R8As aProGuardAlternative tools for code compression (shrinking) and confusion (obfuscation);
Desaccharification is used inAndroidIn the supportJava 8Partial properties.

1. Development history of D8 and R8

1.1 D8 compiler

1.2 the R8 tools

1.3 Android Studio version 3.4 D8 R8 changed

2. Take off the candy

2.1 Lambda Expression

2.2 DX instruction packaging

2.3 D8 instruction packaging

3. Summary

Related Posts

This is how it should be packaged with Kotlin.

The use of Flutter – OKToast

Learn about Android componentization