R8 optimization: Lambda Groups

This is the 30th day of my participation in the Wenwen Challenge

This article is part 17 of jakewharton’s series on D8 and R8.

R8 Optimization: Lambda Groups
Originally written by jakewharton
Translator: Antway

Because of the features of the Kotlin standard library, the use of Lambda feels more prevalent in Kotlin than in Java. Some lambdas are simply syntactic constructs that are eliminated at compile time by using inline functions. The rest is specified as an entire class for use at run time.

The Android Java 8 Support article explains how Lambdas work, but here’s a quick update:

javac 将 LambdaThe principal is promoted to the package private method and is targeted at the calling siteLambdaType of writinginvoke-dynamicThe instructions.JVMRotate the class of the desired type at run time and invoke package-private methods in the method body.AndroidThis runtime support is not provided, thereforeD8Perform compile-time conversions on classes that implement the required types and call package-private methods.
kotlincJust skipinvoke-dynamicBytecode (even forjava8+), directly generate the complete class.

There are two Kotlin classes and some Lambda uses that we can experiment with.

class Employee(
  val id: String,
  val joined: LocalDate,
  val managerId: String?
)

class EmployeeRepository(val allEmployees: () -> Sequence<Employee>) {
  fun joinedAfter(date: LocalDate) =
      allEmployees()
          .filter { it.joined >= date }
          .toList()

  fun reports(manager: Employee) =
      allEmployees()
          .filter { it.managerId == manager.id }
          .toList()
}
Copy the code

The EmployeeRepository class takes a Lambda that generates a sequence of employees and has two public functions that list the employees that joined after a specific date and those that report to a specific Employee. Both functions use Lambda’s filter to filter to the desired items, which are then converted to lists.

Kotlin’s methods on lambdas are immediately visible after compiling the class.

$ kotlinc EmployeeRepository.kt
$ ls *.class
Employee.class
EmployeeRepository.class
EmployeeRepository$joinedAfter$1.class
EmployeeRepository$reports$1.class
Copy the code

Each lambda has a unique name, formed by concatenating the enclosing class name, enclosing function name, and monotone value.

1. Kotlin Lambdas and D8

Because we are not actually using these apis, we need to leave them explicitly, or R8 will generate an empty dex file.

-keep class Employee {*; } -keepclass EmployeeRepository {*; } -dontobfuscateCopy the code

With the two classes in place, let’s run R8 and see what happens.

$ java -jar $R8_HOME/build/libs/r8.jar \
      --lib $ANDROID_HOME/platforms/android-29/android.jar \
      --release \
      --output . \
      --pg-conf rules.txt \
      *.class kotlin-stdlib-*.jar
Copy the code

We can see what happens to the body of the joinedAfter and Reports functions.

[000dd4] EmployeeRepository.joinedAfter:(Ljava/time/LocalDate;) Ljava/util/List;0000: iget-object v0, v3, LEmployeeRepository; .allEmployees:Lkotlin/jvm/functions/Function0;0002: invoke-interface {v0}, Lkotlin/jvm/functions/Function0; .invoke:()Ljava/lang/Object;0005: move-result-object v0
-0006: new-instance v1, LEmployeeRepository$joinedAfter$1;
-0008: invoke-direct {v1, v3}, LEmployeeRepository$joinedAfter$1; .<init>:(Ljava/time/LocalDate;) V +0006: new-instance v1, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; + 0008:const/4 v2, #int 0+0009: invoke-direct {v1, v2, v4}, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; .<init>:(ILjava/lang/Object;) V000d: invoke-static{v0, v1}, Lkotlin/sequences/SequencesKt; .filter:(Lkotlin/sequences/Sequence; Lkotlin/jvm/functions/Function1;) Lkotlin/sequences/Sequence;0010: move-result-object v0
 0011: invoke-static{v0}, Lkotlin/sequences/SequencesKt; .toList:(Lkotlin/sequences/Sequence;) Ljava/util/List;0014: move-result-object v0
 0015: return-object v0

 [000e34] EmployeeRepository.reports:(LEmployee;) Ljava/util/List;0000: iget-object v0, v3, LEmployeeRepository; .allEmployees:Lkotlin/jvm/functions/Function0;0002: invoke-interface {v0}, Lkotlin/jvm/functions/Function0; .invoke:()Ljava/lang/Object;0005: move-result-object v0
-0006: new-instance v1, LEmployeeRepository$reports$1;
-0008: invoke-direct {v1, v3}, LEmployeeRepository$reports$1; .<init>:(LEmployee;) V +0006: new-instance v1, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; + 0008:const/4 v2, #int 1+0009: invoke-direct {v1, v2, v4}, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; .<init>:(ILjava/lang/Object;) V000d: invoke-static{v0, v1}, Lkotlin/sequences/SequencesKt; .filter:(Lkotlin/sequences/Sequence; Lkotlin/jvm/functions/Function1;) Lkotlin/sequences/Sequence;0010: move-result-object v0
 0011: invoke-static{v0}, Lkotlin/sequences/SequencesKt; .toList:(Lkotlin/sequences/Sequence;) Ljava/util/List;0014: move-result-object v0
 0015: return-object v0
Copy the code

Let’s break down the new bytecode:

in0006A name named-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloAIt is worth noting that both functions are now creating instances of the same class;
in0008Position tojoinedAfterSave the value0, as well as toreportsFunction save value1;
0009Calls the class constructor and passes integers and dates or managers (but as objects).

Both functions now instantiate the same class for their Lambda. Let’s look at that class.

Class #15            -
  Class descriptor  : 'L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; '
  Access flags      : 0x0011 (PUBLIC FINAL)
  Interfaces        -
    #0              : 'Lkotlin/jvm/functions/Function1; '
  Instance fields   -
    #0              : (in L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA;)
      name          : '$capture$0'
      type          : 'Ljava/lang/Object; '
      access        : 0x1011 (PUBLIC FINAL SYNTHETIC)
    #1              : (in L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA;)
      name          : '$id$'
      type          : 'I'
      access        : 0x1011 (PUBLIC FINAL SYNTHETIC)
  Direct methods    -
    #0              : (in L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA;)
      name          : '<init>'
      type          : '(ILjava/lang/Object;) V'
      access        : 0x10001(PUBLIC CONSTRUCTOR) code - [000db0] -$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA.<init>:(ILjava/lang/Object;) V0000: iput v1, v0, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; .$id$:I0002: iput-object v2, v0, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; .$capture$0:Ljava/lang/Object;
0004: return-void
Copy the code

As you can see from the bytecode, this class implements the Function1 interface and has two fields, an ID of type Object and an ID of type int, and two parameters of type Object and int in the constructor to assign values to fields.

Now let’s look at the implementation of the Invoke function.

[000d14] -$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA.invoke:(Ljava/lang/Object;) Ljava/lang/Object; 0000: iget v0, v4, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; .$id$:I 0002: iget-object v1, v4, L-$$LambdaGroup$ks$D2r6uJKXMyXfodlTO7Kw1WcCloA; .$capture$0:Ljava/lang/Object;
0004: if-eqz v0, 002c
0006: const/4 v2, #int 1
0007: if-ne v0, v2, 002a

000a: check-cast v1, LEmployee;
 ⋮
0029: return-object v5

002a: const/4 v5, #int 0002b: throw v5 002c: check-cast v0, Ljava/time/LocalDate; ⋮ 0044: return - object v5Copy the code

I cropped a lot, but let’s break it down:

0000behavioridField loading value;
0002Line aobjectField loading value;
0004checkidWhether is equal to the0If yes, jump to002cLine;
0006-0007.checkidIs it another value? If yes, jump to002a;
000a-0029 在 reports 的 lambdaImplementation, strong turnobject 为 EmployeeObject, remember, the execution condition here isid ! = 1Failure;
002a-002acauseNullPointerExceptionThe exception;
002c-0044 在 joinedAfter 的 LambdaImplementation, willObjectconvertLocalDate.

It is difficult to understand exactly what this transformation means just by looking at the Dalvik bytecode. We can do the equivalent conversion in the source code to make it more clear.

 class EmployeeRepository(val allEmployees: () -> Sequence<Employee>) {
   fun joinedAfter(date: LocalDate) =
       allEmployees()
-          .filter { it.joined >= date }
+          .fitler(MyLambdaGroup(date, 0))
           .toList()

   fun reports(manager: Employee) =
       allEmployees()
-          .filter { it.managerId == manager.id }
+          .filter(MyLambdaGroup(manager, 1))
           .toList()
 }
+
+private class MyLambdaGroup(+private valcapture0: Any? , +private val id: Int
+) : (Employee) -> Boolean{+override fun invoke(employee: Employee): Boolean
+    return when (id) {
+      0 -> employee.joinedAfter >= (capture0 as LocalDate)
+      1 -> employee.managerId == (capture0 as Employee).id
+      else -> throw NullPointerException()
+    }
+  }
+}
Copy the code

Two lambdas that would have yielded two classes have been replaced by a class with integers. By merging Lambda bodies, you can reduce the number of classes in APK.

This only works if two Lambdas have the same format. They don’t have to be exactly the same as what we saw in the example. One Lambda captures LocalDate, while the other captures Employee. Because both capture only one value, they have the same structure and can be merged into the lambda Group class.

2. Java Lambdas and R8

Let’s rewrite the example in Java to see what happens.

final class EmployeeRepository {
  private final Function0<Sequence<Employee>>allEmployees;

  EmployeeRepository(Function0<Sequence<Employee>> allEmployees) {
    this.allEmployees = allEmployees;
  }

  List<Employee> joinedAfter(LocalDate date) {
    return SequencesKt.toList(
      SequencesKt.filter(
          allEmployees.invoke(),
          e -> e.getJoined().compareTo(date) >= 0));
  }

  List<Employee> reports(Employee manager) {
    returnSequencesKt.toList( SequencesKt.filter( allEmployees.invoke(), e -> Objects.equals(e.getManagerId(), manager.getId()))); }}Copy the code

We use Kotlin’s Function0 instead of Supplier, Sequence instead of Stream, and Sequence extensions as static helpers to make the two examples as close to each other as possible. We can compile and reuse the same R8 calls using Javac.

$ rm EmployeeRepository*.class
$ javac -cp . EmployeeRepository.class
$ java -jar $R8_HOME/build/libs/r8.jar \
      --lib $ANDROID_HOME/platforms/android-29/android.jar \
      --release \
      --output . \
      --pg-conf rules.txt \
      *.class kotlin-stdlib-*.jar
Copy the code

The joinedAfter and Reports function bodies should look the same as when written in Kotlin, right?

[000d2c] EmployeeRepository.joinedAfter:(Ljava/time/LocalDate;) Ljava/util/List; ⋮ 0008:new-instance v1, L-$$Lambda$EmployeeRepository$RwNrgP_DBeZWqltgaXgoLCrPfqI; 000a: invoke-direct {v1, v4}, L-$$Lambda$EmployeeRepository$RwNrgP_DBeZWqltgaXgoLCrPfqI; .<init>:(Ljava/time/LocalDate;) V ⋮ d80 [000] EmployeeRepository. Reports: (LEmployee) Ljava/util/List; ⋮ 0008:new-instance v1, L-$$Lambda$EmployeeRepository$JjZ4a6TbrR3768PIUyNflFlLVF8; 000a: invoke-direct {v1, v4}, L-$$Lambda$EmployeeRepository$JjZ4a6TbrR3768PIUyNflFlLVF8; .<init>:(LEmployee;) V ⋮Copy the code

They didn’t! Instead of using Lambda groups, each implementation calls its own Lambda class.

As far as I know, there are no technical limitations as to why this applies only to kotlin Lambdas and not to Java Lambdas. The work is not finished yet. Question 153773246 tracks support for merging Java lambda into a lambda group.

By merging lambdas of the same structure together, R8 reduces the APK size impact and runtime class-loading burden at the expense of increasing the method body of the lambda.

While tuning does run across the entire application, by default, merging only happens in packages. This ensures that any package-private methods or types used in the lambda body are accessible. Add the -AllowCCessModification directive to the Shrinker rules so that R8 can globally merge lambdas by increasing the visibility of referenced methods and types when needed.

You may have noticed that there seems to be some kind of hash in the names of the classes generated for Java lambda and Lambda Groups. In the next article, we’ll delve into the unique naming of these classes.

1. Kotlin Lambdas and D8

2. Java Lambdas and R8

Related Posts

JAVA Object-oriented abstract classes

Seductive scale control, next door products are greedy cry

Kotlin Starter 3 – Frequently asked Questions for beginners