Author/Morten Krogh-Jespeersen, Mads Ager

R8 is Android’s default application reductor, which can reduce the size of Android apps by removing unused code and optimizing the rest of the code. R8 also supports reducing the size of Android libraries. In addition to generating smaller library files, library compression can also hide new features in the development library until they are relatively stable or available to the public.

Kotlin is a great development language for writing Android applications and libraries. However, using Kotlin reflection to shrink the Kotlin development library or application is not so simple. Kotlin uses metadata in Java class files to identify structures in the Kotlin language. If the application reductor does not maintain and update Kotlin’s metadata, the corresponding development library or application will not work properly.

R8 now supports maintaining and rewriting Kotlin’s metadata to fully support the use of Kotlin reflection to compress Kotlin development libraries and applications. This feature works with Android Gradle plugin version 4.1.0-beta03. Please feel free to try it out and give us feedback on your overall experience and any problems you encounter on the Issue Tracker page.

The rest of this article introduces information about Kotlin metadata and R8 support for overwriting Kotlin metadata.

Kotlin metadata

Kotlin metadata is some additional information stored in annotations in Java class files that are generated by the Kotlin JVM compiler. Metadata determines which Kotlin code makes up the classes and methods in the class file. For example, Kotlin metadata can tell the Kotlin compiler that a method in a class file is actually a Kotlin extension function.

As a simple example, the following library code defines a hypothetical base class for instruction building, used to build compiler instructions.

package com.example.mylibrary

/** CommandBuilderBase contains options common to D8 and R8 */

abstract class CommandBuilderBase {
    internal var minApi: Int = 0
    internal var inputs: MutableList<String> = mutableListOf()

    abstract fun getCommandName(a): String
    abstract fun getExtraArgs(a): String

    fun build(a): String {
        val inputArgs = inputs.joinToString(separator = "")
        return "${getCommandName()} --min-api=$minApi $inputArgs ${getExtraArgs()}"}}fun <T : CommandBuilderBase> T.setMinApi(api: Int): T {
    minApi = api
    return this
}

fun <T : CommandBuilderBase> T.addInput(input: String): T {
    inputs.add(input)
    return this
}
Copy the code

We can then define a concrete implementation of the hypothetical D8CommandBuilder, which inherits from CommandBuilderBase and is used to build simplified D8 instructions.

package com.example.mylibrary

/** D8CommandBuilder to build a D8 command. */
class D8CommandBuilder: CommandBuilderBase() {
    internal var intermediateOutput: Boolean = false
    override fun getCommandName(a) = "d8"
    override fun getExtraArgs(a) = "--intermediate=$intermediateOutput"
}

fun D8CommandBuilder.setIntermediateOutput(intermediate: Boolean) : D8CommandBuilder {
    intermediateOutput = intermediate
    return this
}
Copy the code

The example above uses the extension function to ensure that when you call the setMinApi method on D8CommandBuilder, the object type returned is D8CommandBuilder instead of CommandBuilderBase. In our example, these extension functions are top-level functions and exist only in the CommandBuilderKt class file. Let’s take a look at the output from the simplified Javap command.

$ javap com/example/mylibrary/CommandBuilderKt.class Compiled from "CommandBuilder.kt" public final class CommandBuilderKt { public static final <T extends CommandBuilderBase> T addInput(T, String); public static final <T extends CommandBuilderBase> T setMinApi(T, int); . }Copy the code

You can see from the javap output that the extension function is compiled as a static method whose first argument is the extension sink. However, this information is not enough to tell the Kotlin compiler that these methods need to be called as extension functions in Kotlin code. So the Kotlin compiler also adds a Kotlin.metadata annotation to the class file. The metadata in the annotations contains information specific to Kotlin in this class. You can see these annotations in the javAP output if you use the verbose option.

$ javap -v com/example/mylibrary/CommandBuilderKt.class ... RuntimeVisibleAnnotations: 0: kotlin/Metadata( mv=[...] , bv=[...] , k=... , xi=... , d1=["^@.\n^B^H^B\n^B^X^B\n^@\n^B^P^N\n^B...^D"], d2=["setMinApi", ...] )Copy the code

The D1 field of the metadata annotations contains most of the actual content in the form of protocol Buffer messages. The specific meaning of the metadata content is not important. The important thing is that the Kotlin compiler reads this and determines that these methods are extension functions, as shown in the Kotlinp dump output below.

$ kotlinp com/example/mylibrary/CommandBuilderKt.class
package {

// signature:   addInput(CommandBuilderBase,String)CommandBuilderBase
public final fun <T : CommandBuilderBase> T.addInput(input: kotlin/String): T

// signature: setMinApi(CommandBuilderBase,I)CommandBuilderBase
public final fun <T : CommandBuilderBase> T.setMinApi(api: kotlin/Int): T

...
}
Copy the code

This metadata indicates that these functions will be used as Kotlin extension functions in Kotlin user code:

D8CommandBuilder().setMinApi(12).setIntermediate(true).build()
Copy the code

How did R8 break the Kotlin development library in the past

As mentioned earlier, Kotlin’s metadata is important in order to be able to use the Kotlin API in the library. However, the metadata exists in annotations and in the form of Protocol Buffer messages, which are not recognized by R8. Therefore, R8 will choose one of two options:

  • Removing metadata
  • Keep the original metadata

But neither option is desirable.

If metadata is removed, the Kotlin compiler can no longer correctly identify extension functions. For example, in our example, when compiling code like D8CommandBuilder().setminapi (12), the compiler will report an error indicating that the method does not exist. This makes perfect sense, since without metadata, the only thing the Kotlin compiler sees is a Java static method with two arguments.

Keeping raw metadata is also problematic. First, the class retained in the Kotlin metadata is the type of the parent class. So, assuming that when we shrink the development library, we only want the D8CommandBuilder class to keep its name. This means that CommandBuilderBase will be renamed, usually to A. If we keep the original Kotlin metadata, the Kotlin compiler looks for the superclass of D8CommandBuilder in the metadata. If raw metadata is used, the superclass that is logged is CommandBuilderBase instead of A. An error is reported and the CommandBuilderBase type does not exist.

R8 overwrites the Kotlin metadata

To address these issues, the extended R8 adds the ability to maintain and rewrite Kotlin metadata. It is embedded with the Kotlin metadata development library developed by JetBrains in R8. The metadata development library can read the Kotlin metadata in raw input. Metadata information is stored in R8’s internal data structures. When R8 is done optimizing and miniaturizing the development library or application, it synthesizes new and correct metadata for all Kotlin classes whose claims are retained.

Let’s take a look at the changes to our example. We added the sample code to an Android Studio library project. In the gradle.build file, by setting minifyEnbled to true to enable package reduction, we update the reductor configuration to include the following:

# - keep D8CommandBuilder and all of its method and class com. Example. Mylibrary. D8CommandBuilder {< the methods >; } # keep extension function - keep class com.example.mylibrary.Com mandBuilderKt {< the methods >; } # retain kotlin. Metadata annotation on retaining project thereby maintaining Metadata - keepattributes RuntimeVisibleAnnotations - keep class kotlin. Metadata {*; }Copy the code

The above tells R8 to retain all extension functions in D8CommandBuilder and CommandBuilderKt. It also tells R8 to keep annotations, especially the Kotlin.metadata annotation. These rules apply only to classes that are explicitly declared to be reserved. Therefore, only metadata for D8CommandBuilder and CommandBuilderKt will be retained. But the metadata in CommandBuilderBase is not retained. We do this to reduce unnecessary metadata in our application and development libraries.

Now, with the reduced generated library enabled, the CommandBuilderBase inside is renamed to A. In addition, the Kotlin metadata for the retained classes is also overwritten so that all references to CommandBuilderBase are replaced with references to A. The development library is then ready for normal use.

As a final note, not keeping Kotlin metadata in CommandBuilderBase means that the Kotlin compiler treats the generated classes as Java classes. This can lead to strange results for the Java implementation details of the Kotlin class in the library. To avoid such problems, you need to preserve classes. If the class is retained, the metadata is retained. We can use the AllowObfuscation modifier in the retention rule to allow R8 to rename the class to generate Kotlin metadata so that both the Kotlin compiler and Android Studio treat the class as a Kotlin class.

-keep,allowobfuscation class com.example.mylibrary.CommandBuilderBase
Copy the code

Here, we have introduced the role of library reduction and Kotlin metadata for Kotlin development libraries. Applications that use Kotlin reflection through the Kotlin-Reflect library also require Kotlin metadata. The problems facing applications and development libraries are the same. If the Kotlin metadata is deleted or not updated correctly, the Kotlin-Reflect library cannot process the code as Kotlin code.

For a simple example, let’s say we want to find and call an extension function in a class at run time. We want to enable method renaming because we don’t care about the function name, as long as it can be found and called at run time.

class ReflectOnMe() {
    fun String.extension(a): String {
        return capitalize()
    }
}

fun reflect(receiver: ReflectOnMe): String {
    return ReflectOnMe::class
        .declaredMemberExtensionFunctions
        .first()
        .call(receiver, "reflection") as String
}
Copy the code

In the code, we add a call: Reflect (ReflectOnMe()). It finds the extension function defined in ReflectOnMe and calls it using the passed ReflectOnMe instance as the receiver and “Reflection” as the extension receiver.

Now that R8 can properly override Kotlin metadata in all reserved classes, we can enable rewriting by using the following reductor configuration.

Allowobfuscation class ReflectOnMe {<methods>; } # retain kotlin. Metadata annotation on retaining project thereby maintaining Metadata - keepattributes RuntimeVisibleAnnotations - keep class kotlin. Metadata {*; }Copy the code

This configuration allows the reductor to rename ReflectOnMe and extend functions while still maintaining and overwriting the Kotlin metadata.

Give it a try!

Welcome to R8 for the Kotlin metadata rewrite feature in the Kotlin library project and the use of Kotlin reflection in the Kotlin project. This feature is available in Android Gradle Plugin 4.1.0-beta03 and later. If you encounter any problems while using it, please submit questions on our Issue Tracker page.