Learn bytecode staking if you want to do things at compile time, such as the usual no-trace embedding, method time statistics, and automatic injection in component communication. Bytecode staking is actually modifying the compiled class file, adding your own bytecode to it, and then printing out the package is the modified class file. Before you start developing, you need to know how to customize the Gradle plug-in and how to customize the Transform. Let’s take a look.
Gradle plugin
Gradle documentation currently defines plug-ins in three different ways:
Scripting plug-ins: Write the code for the plug-in directly in the build script, and the compiler automatically compiles the plug-in and adds it to the build script’s classpath.buildSrc project: When executing Gradle, the buildSrc directory in the root directory is compiled as the plugin source directory. After compilation, the results are added to the build script’s classpath and are available to the entire project.Standalone project: Plug-ins can be developed in a standalone project and then jar the project to be published locally or on a MAve server.
Example code is available for referenceGradleTestDemo
1.1 Is implemented directly in build.gradle file
// Application plug-in
apply plugin: CustomPluginA
// A custom plug-in example
class CustomPluginA implements Plugin<Project> {
@Override
void apply(Project target) {
println 'Hello gradle! '}}Copy the codeThis way is not visible outside of the build script, so you can only reference the plugin in the Gradle script that defines it.
1.2 implemented in the default directory buildSrc
The buildSrc directory is one of gradle’s default directories. It is automatically compiled and packaged at build time, so it can be referenced directly by gradle scripts in other modules without any additional configuration.
- Directory structure created
buildscript {
ext {
kotlin_version = '1.5.31'
apg_Version = '3.4.0'
booster_version = '4.0.0'
}
repositories {
mavenCentral()
google()
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "com.android.tools.build:gradle:$apg_Version"
}
}
apply plugin: 'maven'
apply plugin: 'maven-publish'
apply plugin: 'java'
apply plugin: 'groovy'
apply plugin: 'kotlin'
apply plugin: 'kotlin-kapt'
apply plugin: 'kotlin-android-extensions'
repositories {
mavenCentral()
google()
jcenter()
}
dependencies {
implementation gradleApi()
implementation localGroovy()
// The utility class for the operation
implementation "Commons - the IO: the Commons - IO: 2.6"
// Android DSL Android compiled most of the gradle source code
implementation 'com. Android. Tools. Build: gradle: 3.4.0'
//ASM
implementation 'org. Ow2. Asm: asm: 7.1'
implementation 'org. Ow2. Asm: asm - util: 7.1'
implementation 'org. Ow2. Asm: asm - Commons: 7.1'
}
Copy the codeGradle plugins can be written in Java, Groovy, and Kotlin, so you can import dependencies based on your needs.
-
Create a new resources/ mate-INF /gradle-plugins directory in the main directory:
Create a new “HelloPlugin.properties” file, where “HelloPlugin” is an arbitrary name, which is the name of your plugin. You can then reference the plugin by applying Plugin: ‘HelloPlugin’.
The contents of the helloplugin.properties file are:
implementation-class=transform.hello.HelloTransformPlugin Copy the codeIn addition, plugins defined under buildSrc can be imported directly with apply Plugin: HelloPlugin, which is the class name of the plugin you define.
Note: Plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins: resources/ mate-INF /gradle-plugins The directory address becomes resources/ mate-INF.gradle-plugins.
-
Custom Gradle plugin:
Create the HelloTransformPlugin class in the Transform directory and implement the Plugin interface.
class HelloTransformPlugin implements Plugin<Project> { @Override void apply(Project project) { println "Hello TransformPlugin" // Register Extension with Plugin def extension = project.extensions.create("custom", CustomExtension) AppExtension is an application Plugin AppExtension appExtension = project.extensions.getByType(AppExtension) appExtension.registerTransform(new HelloTransform()) // A task is generated in TransformManager#addTransform after registration. // Register mode 2 //project.android.registerTransform(new HelloTransform())}}Copy the codeYou’ll see that there’s a Transform class here which is what we’re going to talk about next. CustomExtension is a custom attribute class that can be passed to the build.gradle file of the main project, so that the attributes can be extended in the script:
class CustomExtension { String extensionArgs = "" } Copy the codeThen name the main project build.gradle the same as when it was registered:
custom{ extensionArgs = "I'm a parameter." } Copy the codeIn the project. Extensions. The create method within the essence of which is through the project. The extensions. The create () method to get the custom closures defined content and through reflection of the content of the closure is transformed into a CustomExtension object.
1.3 Implemented in independent project development
This approach is similar to the second, except that to introduce the plug-in, you need to publish it locally or on the Mave server.
-
Modify the build.gradle content to add the code uploaded to the local directory as follows:
apply plugin: 'groovy' apply plugin: 'java' apply plugin: 'maven' repositories { jcenter() } uploadArchives { repositories.mavenDeployer { // Specify maven's repository URL, IP+ port + directory // repository(url: "http://localhost:8081/nexus/content/repositories/releases/") { // // Fill in your Nexus account password // authentication(userName: "admin", password: "123456") / /} // Configure the local repository path, which is in the maven directory under the project root directory repository(url: uri('.. /repo')) // The unique identifier is usually the module package name or other pom.groupId = "com.xiam.plugin" // Project name (usually module name or other pom.artifactId = "startplugin" // Release number pom.version = "1.0.0" } } dependencies { implementation gradleApi() implementation localGroovy() // Android DSL Android compiled most of the gradle source code implementation 'com. Android. Tools. Build: gradle: 3.4.0' } Copy the code-
Modify the relevant build.gradle file, add dependencies, and add to the root project build.gradle:
-
Run the uploadArchives task./gradlew uploadArchivers command to execute this task:
-
2. Transform
Google officially provides the Transform API for Android GradleV1.5.0, which allows third-party plugins to process a compiled class file before converting it to a dex file. All we need to do is implement the Transform to the.class file to get all the methods, and then replace the source file when we’re done. Take a look at the Transform version history.
2.1 Use of Transform
We have already seen how to register a transform in our custom plugin by using the following, which I have chosen to do with Kotlin:
class HelloPlugin: Plugin<Project> {
override fun apply(target: Project) {
target.extensions.findByType(AppExtension::class.java)? .run { registerTransform(HelloTransform(target)) } } }Copy the codeThe Transform of the custom is to inherit from com. Android. Build. API. The Transform. Transform, may have a look the Transform documents, now let’s define a custom Transform (does not support incremental) :
class HelloTransform: Transform() {
/** * Returns the Task name. * /
override fun getName(a): String = "HelloTransform"
There are two types of input files we can work with: compiled Java code and resource files (not files under RES but resources within Assests) */
override fun getInputTypes(a): MutableSet<QualifiedContent.ContentType> = TransformManager.CONTENT_CLASS
/** * Whether delta is supported * If delta execution is supported, the change input may contain a list of modified/deleted/added files */
override fun isIncremental(a): Boolean = false
/** * Specifies the scope of the plug-in. * /
override fun getScopes(a): MutableSet<in QualifiedContent.Scope> = TransformManager.SCOPE_FULL_PROJECT
/** * The main function of the transform performs the specific conversion process */
override fun transform(transformInvocation: TransformInvocation?).{ transformInvocation? .inputs? .forEach {// The input source is the folder type (the directory where the class files compiled by the local project are stored)
it.directoryInputs.forEach {directoryInput->
with(directoryInput){
// Get the output path of the class file
val dest = transformInvocation.outputProvider.getContentLocation(
name,
contentTypes,
scopes,
Format.DIRECTORY
)
// Copy the modified bytecode to dest to achieve the purpose of intervening bytecode during compilation
file.copyTo(dest)
}
}
// Input source is jar package type (each dependency is compiled into jar file)
it.jarInputs.forEach { jarInput->
with(jarInput){
// Get the output path of the JAR package
val dest = transformInvocation.outputProvider.getContentLocation(
name,
contentTypes,
scopes,
Format.JAR
)
// Copy the modified bytecode to dest to achieve the purpose of intervening bytecode during compilation
file.copyTo(dest)
}
}
}
}
}
Copy the code2.1.1 getName ()
This method returns our Transform name, which will appear in the Build stream:
How did the name come to be? Gradle source code has a class TransformManager that manages all subclasses of Transform. You can find a getTaskNamePrefix method. It will start with tansform, and then concatenate contentType, which is the input type of the Transform, Classes and Resources, and then the Name of the Transform.
#TransformManager
static String getTaskNamePrefix(@NonNull Transform transform) {
StringBuilder sb = new StringBuilder(100);
sb.append("transform");
sb.append(
transform
.getInputTypes()
.stream()
.map(
inputType ->
CaseFormat.UPPER_UNDERSCORE.to(
CaseFormat.UPPER_CAMEL, inputType.name()))
.sorted() // Keep the order stable.
.collect(Collectors.joining("And")));
sb.append("With");
StringHelper.appendCapitalized(sb, transform.getName());
sb.append("For");
return sb.toString();
}
Copy the code2.1.2 getInputTypes ()
Transformmanager.content_class: transformManager.transformManager.transformmanager.content_class: transformManager.transformManager.class: transformManager.content_class Transformmanager.content_resources is returned.
In addition to CLASSES and RESOURCES, there are other types that we can’t use during development, such as the DEX file. These hidden types are in a separate enumerated class, ExtendedContentType, which is only available to the Android compiler.
2.1.3 getScopes ()
This specifies which file needs to be processed, which is used to indicate scope. Take a look at the options:
/** * indicates the Scope of the Transform. Currently, there are five basic types of Scope: * 1, PROJECT has only PROJECT contents * 2, SUB_PROJECTS has only subprojects * 3, EXTERNAL_LIBRARIES has only external libraries * 4, TESTED_CODE is the code tested by the current variant (including dependencies) * PROVIDED_ONLY provides only local or remote dependencies * SCOPE_FULL_PROJECT is a Scope collection containing scope.project, */
enum Scope implements ScopeType {
/** Only the project (module) content */
PROJECT(0x01),
/** Only the sub-projects (other modules) */
SUB_PROJECTS(0x04),
/** Only the external libraries */
EXTERNAL_LIBRARIES(0x10),
/** Code that is being tested by the current variant, including dependencies */
TESTED_CODE(0x20),
/** Local or remote dependencies that are provided-only */
PROVIDED_ONLY(0x40),
/**
* Only the project's local dependencies (local jars)
*
* @deprecated local dependencies are now processed as {@link #EXTERNAL_LIBRARIES}
*/
@Deprecated
PROJECT_LOCAL_DEPS(0x02),
/**
* Only the sub-projects's local dependencies (local jars).
*
* @deprecated local dependencies are now processed as {@link #EXTERNAL_LIBRARIES}
*/
@Deprecated
SUB_PROJECTS_LOCAL_DEPS(0x08); . }Copy the codeSCOPE_FULL_PROJECT: transformmanager. SCOPE_FULL_PROJECT: transformmanager. SCOPE_FULL_PROJECT: transformmanager. SCOPE_FULL_PROJECT
public static final Set<Scope> SCOPE_FULL_PROJECT =
Sets.immutableEnumSet(
Scope.PROJECT,
Scope.SUB_PROJECTS,
Scope.EXTERNAL_LIBRARIES);
Copy the code2.1.4 inIncremental ()
Indicates whether incremental compilation is supported. When closed, full compilation takes place and the last output is deleted. When we open the incremental compilation, the input contains the changed/removed/added/notchanged four kinds of state:
NOTCHANGED: The current file is unchanged and does not need to be processed, even copiedADDED, CHANGED,: There are modification files and output to the next taskREMOVED: The file corresponding to the outputProvider path is removed
2.1.5 the transform ()
In this method, we assign each JAR and class file to the dest path, which is the input of the next Transform. During copying, we can modify the bytecode of the jar and class file (ASM is passing through here).
After processing the class/jar packages can be in/build/intermediates/transforms/HelloTransform/view, you will see all the jars are increasing to 123456. Here’s how to get the output path:
# IntermediateFolderUtils
public synchronized File getContentLocation(String name, Set<ContentType> types, Set<? super Scope> scopes, Format format) { Preconditions.checkNotNull(name); Preconditions.checkNotNull(types); Preconditions.checkNotNull(scopes); Preconditions.checkNotNull(format); Preconditions.checkState(! name.isEmpty()); Preconditions.checkState(! types.isEmpty()); Preconditions.checkState(! scopes.isEmpty()); Iterator var5 =this.subStreams.iterator();
SubStream subStream;
do {
if(! var5.hasNext()) {// You can see here that it is incremented by position
SubStream newSubStream = new SubStream(name, this.nextIndex++, scopes, types, format, true);
this.subStreams.add(newSubStream);
return new File(this.rootFolder, newSubStream.getFilename());
}
subStream = (SubStream)var5.next();
} while(! name.equals(subStream.getName()) || ! types.equals(subStream.getTypes()) || ! scopes.equals(subStream.getScopes()) || format ! = subStream.getFormat());return new File(this.rootFolder, subStream.getFilename());
}
Copy the code2.2 Principle of Transform
After explaining how to use Transfoem, let’s look at how it works (gradle plugin version 7.0.2).
First let’s take a look at the process from Java source to APK, as shown below:
It is clear that gradle’s packaging process is basically done through the official Transform. Each Transform is actually a Gradle Task. The TaskManager in the Android compiler concatenates each Transform. The first Transform receives the result from the Javac compilation. As well as local third-party dependencies and resource resources in the Asset directory. These compiled intermediates then flow through a chain of transforms, with each Tansform processing the class before passing it on to the next Transform.
Our custom Transform will be inserted at the front of the Tansform chain before ProguardTransform, so there will be no confusion and no class information will be scanned.
2.2.1 TransformManager
When registerTransform is called in the previous custom plugin to register the transform, it is actually put into the BaseExtension class’s list array. The TaskManager then calls the addTransform method of The TransformManager. Here the TransformManager manages all the Transform objects for the variant of the project.
Let’s look at the addTransform method implementation:
# TransformManager
public <T extends Transform> Optional<TaskProvider<TransformTask>> addTransform(
@NonNull TaskFactory taskFactory,
@NonNull TransformVariantScope scope,
@NonNull T transform,
@Nullable PreConfigAction preConfigAction,
@Nullable TaskConfigAction<TransformTask> configAction,
@Nullable TaskProviderCallback<TransformTask> providerCallback) {
......
List<TransformStream> inputStreams = Lists.newArrayList();
// Name the transform task
String taskName = scope.getTaskName(getTaskNamePrefix(transform));
// Get a reference-only stream
List<TransformStream> referencedStreams = grabReferencedStreams(transform);
// Find the input stream and calculate the output stream through the transformIntermediateStream outputStream = findTransformStreams( transform, scope, inputStreams, taskName, scope.getGlobalScope().getBuildDir()); . transforms.add(transform);// Create the transform task
return Optional.of(
taskFactory.register(
new TransformTask.CreationAction<>(
scope.getFullVariantName(),
taskName,
transform,
inputStreams,
referencedStreams,
outputStream,
recorder),
preConfigAction,
configAction,
providerCallback));
}
Copy the code-
The name of the task is defined in the getTaskNamePrefix method, which was examined earlier.
-
Then, in grabReferencedStreams method, the data input of transform is filtered through the Scope and ContentType of the internally defined referential input. As you can see, the grabReferencedStreams method takes the intersection of the Streams scope and type to get the corresponding stream and defines it as the referenced stream we need.
private List<TransformStream> grabReferencedStreams(@NonNull Transform transform) { Set<? superScope> requestedScopes = transform.getReferencedScopes(); . List<TransformStream> streamMatches = Lists.newArrayListWithExpectedSize(streams.size()); Set<ContentType> requestedTypes = transform.getInputTypes();for (TransformStream stream : streams) { Set<ContentType> availableTypes = stream.getContentTypes(); Set<? super Scope> availableScopes = stream.getScopes(); Set<ContentType> commonTypes = Sets.intersection(requestedTypes, availableTypes); Set<? super Scope> commonScopes = Sets.intersection(requestedScopes, availableScopes); if(! commonTypes.isEmpty() && ! commonScopes.isEmpty()) { streamMatches.add(stream); }}return streamMatches; } Copy the code -
Then in the findTransformStreams method, the corresponding consumptive input streams are obtained according to the SCOPE and INPUT_TYPE defined, and the consumptive input streams are removed. Create a single composite output stream for all types and ranges and add it to the list of available streams for the next transformation.
private IntermediateStream findTransformStreams( @NonNull Transform transform, @NonNull TransformVariantScope scope, @NonNull List<TransformStream> inputStreams, @NonNull String taskName, @NonNull File buildDir) {... Set<ContentType> requestedTypes = transform.getInputTypes();// Remove the corresponding consumption input streams in Streams consumeStreams(requestedScopes, requestedTypes, inputStreams); // Create an output stream Set<ContentType> outputTypes = transform.getOutputTypes(); File outRootFolder = FileUtils.join( buildDir, StringHelper.toStrings( AndroidProject.FD_INTERMEDIATES, FD_TRANSFORMS, transform.getName(), scope.getDirectorySegments())); // Create an output stream IntermediateStream outputStream = IntermediateStream.builder( project, transform.getName() + "-" + scope.getFullVariantName(), taskName) .addContentTypes(outputTypes) .addScopes(requestedScopes) .setRootLocation(outRootFolder) .build(); streams.add(outputStream); return outputStream; } Copy the code -
Finally, the newly created TransformTask is registered with the TaskManager.
2.2.2 TransformTask
In this class we see the tansForm method of the final Transform called, executed in its TaskAction:
# TransformTask
@TaskAction
void transform(final IncrementalTaskInputs incrementalTaskInputs)
throws IOException, TransformException, InterruptedException {
final ReferenceHolder<List<TransformInput>> consumedInputs = ReferenceHolder.empty();
final ReferenceHolder<List<TransformInput>> referencedInputs = ReferenceHolder.empty();
final ReferenceHolder<Boolean> isIncremental = ReferenceHolder.empty();
final ReferenceHolder<Collection<SecondaryInput>> changedSecondaryInputs =
ReferenceHolder.empty();
isIncremental.setValue(transform.isIncremental() && incrementalTaskInputs.isIncremental());
// In incremental mode, determine changes in input streams (referential and consumable). recorder.record( ExecutionType.TASK_TRANSFORM, executionInfo, getProject().getPath(), getVariantName(),new Recorder.Block<Void>() {
@Override
public Void call(a) throws Exception {
transform.transform(
new TransformInvocationBuilder(TransformTask.this) .addInputs(consumedInputs.getValue()) .addReferencedInputs(referencedInputs.getValue()) .addSecondaryInputs(changedSecondaryInputs.getValue()) .addOutputProvider( outputStream ! =null
? outputStream.asOutput(
isIncremental.getValue())
: null)
.setIncrementalMode(isIncremental.getValue())
.build());
if(outputStream ! =null) {
outputStream.save();
}
return null; }}); }Copy the codeSo far we have seen the data flow principle of Transform, the type of input, and the filtering mechanism.
2.3 Increment and concurrency of Transform
After learning the above, we can easily define a Transform. However, every time the transform method is compiled, it iterates through all of the class files, decompresses all of the JAR files, and then recompresses them into all of the JAR files, which slows down the compile time. So to solve that, we’re using incremental compilation.
However, not all compilations can be incremental compilations. After all, the directory. ChangedFiles is empty after the first compilation or clean recompilation. If it is not incremental compilation, then the output directory is emptied and processed class by jar as before. If it is incremental compilation to check the Status of each file, the Status is divided into four NOTCHANGED/ADDED/CHANGED/REMOVED, and is not the same to the operation of the four types of files.
You can see the code implementation of incremental compilation. For details, see BaseTransform
// Perform the specific conversion process.
override fun transform(transformInvocation: TransformInvocation) {
Log.log("transform start--------------->")
onTransformStart()
val outputProvider = transformInvocation.outputProvider
val context = transformInvocation.context
val isIncremental = transformInvocation.isIncremental
val startTime = System.currentTimeMillis()
// Not incremental update, delete the output before
if(! isIncremental){ outputProvider.deleteAll() } transformInvocation?.inputs?.forEach{input ->// Enter the folder type (the directory where the class files compiled by the local project are stored)
input.directoryInputs.forEach{directoryInput ->
submitTask {
handleDirectory(directoryInput, outputProvider, context, isIncremental)
}
}
// Enter the jar package type (the compiled JAR files for each dependency)
input.jarInputs.forEach{jarInput ->
submitTask {
handleJar(jarInput, outputProvider, context, isIncremental)
}
}
}
val taskListFeature = executorService.invokeAll(taskList)
taskListFeature.forEach{
it.get()
}
onTransformEnd()
Log.log("transform end--------------->" + "duration : " + (System.currentTimeMillis() - startTime) + " ms")}Copy the codeFor jar package type input:
private fun handleJar(jarInput: JarInput, outputProvider: TransformOutputProvider, context: Context, isIncremental: Boolean) {
// Get the last Transform input file
val inputJar = jarInput.file
// Get the current Transform output JAR file
val outputJar = outputProvider.getContentLocation(
jarInput.name, jarInput.contentTypes,
jarInput.scopes, Format.JAR
)
// Incremental processing
if (isIncremental){
when(jarInput.status){
// The file has not changed
Status.NOTCHANGED -> {
}
// There are modification files
Status.ADDED,Status.CHANGED -> {
}
// The file is removed
Status.REMOVED -> {
if (outputJar.exists()){
FileUtils.forceDelete(outputJar)
}
return
}
else- > {return}}}if (outputJar.exists()){
FileUtils.forceDelete(outputJar)
}
// The modified file
val modifiedJar = if (ClassUtils.isLegalJar(jarInput.file)) {
modifyJar(jarInput.file, context.temporaryDir)
} else {
Log.log("Not processed:" + jarInput.file.absoluteFile)
jarInput.file
}
FileUtils.copyFile(modifiedJar, outputJar)
}
Copy the codeHandle the input as folder type:
private fun handleDirectory(directoryInput: DirectoryInput, outputProvider: TransformOutputProvider, context: Context, isIncremental: Boolean) {
// Get the last Transform input file directory
val inputDir = directoryInput.file
// Get the current Transform
val outputDir = outputProvider.getContentLocation(
directoryInput.name, directoryInput.contentTypes,
directoryInput.scopes, Format.DIRECTORY
)
val srcDirPath = inputDir.absolutePath
val destDirPath = outputDir.absolutePath
// The directory to write temporary files to
val temporaryDir = context.temporaryDir
// Create directory
FileUtils.forceMkdir(outputDir)
if (isIncremental){
directoryInput.changedFiles.entries.forEach { entry ->
val inputFile = entry.key
// The path where the final file should be stored
// val destFilePath = inputFile.absolutePath.replace(srcDirPath, destDirPath)
// val destFile = File(destFilePath)
when(entry.value){
Status.ADDED, Status.CHANGED ->{
// Process the class file
modifyClassFile(inputFile, srcDirPath, destDirPath, temporaryDir)
}
Status.REMOVED -> {
val destFilePath = inputFile.absolutePath.replace(srcDirPath, destDirPath)
val destFile = File(destFilePath)
if (destFile.exists()){
destFile.delete()
}
}
Status.NOTCHANGED -> {
}
}
}
} else {
// Filter out files, not directories
directoryInput.file.walkTopDown().filter { it.isFile }
.forEach { classFile ->
modifyClassFile(classFile, srcDirPath, destDirPath, temporaryDir)
}
}
}
Copy the codeThis will provide incremental features for our compiled plug-in.
Three, endnotes
In this article, you learned how to customize a Gradle plug-in, how to define a Transform, and the inner workings of a Transform. That’s not enough, but combined with ASM, which we’ll cover later, you can do whatever you want with bytecode staking.
Reference:
Build Gradle automatically
Write a more cow force Transform | plugins advanced tutorial
Android Transform incremental compilation
Gradle+Transform+Asm automatic injection code
Transform API
How to develop a High performance Gradle Transform
Play with bytecode in Android projects
Deeper understanding of Transform
The Transform,