preface
The main code for this article is available on Github github.com/Leifzhang/A…
It’s my 90-year old Android, and this is a self-congratuatory piece of bullshit. First of all, I wrote this AndroidAutoTrack Demo for a very simple reason, I just think it is fun, and at the same time for my own technical level will actually grow. I’ve been off work optimizing an automated burial site I wrote earlier. I’ve read a lot of articles about this, but I think they’re all about getting started, and it’s hard to go a little further.
Plugin or Transform, I personally think it is not difficult to say, but for new beginners, this thing is very unfriendly, the official gralde information is in English, and the gralde Plugin is cumbersome to write and debug, if you run into some problems, if no one to take you, You’re at a dead end, too.
So it’s not like you’re going to be able to completely understand this by reading the article, and I personally think it’s very difficult to learn this without someone to guide you. There is a simple and convenient way to write a Gradle Plugin through ComposeBuilding in my project, which is also introduced in my last article. Coroutines routing componentization 1 + 1 + 1 > 3 | Denver annual essay
Start my act
A qualified Transform plug-in requires incremental compilation, I take the data from the previous addendum to give you a better.
In the case of full compilation
In the case of quadratic incremental compilation
Apart from other tasks, the full compilation of the same Transform takes 2784ms, while the incremental compilation of code changes takes only 68ms. The gap is large enough to justify writing incremental compilation.
As the code continues to grow later in the project, there will inevitably be more Transform, and any one of the transformations that is not incremental will cause the entire Transform compilation process to become full. In fact, there are many system Transform tasks, such as Shrink and Dex merging, etc.
If you think about it, if one person optimizes a minute and a half of compile time, then if the team has a large number of people, then it is not a GOOD Kpi.
How to implement an addendum
Interested leaders can check out this address github.com/Leifzhang/A…
First, I abstracted the Transform process, mainly because I was lazy, and I would not like to copy and paste the same code and function twice. So I first combed through this part of the code and sorted out two parts, the first is the copy of the file, the second is the ASM operation of the file. I think the first part of the code can be integrated, and then there is the logic.
Let’s start with Transform. Simply put, Transform is a process that sets the input file Collection
In the case of incremental compilation, where the input stream changes slightly, TransformInput tells us what Class it changed to, where the changes are defined as three, the same for both Jar and Class.
-
NOTCHANGED The current file does not need to be skipped.
-
Since we use Temp first and then overwrite the current file, we will do the same.
-
REMOVED Removes the historical file from the current folder.
So when the addendum is called, we’re just doing different things with these four different operation symbols, just a couple of if else’s.
While we are using asm, we all can only operate the Class files, and then, according to the file name of the Class + path for a simple judgment, the current Class if we need to do a pile or scanning operation, then we will read the file byte array, after the completion of asm operation returns a byte array, Then overwrite the original file. So here I abstracted it for the first time, and I defined the ASM operation as an interface.
package com.kronos.plugin.base
interface TransformCallBack {
fun process(className: String, classBytes: ByteArray?).: ByteArray?
}
Copy the codeThe interface accepts a filename and a byte array, and returns a byte array at the end of the method. If the byte array is not empty, it means that the current class has been bytecode modified, and then we simply overwrite the file once. With this abstraction, we can integrate the above file operations with ASM operations, and SDK users are only responsible for this interface.
All that’s left to do is encapsulate this part of the file. How did I do it? I refer to the idea of multi-threaded transform optimization of another big guy, whose project address Is Leaking/Hunter
- All input files are copied first
- Forecah traversal pushes each file operation into the thread pool for execution
- Getting the file name and byte array calls the abstract interface we defined
- The temp file is generated based on the byte returned by interface, and the file is overwritten
- The thread pool waits for all tasks to complete before ending the Transform
DoubleTap compilation speed optimization
The original DoubleTap Plugin was used to scan the entire project’s code. Although incremental compilation was completed and I filtered out a lot of invalid scanning logic, it actually slowed down the compilation. It wasn’t until I studied a StringFog project from another big guy a while ago that I discovered that Big guy’s constant-encrypted Transform works directly on modules.
Code address github.com/Leifzhang/A…
public class DoubleTapPlugin implements Plugin<Project> {
private static final String EXT_NAME = "doubleTab";
@Override
public void apply(Project project) {
boolean isApp = project.getPlugins().hasPlugin(AppPlugin.class);
project.getExtensions().create(EXT_NAME, DoubleTabConfig.class);
project.afterEvaluate(project1 -> {
DoubleTabConfig config = (DoubleTabConfig) project1.getExtensions().findByName(EXT_NAME);
if (config == null) {
config = new DoubleTabConfig();
}
config.transform();
});
if (isApp) {
AppExtension appExtension = project.getExtensions().getByType(AppExtension.class);
appExtension.registerTransform(new DoubleTapAppTransform());
return;
}
if (project.getPlugins().hasPlugin("com.android.library")) {
LibraryExtension libraryExtension = project.getExtensions().getByType(LibraryExtension.class);
libraryExtension.registerTransform(newDoubleTapLibraryTransform()); }}}Copy the codeIn the past, I only registered a Transform for AppExtension, but I can also register a Transform for LibraryExtension. Registering with LibraryExtension causes the bytecode operations to be used on modules that use the Plugin.
Tip: This Transform also works with aArs, not just local artifacts.
The biggest difference in the Transform code is that it is the same except for the input product and type.
class DoubleTapLibraryTransform : DoubleTapTransform(a){
override fun getScopes(a): MutableSet<in QualifiedContent.Scope> {
return ImmutableSet.of(
QualifiedContent.Scope.PROJECT
)
}
override fun getInputTypes(a): Set<QualifiedContent.ContentType>? {
return TransformManager.CONTENT_CLASS
}
}
Copy the codeThere’s a Scope, there’s InputTypes and you can look at other people’s articles to get a deeper understanding of Transform
If it’s a Transform that needs to be modified within a Module, you don’t need to write a global Transform at all. You just need to operate on each Module. This has several benefits. The scan is faster because we don’t need to operate on extraneous jars. In addition, if the Module does not participate in compilation without changing, it can be faster.
Automatic transmission of buried point parameters
When I was writing automated burial demos, I never really solved the problem of parameters. There was a hole in the past where I could only use properties defined in anonymous inner classes, whereas I actually felt uncomfortable writing in the outer classes because of ClassVisitor writing in ASM, which is event-based. When a method is triggered you record the value and then insert it into another function.
When I was doing ThreadPoolHook, I learned that all asm in The Booster of Didi used ClassNode. Here I will briefly talk about ClassNode.
ClassNode profile
If you read the bytecode article carefully, you should know that in Java, when a method is called, a Stack Frame is generated. This means that the method is contained in the Stack Frame. The Stack Frame consists of three parts: the local variables area, the operand Stack area, and the Frame data area. Next we mainly use the local variable area and operand stack area.
In general, a simple sentence of Java code will multiply in complexity when translated into bytecode, especially the stack frames of Java bytecode. Passing a parameter to a method is the operation of pressing the stack, so when I do it directly with ClassVisitor, and I want to change a line of code, it’s actually very difficult.
ClassNode is an implementation class of ClassVisitor. Compared to ClassVisitor,ClassNode has stored and recorded all the information of ClassVisitor, constructed the syntax tree, including the code and line numbers in the method, as well as the current class attributes, class information, and so on. Its core is to sacrifice memory, but because of the record of all class information, so for complex multi-class linkage operation, it will be more convenient and practical.
However, the TreeAPI is about 30% slower than the CoreAPI and has a high memory footprint.
To change the Class, we simply use the ClassTransformer and modify the corresponding ClassNode in the transform method. Using the TreeAPI is more time consuming and memory intensive than the CoreAPI, but it is relatively easy to make complex changes. The treeAPI is designed to be used in scenarios that need to be parsed multiple times, which cannot be done once using the coreAPI.
If you are interested in learning more about this section, you can see the article ah, Java Bytecode (ASM) and a simple explanation.
ClassNode passes in parameters
Ok, show me the code
class AutoTrackHelper : AsmHelper {
private val classNodeMap = hashMapOf<String, ClassNode>()
@Throws(IOException::class)
override fun modifyClass(srcClass: ByteArray): ByteArray {
val classNode = ClassNode(ASM5)
val classReader = ClassReader(srcClass)
//1 Converts the read bytes into classNode
classReader.accept(classNode, 0)
classNodeMap[classNode.name] = classNode
// Determine whether the current class implements the OnClickListener interface
classNode.interfaces?.forEach {
if (it == "android/view/View\$OnClickListener") { val field = classNode.getField() classNode.methods? .forEach { method ->// Find the onClick method
insertTrack(classNode, method, field)
}
}
}
// Call the Fragment's onHiddenChange method
visitFragment(classNode)
val classWriter = ClassWriter(0)
//3 Convert classNode into a byte array
classNode.accept(classWriter)
return classWriter.toByteArray()
}
private fun insertTrack(node: ClassNode, method: MethodNode, field: FieldNode?) {
// Determine the name and description of the method
if (method.name == "onClick" && method.desc == "(Landroid/view/View;) V") {
val className = node.outerClass
val parentNode = classNodeMap[className]
// Obtain the Node of the external class based on outClassNameval parentField = field ? : parentNode? .getField() val instructions = method.instructions instructions? .iterator()? .forEach {// Determine if the code is a cutoff point
if ((it.opcode >= Opcodes.IRETURN && it.opcode <= Opcodes.RETURN) || it.opcode == Opcodes.ATHROW) {
instructions.insertBefore(it, VarInsnNode(Opcodes.ALOAD, 1))
instructions.insertBefore(it, VarInsnNode(Opcodes.ALOAD, 1))
// Get the data parameter
if(parentField ! =null) {
parentField.apply {
instructions.insertBefore(it, VarInsnNode(Opcodes.ALOAD, 0))
instructions.insertBefore(
it, FieldInsnNode(Opcodes.GETFIELD, node.name, parentField.name, parentField.desc)
)
}
} else {
instructions.insertBefore(it, LdcInsnNode("1234"))
}
instructions.insertBefore(
it, MethodInsnNode(
Opcodes.INVOKESTATIC,
"com/wallstreetcn/sample/ToastHelper"."toast"."(Ljava/lang/Object; Landroid/view/View; Ljava/lang/Object;) V".false)}}}}// Determine whether the Field contains an annotation
private fun ClassNode.getField(): FieldNode? {
returnfields? .firstOrNull { field ->var hasAnnotation = falsefield? .visibleAnnotations? .forEach { annotation ->if (annotation.desc == "Lcom/wallstreetcn/sample/adapter/Test;") {
hasAnnotation = true
}
}
hasAnnotation
}
}
}
Copy the code
This time, I have drawn a flowchart of this part of the logic, so that you can understand this part of the code. Here by the way, I will expand the pain of using ClassVisitor before, this place may be my way of operation. Asm operates on.class files. Each inner class is actually a.class file. This part of the scan is separate. Because the instances of both classes are different, and then MY whole body feels a little split.
This time with ClassNode, I save most classNodes in HashMap, and then use outClassName to get the ClassNode instance so that I can modify it.
This is basically all of my code for the stub, which is basically string matching, just because bytecode and Java are not the same, and bytecode readability is a little bit worse, I also amway you with asm Bytecode Viewer. It still smells good.
conclusion
I’ve been working on this Github project on and off for about three years. From the first version of the code can only copy others, and then each debug can only send a local AAR, and then re-compile debugging. The ability to compile incrementally and buildSrc is completed when you double click later. Then at the beginning of this year, I completed the Transform of ThreadPool Hook and completed the bytecode replacement calling class, mainly to solve the online stability problem. At the end of the year, I changed the compilation mode, and recently I refactored the original AutoTrack and passed the parameters.
I think this project has actually seen my skills grow. I don’t think I’m a talented person. I think sometimes you can work hard and push yourself to learn something that you didn’t know before.