preface
In fact, the Transform API plays a very important role in the packaging process of an Android project, such as the well-known obfuscating processing, class file to dex file processing, are completed through the Transform API. This article mainly focuses on Transform:
Transform APIThe use and principle of- Bytecode processing framework
ASMUse skills Transform APIUse groping in application engineering
The use and principle of Transform
What is the Transform
Since version 1.5.0-Beta1, the Android Gradle plugin includes a Transform API, which allows third-party plug-ins to process compiled class files before they are converted into dex files. Using the Transform API, we can completely ignore the process of generating and executing tasks. It allows us to focus on how to process the input class file
The use of the Transform
The registration and use of Transform is straightforward, and within our custom plugin, We can through the android. RegisterTransform (theTransform) or android. RegisterTransform (theTransform, dependencies). You can register.
class DemoPlugin: Plugin<Project> {
override fun apply(target: Project) {
val android = target.extensions.findByType(BaseExtension::class.java)android? .registerTransform(DemoTransform()) } }Copy the codeAnd we are a custom Transform inheritance in com. Android. Build. API. The Transform. Transform, we can see the javaDoc, the following code is relatively common Transform process template
class DemoTransform: Transform() {
/** * transform name */
override fun getName(a): String = "DemoTransform"
There are two types of input files we can work with: compiled Java code and resource files (not files under RES but resources within Assests) */
override fun getInputTypes(a): MutableSet<QualifiedContent.ContentType> = TransformManager.CONTENT_CLASS
/** * Whether delta is supported * If delta execution is supported, the change input may contain a list of modified/deleted/added files */
override fun isIncremental(a): Boolean = false
/** * specifies the scope */
override fun getScopes(a): MutableSet<in QualifiedContent.Scope> = TransformManager.SCOPE_FULL_PROJECT
/** * execute the main function of the transform */
override fun transform(transformInvocation: TransformInvocation?).{ transformInvocation? .inputs? .forEach {// Input source is folder type
it.directoryInputs.forEach {directoryInput->
with(directoryInput){
// TODO performs bytecode operations on folders
val dest = transformInvocation.outputProvider.getContentLocation(
name,
contentTypes,
scopes,
Format.DIRECTORY
)
file.copyTo(dest)
}
}
// Input source is jar package type
it.jarInputs.forEach { jarInput->
with(jarInput){
// TODO handles Jar files
val dest = transformInvocation.outputProvider.getContentLocation(
name,
contentTypes,
scopes,
Format.JAR
)
file.copyTo(dest)
}
}
}
}
}
Copy the codeEach Transform declares its scope, what it’s working on, what it does, and what it outputs.
scope
The transform #getScopes method lets you declare a custom transform scope. The scopes specified include the following
| QualifiedContent.Scope | |
|---|---|
| EXTERNAL_LIBRARIES | Contains only external libraries |
| PROJECT | Only works on the project itself |
| PROVIDED_ONLY | Remote dependencies on compileOnly are supported |
| SUB_PROJECTS | Submodule content |
| TESTED_CODE | The code for the current variant test and the dependencies that include the test |
Function object
Transform#getInputTypes allows us to declare its objects. There are only two types of objects that we can specify
| QualifiedContent.ContentType | |
|---|---|
| CLASSES | Compiled content of Java code, including folders and compiled class files in Jar packages |
| RESOURCES | Content retrieved based on the resource |
SCOPE_FULL_PROJECT: transformManager.scope_full_project: transformManager.scope_full_project: transformManager.scope_full_project If it is a library-registered transform, we can only specify transformManager.project_only, We can crash at LibraryTaskManager# createTasksForVariantScope see restrictions related to an error code
Sets.SetView<? super Scope> difference =
Sets.difference(transform.getScopes(), TransformManager.PROJECT_ONLY);
if(! difference.isEmpty()) { String scopes = difference.toString(); globalScope .getAndroidBuilder() .getIssueReporter() .reportError( Type.GENERIC,new EvalIssueException(
String.format(
"Transforms with scopes '%s' cannot be applied to library projects.",
scopes)));
}
Copy the codeThe main object we use is transformManager.content_class
TransformInvocation
We handle our intermediate transformation by implementing the Transform#transform method, and the intermediate information is passed through the TransformInvocation object
public interface TransformInvocation {
/** * context of the transform */
@NonNull
Context getContext(a);
/** * returns the input source of the transform */
@NonNull
Collection<TransformInput> getInputs(a);
/** * returns the referenced input source */
@NonNull Collection<TransformInput> getReferencedInputs(a);
/** * Additional input source */
@NonNull Collection<SecondaryInput> getSecondaryInputs(a);
/**
* 输出源
*/
@Nullable
TransformOutputProvider getOutputProvider(a);
/** * whether to increment */
boolean isIncremental(a);
}
Copy the codeIn terms of input sources, we can roughly divide them into consumption and reference sources and additional input sources
A consumerThat’s the kind of object that we need to do the transform operation, and that’s the kind of object that after we process it we have to specify that the output goes to the next level, and we’re going to go throughgetInputs()Gets the input source to consume, and after the transformation, we must also pass the SettingsgetInputTypes()andgetScopes()To specify the output source to be transmitted to the next transform.- The reference input source means that we do not perform the transform operation, but may be used when viewing, so we do not need to output to the next level, through overwrite
getReferencedScopes()After specifying the scope of our referential input source, we can passTransformInvocation#getReferencedInputs()Gets the referenced input source - In addition, we can also define additional input sources for use by the next level, which we rarely use in normal development, but like
ProGuardTransformTXT to the next level; The same likeDexMergerTransformIf it is openedmultiDexThe maindexlist. TXT file will be passed to the next level
The principle of the Transform
The execution chain of Transform
Now that we have a rough idea of how it works, let’s see how it works (this source code is based on gradle plugin version 3.3.2). The com.android.application and com.android.library plug-ins of Android inherit from BasePlugin, and their main execution sequence can be divided into three steps
- The project configuration
- The extension of the configuration
- The task of creating
Inside the BaseExtension maintains a transforms collection objects, the android. RegisterTransform (theTransform) is in fact our custom transform instance added to the list of objects. In the 3.3.2 source code, it can also be understood in this way. In BasePlugin#createAndroidTasks, we create the associated build tasks for each variant through VariantManager#createAndroidTasks, Finally through TaskManager# createTasksForVariantScope (application plug-in finally realization method in TaskManager# createPostCompilationTasks, And library plug-in finally realization method in LibraryTaskManager# createTasksForVariantScope) method to obtain BaseExtension maintenance transforms in the object, Transform the corresponding Transform object into a Task via TransformManager#addTransform, registered in the TaskFactory. Here, about the execution process of a series of Transform tasks, we can choose to look at the relevant Transform process in application. Due to the length, we can look at the relevant source code by ourselves. Here’s the transform The task flow is from Desugar->MergeJavaRes-> custom Transform ->MergeClasses->Shrinker ->MultiD Ex ->BundleMultiDex->Dex->ResourcesShrinker->DexSplitter.
TransformManager
The TransformManager manages all the transforms of the project. It maintains streams, a TransformStream collection, inside the TransformManager. Each new Transform consumes the corresponding stream. Then add the processed streams to Streams
public class TransformManager extends FilterableStreamCollection{
private final List<TransformStream> streams = Lists.newArrayList();
}
Copy the codeWe can look at its core method, addTransform
@NonNull
public <T extends Transform> Optional<TaskProvider<TransformTask>> addTransform(
@NonNull TaskFactory taskFactory,
@NonNull TransformVariantScope scope,
@NonNull T transform,
@Nullable PreConfigAction preConfigAction,
@Nullable TaskConfigAction<TransformTask> configAction,
@Nullable TaskProviderCallback<TransformTask> providerCallback) {
...
List<TransformStream> inputStreams = Lists.newArrayList();
// The transform task's naming rule definition
String taskName = scope.getTaskName(getTaskNamePrefix(transform));
// Get the reference stream
List<TransformStream> referencedStreams = grabReferencedStreams(transform);
// Find the input stream and calculate the output stream through the transform
IntermediateStream outputStream = findTransformStreams(
transform,
scope,
inputStreams,
taskName,
scope.getGlobalScope().getBuildDir());
// The ellipsis code is used to check whether the input stream and reference stream are empty. Theoretically, they cannot be empty. If they are empty, then there is a problem with the transform. transforms.add(transform);// Create the transform task
return Optional.of(
taskFactory.register(
new TransformTask.CreationAction<>(
scope.getFullVariantName(),
taskName,
transform,
inputStreams,
referencedStreams,
outputStream,
recorder),
preConfigAction,
configAction,
providerCallback));
}
Copy the codeAdd a Transform management to TransformManager, and the process can be divided into the following steps
- Define the transform Task name
static String getTaskNamePrefix(@NonNull Transform transform) {
StringBuilder sb = new StringBuilder(100);
sb.append("transform");
sb.append(
transform
.getInputTypes()
.stream()
.map(
inputType ->
CaseFormat.UPPER_UNDERSCORE.to(
CaseFormat.UPPER_CAMEL, inputType.name()))
.sorted() // Keep the order stable.
.collect(Collectors.joining("And")));
sb.append("With");
StringHelper.appendCapitalized(sb, transform.getName());
sb.append("For");
return sb.toString();
}
Copy the codeFrom the code above, Transform ${inputtype1. name}And${inputtype2. name}With${transform.name}For${variantName} The corresponding transform Task can also be verified by the generated transform Task
streams
private List<TransformStream> grabReferencedStreams(@NonNull Transform transform) {
Set<? superScope> requestedScopes = transform.getReferencedScopes(); . List<TransformStream> streamMatches = Lists.newArrayListWithExpectedSize(streams.size()); Set<ContentType> requestedTypes = transform.getInputTypes();for (TransformStream stream : streams) {
Set<ContentType> availableTypes = stream.getContentTypes();
Set<? super Scope> availableScopes = stream.getScopes();
Set<ContentType> commonTypes = Sets.intersection(requestedTypes,
availableTypes);
Set<? super Scope> commonScopes = Sets.intersection(requestedScopes, availableScopes);
if(! commonTypes.isEmpty() && ! commonScopes.isEmpty()) { streamMatches.add(stream); }}return streamMatches;
}
Copy the code- According to SCOPE and INPUT_TYPE defined in transform, obtain the corresponding consumptive input streams, remove these consumptive input streams in Streams, and retain the streams that cannot match SCOPE and INPUT_TYPE. Build a new output stream and add it to Streams for management
private IntermediateStream findTransformStreams( @NonNull Transform transform, @NonNull TransformVariantScope scope, @NonNull List
inputStreams, @NonNull String taskName, @NonNull File buildDir)
{
Set<? superScope> requestedScopes = transform.getScopes(); . Set<ContentType> requestedTypes = transform.getInputTypes();// Get the consumptive input stream
// Remove the corresponding consumption input streams from Streams
consumeStreams(requestedScopes, requestedTypes, inputStreams);
// Create an output stream
Set<ContentType> outputTypes = transform.getOutputTypes();
// Create a file-related path for the output stream transformation
File outRootFolder =
FileUtils.join(
buildDir,
StringHelper.toStrings(
AndroidProject.FD_INTERMEDIATES,
FD_TRANSFORMS,
transform.getName(),
scope.getDirectorySegments()));
// Create an output stream
IntermediateStream outputStream =
IntermediateStream.builder(
project,
transform.getName() + "-" + scope.getFullVariantName(),
taskName)
.addContentTypes(outputTypes)
.addScopes(requestedScopes)
.setRootLocation(outRootFolder)
.build();
streams.add(outputStream);
return outputStream;
}
Copy the code- Finally, create the TransformTask and register it with TaskManager
TransformTask
How do WE trigger our Transform#transform method to be executed in the TaskAction corresponding to the TransformTask
void transform(final IncrementalTaskInputs incrementalTaskInputs)
throws IOException, TransformException, InterruptedException {
final ReferenceHolder<List<TransformInput>> consumedInputs = ReferenceHolder.empty();
final ReferenceHolder<List<TransformInput>> referencedInputs = ReferenceHolder.empty();
final ReferenceHolder<Boolean> isIncremental = ReferenceHolder.empty();
final ReferenceHolder<Collection<SecondaryInput>> changedSecondaryInputs =
ReferenceHolder.empty();
isIncremental.setValue(transform.isIncremental() && incrementalTaskInputs.isIncremental());
GradleTransformExecution preExecutionInfo =
GradleTransformExecution.newBuilder()
.setType(AnalyticsUtil.getTransformType(transform.getClass()).getNumber())
.setIsIncremental(isIncremental.getValue())
.build();
// Some incremental processing, including determining changes in input streams (referential and consumable) in incremental mode. GradleTransformExecution executionInfo = preExecutionInfo.toBuilder().setIsIncremental(isIncremental.getValue()).build(); . transform.transform(new TransformInvocationBuilder(TransformTask.this) .addInputs(consumedInputs.getValue()) .addReferencedInputs(referencedInputs.getValue()) .addSecondaryInputs(changedSecondaryInputs.getValue()) .addOutputProvider( outputStream ! =null
? outputStream.asOutput(
isIncremental.getValue())
: null)
.setIncrementalMode(isIncremental.getValue())
.build());
if(outputStream ! =null) { outputStream.save(); }}Copy the codeWe now know the timing, location, and principles of the custom Transform execution. So, now that we have all the compiled bytecode, what do we do with it? So we can look at ASM
The use of the ASM
Common frameworks for handling bytecode include AspectJ, Javasist, and ASM. There are many articles about the selection of the framework on the Internet. From the processing speed and memory occupancy rate, ASM is obviously better than the other two frameworks. This article focuses on the use of ASM.
What is the ASM
ASM is a general-purpose Java bytecode manipulation and analysis framework. It can be used to modify existing classes or to generate classes dynamically directly in binary form. ASM provides some common bytecode conversion and analysis algorithms from which you can build custom complex conversion and code analysis tools. The ASM library provides two apis for generating and transforming compiled classes: the Core API provides event-based class representations, and the Tree API provides object-based representations. Because the event-based API(Core API) does not require a number of objects representing the class to be stored in memory, it is superior to the object-based API(Tree API) in terms of execution speed and memory footprint. However, event-based apis are more difficult to use than object-based apis in terms of usage scenarios, such as when we need to adjust for an object. Since a class can only be managed by one API, we should use the API for each scenario
ASM plug-in
The use of ASM requires a certain learning cost, which can be learned by using ASM Bytecode Outline plug-in. The corresponding plug-in can be found in the plug-in browser of AS
@RouteModule
public class ASMTest {}Copy the code
Transform APIGroping for use in application engineering
The role in component communication
Transform the API has many applications in modular engineering direction, at present in our project in the development of routing framework, through its to do the automatic static registration module, at the same time, considering the uncertainty routing through maintenance agreement document (page routing address maintenance is not timely lead to corresponding development unable to update the corresponding code), We did constant management of routing. Firstly, we collected routing information by scanning the code of the whole project and established the original basic information file of routing in accordance with certain rules. Registered by corresponding to the original information through variant# registerJavaGeneratingTask file generated constants corresponding Java file sinking in the basic task of common component, so that the upper relies on the basic components of the project can use routing through direct call constants. In the case of code isolation of each component, the original information file can be transmitted by the aar of the component, and the corresponding constant table can still be generated by following the above steps, while the problem of class duplication exists, and the combination can be processed by custom Transform
The role of service monitoring
In application engineering, we usually have requirements for network monitoring and application performance detection (including page load time and even the time spent on each method call, which may be warned if the threshold is exceeded), which cannot be embedded in business code and can be handled based on the Transform API. As for burying points, we can also realize the function of automatic burying points through Transform, and transfer as much field information as possible through ASM Core and ASM Tree. Some of these have been realized in our project, and some need to be optimized or realized.
other
Transform Core Api and ASM Tree Api: Transform Core Api and ASM Tree Api
Relevant reference
- ASM User Guide
- Play with bytecode in Android projects
- AOP tools: INTRODUCTION to ASM 3.0
- ASM website