preface
To be a good Android developer, you need a complete set ofThe knowledge systemHere, let us grow together into what we think ~.
At present, Gradle automation technology is becoming more and more important. Many students have been able to make their own Gradle plug-ins, but there are always some “memes” left in our minds, reminding us all the time, do you really master it? For example, “Meme 1” : The overall implementation architecture of Gradle plug-ins? I:… What are the most important optimizations or improvements in Android Gradle plugin update history? I:… “Meme 3” : What is the core process of Gradle builds? I:… “, “Meme 4” : How do dependencies work in Gradle? I:… , “Meme 5” : The AppPlugin build process? I:… , “Meme 6” : assembleDebug packaging process? I:… “, “Meme 7” : Can you explain the implementation principles of some important tasks? I:… . Are there many points that are not really understood?
Mind mapping
directory
- Gradle plugin implementation architecture Overview
- Second,Understand the update history of the Android Gradle Plugin
- Android Gradle Plugin V3.5.0 (August 2019)
- 2. Android Gradle Plugin V3.4.0 (April 2019)
- Android Gradle Plugin V3.3.0 (2019 年 1 月)
- 4. Android Gradle Plugin V3.2.0 (September 2018)
- 5. Android Gradle Plugin V3.1.0 (March 2018)
- 6. Android Gradle Plugin V3.0.0 (October 2017)
- 7. Android Gradle Plugin V2.3.0 (February 2017)
- Three,Gradle build core process parsing
- 1, LoadSettings
- 2, the Configure
- 3, TaskGraph
- 4, RunTasks
- 5, Finished
- Four,Gradle dependency implementation principles
- 1. Indirectly call the Add method of DefaultDependencyHandler via MethodMissing mechanism to add dependencies.
- 2. Different dependency declarations are actually converted by different converters.
- Artifacts are Artifacts in nature.
- 4. What is Configuration in Gradle?
- 5. How does the Task publish its Artifacts?
- Five,AppPlugin builds the process
- 1. Preparation
- 2. ConfigureProject configureProject
- 3. ConfigureExtension configureExtension
- 4, TaskManager# createTasksBeforeEvaluate create don’t rely on the flavor of the task
- BasePlugin#createAndroidTasks create a build task
- Vi.AssembleDebug packaging process analysis
- 1. Review of Android packaging process
- 2, assmableDebug packaging process analysis
- Seven,Important tasks implement source code analysis
- 1. Resource processing related tasks
- 2. The process of packaging a Class file into a Dex file
- Eight,conclusion
- The last
5. AppPlugin construction process
Add the Android Gradle Plugin dependency to your project as follows:
compile 'com. Android. Tools. Build: gradle: 3.0.1'
Copy the codeAs we all know, we can build a Moudle as an Android project by configuring the following plugin application code in build.gradle:
apply plugin: 'com.android.application'
Copy the codeWhen the Apply Plugin: ‘com.android.application’ configuration is implemented, the process of building the AppPlugin begins. Let’s analyze the process of building the AppPlugin.
‘. Com. Android application corresponding plug-in properties for ‘com. Android. Internal. Application’, internal indicate the plug-in implementation class as shown below:
implementation-class=com.android.build.gradle.internal.plugins.AppPlugin
Copy the codeThe key implementation code in AppPlugin is as follows:
/** Gradle plugin class for 'application' projects, applied on the base application module */
public class AppPlugin extends AbstractAppPlugin {...// Apply the specified plugin, which is an empty implementation
@Override
protected void pluginSpecificApply(@NonNull Project project) {}...// Get an extension class: Every Application Plugin provides a corresponding Android Extension
@Override
@NonNull
protected Class<? extends AppExtension> getExtensionClass() {
returnBaseAppModuleExtension.class; }... }Copy the codeSo where is the apply method called?
First, let’s sort out the inheritance relationship of AppPlugin, as follows:
AppPlugin => AbstractAppPlugin => BasePlugin
Copy the codeThe Apply method is in the BasePlugin class, the base class that applies to all Android plugins, and some preparatory work is done in the Apply method.
1. Preparation
When the compiler executes a groovy line of apply Plugin, Gradle finally calls back the Apply method of the BasePlugin base class, as shown below:
@Override
public final void apply(@NonNull Project project) {
CrashReporting.runAction(
() -> {
basePluginApply(project);
pluginSpecificApply(project);
});
}
Copy the codeThe basePluginApply method is called in the Apply method, with the following source code:
private void basePluginApply(@NonNull Project project) {.../ / 1, DependencyResolutionChecks will check and make sure that during the configuration phase is not to rely on.
DependencyResolutionChecks.registerDependencyCheck(project, projectOptions);
// 2, apply an AndroidBasePlugin, the purpose is to let other plugin authors distinguish the current application is an Android plugin.
project.getPluginManager().apply(AndroidBasePlugin.class);
// 3. Check whether the project path has any errors and raise StopExecutionException if any.
checkPathForErrors();
// Check the structure of the child moudle: the current version checks whether the two modules have the same identity (group + name) and throws a StopExecutionException if they do. (Componentization requires prefixes for resources in different moudles)
checkModulesForErrors();
// 5. Plug-in initialization must be performed immediately. Also, note that Gradle Deamon never executes two build processes at the same time.
PluginInitializer.initialize(project);
/ / 6, initialization is used to record ProcessProfileWriterFactory configuration information in the process of building the factory instance
RecordingBuildListener buildListener = ProfilerInitializer.init(project, projectOptions);
// 7. Set the Android Plugin version, plugin type, plugin generator, and project options for project
ProcessProfileWriter.getProject(project.getPath())
.setAndroidPluginVersion(Version.ANDROID_GRADLE_PLUGIN_VERSION)
.setAndroidPlugin(getAnalyticsPluginType())
.setPluginGeneration(GradleBuildProject.PluginGeneration.FIRST)
.setOptions(AnalyticsUtil.toProto(projectOptions));
// Configuration project
threadRecorder.record(
ExecutionType.BASE_PLUGIN_PROJECT_CONFIGURE,
project.getPath(),
null.this::configureProject);
/ / configure the Extension
threadRecorder.record(
ExecutionType.BASE_PLUGIN_PROJECT_BASE_EXTENSION_CREATION,
project.getPath(),
null.this::configureExtension);
/ / create the Tasks
threadRecorder.record(
ExecutionType.BASE_PLUGIN_PROJECT_TASKS_CREATION,
project.getPath(),
null.this::createTasks);
}
Copy the codeAs you can see, the first four steps are all checks, and the last three are initialization and configuration of the plug-in. Let’s sort out the tasks in the preparation project, as follows:
- 1.
Plug-in check operation- 1), using DependencyResolutionChecks class to check and make sure that the configuration stage is not to rely on.
- 2) To apply an AndroidBasePlugin, the purpose is to let other plugin authors distinguish the current application is an Android plugin.
- 3) Check the project path for errors and throw StopExecutionException if any.
- 4) Check the structure of the child moudle, the current version will check whether the two modules have the same identity (group + name), if there is a StopExecutionException. (Think of componentization in different moudles that require prefixes for resources)
- 2,
Initialize and configure information about the plug-in- 1) Perform plug-in initialization immediately.
- 2), the initialization is used to record ProcessProfileWriterFactory configuration information in the process of building the factory instance.
- 3) Set android Plugin Version, plug-in type, plug-in generator and Project options for project.
2. ConfigureProject configureProject
After the preparation of Plugin project is completed, it will execute the configureProject method in BasePlugin to configure the project, its source code is as follows:
private void configureProject(a) {...// Create DataBindingBuilder instance.
dataBindingBuilder = new DataBindingBuilder();
dataBindingBuilder.setPrintMachineReadableOutput(
SyncOptions.getErrorFormatMode(projectOptions) == ErrorFormatMode.MACHINE_PARSABLE);
// 2. Force the plugin version to be no less than the currently supported minimum version, otherwise an exception will be thrown.
GradlePluginUtils.enforceMinimumVersionsOfPlugins(project, syncIssueHandler);
// 3, use the Java Plugin.
project.getPlugins().apply(JavaBasePlugin.class);
// 4. If the buildCache option is enabled, buildCache instances will be created so that the cache can be reused later.
@Nullable
FileCache buildCache = BuildCacheUtils.createBuildCacheIfEnabled(project, projectOptions);
// 5. This callback will be executed after the entire project (note not after the current moudle execution). Since this callback is called by each project, it may be executed multiple times.
// After the entire project is built, the resource is reclaimed, the cache is cleared, and all thread pool components started during this process are closed.
gradle.addBuildListener(
new BuildAdapter() {
@Override
public void buildFinished(@NonNull BuildResult buildResult) {
// Do not run buildFinished for included project in composite build.
if(buildResult.getGradle().getParent() ! =null) {
return;
}
ModelBuilder.clearCaches();
Workers.INSTANCE.shutdown();
sdkComponents.unload();
SdkLocator.resetCache();
ConstraintHandler.clearCache();
CachedAnnotationProcessorDetector.clearCache();
threadRecorder.record(
ExecutionType.BASE_PLUGIN_BUILD_FINISHED,
project.getPath(),
null, () - > {if(! projectOptions.get( BooleanOption.KEEP_SERVICES_BETWEEN_BUILDS)) { WorkerActionServiceRegistry.INSTANCE .shutdownAllRegisteredServices( ForkJoinPool.commonPool()); } Main.clearInternTables(); }); DeprecationReporterImpl.Companion.clean(); }}); . }Copy the codeFinally, let’s review the five main tasks performed in configureProject, as follows:
- 1) Create DataBindingBuilder instance
- 2) Force the use of at least the minimum version of the currently supported plug-in, otherwise an exception will be thrown.
- 3) Use Java Plugin.
- 4) if the buildCache option is enabled, buildCache instances are created so that the cache can be reused later.
- 5) This callback will be executed after the entire project execution (note not after the current moudle execution), because this callback will be called by every project, so it may be executed multiple times. Finally, after the entire project is built, the resource is reclaimed, the cache is cleared, and all thread pool components started during this process are shut down.
3. ConfigureExtension configureExtension
Then, we see the configureExtension method of BasePlugin, whose core source code is as follows:
private void configureExtension(a) {
// create a container instance for buildType, productFlavor, signingConfig..final NamedDomainObjectContainer<BaseVariantOutput> buildOutputs =
project.container(BaseVariantOutput.class);
// create an extension property configuration named buildOutputs.
project.getExtensions().add("buildOutputs", buildOutputs); .// create an Android DSL closure.
extension =
createExtension(
project,
projectOptions,
globalScope,
buildTypeContainer,
productFlavorContainer,
signingConfigContainer,
buildOutputs,
sourceSetManager,
extraModelInfo);
// 4. Create the Android DSL closure for the global Settings.
globalScope.setExtension(extension);
/ / 5, and create a ApplicationVariantFactory instance, APKs for production.
variantFactory = createVariantFactory(globalScope);
// create a ApplicationTaskManager instance to create Tasks for the Android application project.
taskManager =
createTaskManager(
globalScope,
project,
projectOptions,
dataBindingBuilder,
extension,
variantFactory,
registry,
threadRecorder);
VariantManager (); VariantManager ();
variantManager =
new VariantManager(
globalScope,
project,
projectOptions,
extension,
variantFactory,
taskManager,
sourceSetManager,
threadRecorder);
// map whenObjectAdded callbacks to singingConfig.
signingConfigContainer.whenObjectAdded(variantManager::addSigningConfig);
If it is not the DynamicFeature (responsible for adding an optional APK module), a Debug signingConfig DSL object is initialized and set to the default buildType DSL.
buildTypeContainer.whenObjectAdded(
buildType -> {
if (!this.getClass().isAssignableFrom(DynamicFeaturePlugin.class)) {
SigningConfig signingConfig =
signingConfigContainer.findByName(BuilderConstants.DEBUG);
buildType.init(signingConfig);
} else {
// initialize it without the signingConfig for dynamic-features.
buildType.init();
}
variantManager.addBuildType(buildType);
});
// map whenObjectAdded callbacks to the productFlavor container.
productFlavorContainer.whenObjectAdded(variantManager::addProductFlavor);
When the whenObjectRemoved callback is executed, an UnsupportedAction exception will be raised.
signingConfigContainer.whenObjectRemoved(
new UnsupportedAction("Removing signingConfigs is not supported."));
buildTypeContainer.whenObjectRemoved(
new UnsupportedAction("Removing build types is not supported."));
productFlavorContainer.whenObjectRemoved(
new UnsupportedAction("Removing product flavors is not supported."));
// create DSLS of types signingConfig DEBUG, buildType Debug, buildType Release in sequence.
variantFactory.createDefaultComponents(
buildTypeContainer, productFlavorContainer, signingConfigContainer);
}
Copy the codeFinally, let’s comb through the tasks in configureExtension as follows:
- Create a container instance for buildType, productFlavor, signingConfig.
- 2) Create an extension property configuration named buildOutputs.
- Create an Android DSL closure.
- 4) Create the Android DSL closure for the global Settings.
- 5), create a ApplicationVariantFactory instance, APKs for production.
- Create an instance of ApplicationTaskManager to create Tasks for the Android application project.
- 7) Create a VariantManager instance to create and manage Variant.
- Map the whenObjectAdded callbacks to the singingConfig container.
- Map whenObjectAdded callbacks to buildType containers. If not the DynamicFeature (responsible for adding an optional APK module), a Debug signingConfig DSL object is initialized and set to the default buildType DSL.
- Map whenObjectAdded callbacks to the productFlavor container.
- 11) Map whenObjectRemoved to the container. When the whenObjectRemoved callback is executed, an UnsupportedAction exception is raised.
- Build DSLS of signingConfig Debug, buildType Debug, buildType Release in sequence.
The core processing can be summarized as the following four aspects:
- 1) Create an AppExtension, the Android DSL in build.gradle.
- 2). Set up variant factories, Task managers, and Variant managers in turn.
- 3) Register new/remove config callback, including signingConfig, buildType, productFlavor.
- 4) Create the default Debug signature, create debug and release buildTypes in sequence.
At the end of the apply method of the BasePlugin, createTasks is called to createTasks as follows:
private void createTasks(a) {
Create Tasks before EVALUATE
threadRecorder.record(
ExecutionType.TASK_MANAGER_CREATE_TASKS,
project.getPath(),
null,
() -> taskManager.createTasksBeforeEvaluate());
// 2. Create Android Tasks
project.afterEvaluate(
CrashReporting.afterEvaluate(
p -> {
sourceSetManager.runBuildableArtifactsActions();
threadRecorder.record(
ExecutionType.BASE_PLUGIN_CREATE_ANDROID_TASKS,
project.getPath(),
null.this::createAndroidTasks);
}));
}
Copy the codeAs you can see, createTasks is divided into two kinds of Task creation method, namely createTasksBeforeEvaluate and createAndroidTasks.
Next, let’s analyze the implementation process in detail.
4, TaskManager# createTasksBeforeEvaluate create don’t rely on the flavor of the task
TaskManager createTasksBeforeEvaluate method to Task container registered a series of Task, including UninstallAllTask, deviceCheckTask, connectedCheckTask, preBuild, extractProguardFiles, sourceSetsTask, assembleAndroidTest, com PileLintTask and so on.
BasePlugin#createAndroidTasks create a build task
In the createAndroidTasks method of BasePlugin, flavors related data is generated, and corresponding Task instances are created according to flavor and registered in the Task container. The core source code is as follows:
@VisibleForTesting
final void createAndroidTasks(a) {
CompileSdkVersion, plugin configuration conflict detection (e.g. JavaPlugin, Retrolambda).
// Create some basic or generic Tasks.
// 2. Set Project Path, CompileSdk, BuildToolsVersionThe information such as, Splits, KotlinPluginVersion, FirebasePerformancePluginVersion write to the configuration of the Project. ProcessProfileWriter.getProject(project.getPath()) .setCompileSdk(extension.getCompileSdkVersion()) .setBuildToolsVersion(extension.getBuildToolsRevision().toString()) .setSplits(AnalyticsUtil.toProto(extension.getSplits())); String kotlinPluginVersion = getKotlinPluginVersion();if(kotlinPluginVersion ! =null) {
ProcessProfileWriter.getProject(project.getPath())
.setKotlinPluginVersion(kotlinPluginVersion);
}
AnalyticsUtil.recordFirebasePerformancePluginVersion(project);
// create Tasks for the application.
List<VariantScope> variantScopes = variantManager.createAndroidTasks();
// Create basic or generic Tasks and do generic processing.
}
Copy the codeCreateAndroidTasks does three things in addition to creating some basic or generic Tasks and doing some generic processing:
- 1) CompileSdkVersion, plug-in configuration conflict detection (e.g. JavaPlugin, Retrolambda plug-in).
- 2), the Project Path, CompileSdk, BuildToolsVersion, Splits, KotlinPluginVersion, FirebasePerformancePluginVersion information written to the Project In the configuration of.
- 3) Create application Tasks.
We need to focus on the createAndroidTasks method of variantManager.
/** Variant/Task creation entry point. */
public List<VariantScope> createAndroidTasks(a) {...// Create a test task at the engineering level.taskManager.createTopLevelTestTasks(! productFlavors.isEmpty());// Create Tasks for all variantScopes.
for (final VariantScope variantScope : variantScopes) {
createTasksForVariantData(variantScope);
}
// 3. Create Tasks related to the report.
taskManager.createReportTasks(variantScopes);
return variantScopes;
}
Copy the codeAs you can see, there are three processes in the createAndroidTasks method, as follows:
- 1) Create engineering level test tasks.
- 2) Iterate over all variantscopes and create corresponding Tasks for their variant data.
- 3) Create Tasks related to the report.
Then, we continue to look at how createTasksForVariantData method for each specified Variant type to create the corresponding Tasks, its core source as shown below:
// Create Tasks for each Variant type specified
public void createTasksForVariantData(final VariantScope variantScope) {
final BaseVariantData variantData = variantScope.getVariantData();
final VariantType variantType = variantData.getType();
final GradleVariantConfiguration variantConfig = variantScope.getVariantConfiguration();
// create Assemble Task.
taskManager.createAssembleTask(variantData);
// 2. If the variantType is base moudle, the corresponding bundle Task will be created. Note that a base moudle is a moudle that contains functionality, while a moudle for test does not.
if (variantType.isBaseModule()) {
taskManager.createBundleTask(variantData);
}
// 3. If the variantType is a Test moudle (which is a component of a test), the corresponding test variant is created.
if (variantType.isTestComponent()) {
// add varier-specific, build type multi-flavor, defaultConfig dependencies to the current variantData..// 2) add the dependency of render script if it is supported.
if (testedVariantData.getVariantConfiguration().getRenderscriptSupportModeEnabled()) {
project.getDependencies()
.add(
variantDep.getCompileClasspath().getName(),
project.files(
globalScope
.getSdkComponents()
.getRenderScriptSupportJarProvider()));
}
// 3) AndroidTestVariantTask is created if the Variant currently outputs an APK, which is an Android test (commonly used to automate UI tests).
if (variantType.isApk()) { // ANDROID_TEST
if (variantConfig.isLegacyMultiDexMode()) {
String multiDexInstrumentationDep =
globalScope.getProjectOptions().get(BooleanOption.USE_ANDROID_X)
? ANDROIDX_MULTIDEX_MULTIDEX_INSTRUMENTATION
: COM_ANDROID_SUPPORT_MULTIDEX_INSTRUMENTATION;
project.getDependencies()
.add(
variantDep.getCompileClasspath().getName(),
multiDexInstrumentationDep);
project.getDependencies()
.add(
variantDep.getRuntimeClasspath().getName(),
multiDexInstrumentationDep);
}
taskManager.createAndroidTestVariantTasks(
(TestVariantData) variantData,
variantScopes
.stream()
.filter(TaskManager::isLintVariant)
.collect(Collectors.toList()));
} else { // UNIT_TEST
UnitTestVariantTask UnitTestVariantTask UnitTestVariantTask UnitTestVariantTask taskManager.createUnitTestVariantTasks((TestVariantData) variantData);}}else {
/ / 4, if it is not a Test moudle, will call ApplicationTaskManager createTasksForVariantScope method.taskManager.createTasksForVariantScope( variantScope, variantScopes .stream() .filter(TaskManager::isLintVariant) .collect(Collectors.toList())); }}Copy the codeIn createTasksForVariantData approach for each specified Variant type created with the corresponding Tasks, the method of processing logic is as follows:
- 1. Create Assemble Task.
- 2. If the variantType is a Base moudle, the corresponding Bundle Task is created. Note that a base moudle is a moudle that contains functionality, while a moudle for test does not.
- 3,If the variantType is a Test moudle (which acts as a component of a test), the corresponding test variant is created.
- 1) Add varier-Specific, Build Type multi-flavor and defaultConfig dependencies to the current variantData.
- 2) Add render script dependencies if supported.
- 3) If the Variant currently outputs an APK, that is, an Android test (commonly used for UI automated testing), the corresponding AndroidTestVariantTask will be created.
- 4) Otherwise, the Test moudle is used to execute unit tests, and UnitTestVariantTask is created.
- 4, if it is not a Test moudle, will call ApplicationTaskManager createTasksForVariantScope method.
In the end, will perform to the ApplicationTaskManager createTasksForVariantScope method, in this method to create a suitable for application to build a series of Tasks.
Let’s examine these Tasks using assembleDebug’s packaging process.
6. Analysis of assembleDebug packaging process
Before analyzing a series of tasks in the assembleDebug build process, we need to review the Android packaging process (skip this if you’re familiar with it).
1. Review of Android packaging process
Android’s official compilation and packaging flow chart is as follows:
The rough packaging process can be summarized as the following four steps:
- 1) The compiler converts the APP source code into a DEX (Dalvik Executable) file (including bytecode running on Android devices) and all other content into compiled resources.
- 2) APK packer combines DEX files and compiled resources into a single APK. However, an APK must be signed before the application can be installed and deployed to an Android device.
- 3) The APK wrapper will use the corresponding keystore to publish the keystore to sign the APK.
- 4) Before generating the final APK, the packer optimizes the application using the Zipalign tool to reduce the amount of memory it takes up when running on the device.
To learn more about the details of the packaging process, we need to look at a more detailed old APK packaging flowchart, as shown below:
The detailed packaging process can be summarized as the following eight steps:
- 1. First, the. Aidl (Android Interface Description Language) file needs to be converted into a Java Interface file that can be processed by the compiler using the AIDL tool.
- 2. Meanwhile, Resource files (androidmanifest.xml, layout files, various XML resources, etc.) will be replaced by AAPT (Asset Packaging Tool) (Android Gradle Plugin 3.0.0 and later using AAPT2 AAPT is processed as the final resources.arsc, and an R.java file is generated to ensure easy access to these resources when the source code is written.
- 3. Then, r.Java, Java interface files, and Java source files are compiled with the Java Compiler, and eventually they are uniformly compiled into.class files.
- 4. Since.class is not a format recognized by the Android system, they need to be converted into Dalvik bytecodes (including compressing constant pools and removing redundant information) using the dex tool. This process also adds all the “third-party libraries” that the app depends on.
- 5. Next, use ApkBuilder to package resource files and DEX files to generate APK files.
- 6. Then, the system generates the initial APK file package through APkBuilder with DEX, resource package and other resources generated above.
- 7. Then, use Jarsigner or other signature tools to sign the APK to obtain the signed APK. In Debug mode, the keystore used for signing is the default value provided by the system; otherwise, you need to provide your own private key to complete the signing process.
- 8. Finally, if it is the official VERSION of APK, the ZipAlign tool will be used for alignment processing to improve the loading and running speed of the program. The process of alignment is to offset all resource files in the APK file by an integer multiple of 4 bytes from the starting position of the file. This will make the APK file faster to access through MMAP and reduce the memory footprint of the APK file when running on the device.
At this point, we have seen the entire APK compilation and packaging process.
So why compile XML resource files from text format to binary format?
The main reasons are as follows:
-
1. Less space: Because all the strings associated with the tags, attribute names, attribute values, and content of XML elements are uniformly collected into a string resource pool and de-duplicated. With this string resource pool, the original use of strings is replaced with an integer value indexed to the string resource pool, thereby reducing the file size.
-
2. Higher parsing efficiency: XML files in binary format are parsed faster. This is because XML elements in binary format no longer contain string values, thus avoiding string parsing and thus improving parsing efficiency.
How does the Android resource management framework quickly locate the most suitable resources?
It is based on two files, as follows:
- 1.
Resource ID file R.java:Each non-assets resource is given an ID value, which is defined as a constant in the R.java file. - 2,
Resource index table Resources.arsc:Describes the configuration information for resources that have ID values.
2, assmableDebug packaging process analysis
We can get the tasks we need to execute to package a Debug APK by using the following command:
quchao@quchaodeMacBook-Pro CustomPlugin % ./gradlew app:assembleDebug --console=plain
...
> Task :app:preBuild UP-TO-DATE
> Task :app:preDebugBuild UP-TO-DATE
> Task :app:generateDebugBuildConfig
> Task :app:javaPreCompileDebug
> Task :app:mainApkListPersistenceDebug
> Task :app:generateDebugResValues
> Task :app:createDebugCompatibleScreenManifests
> Task :app:extractDeepLinksDebug
> Task :app:compileDebugAidl NO-SOURCE
> Task :app:compileDebugRenderscript NO-SOURCE
> Task :app:generateDebugResources
> Task :app:processDebugManifest
> Task :app:mergeDebugResources
> Task :app:processDebugResources
> Task :app:compileDebugJavaWithJavac
> Task :app:compileDebugSources
> Task :app:mergeDebugShaders
> Task :app:compileDebugShaders
> Task :app:generateDebugAssets
> Task :app:mergeDebugAssets
> Task :app:processDebugJavaRes NO-SOURCE
> Task :app:checkDebugDuplicateClasses
> Task :app:dexBuilderDebug
> Task :app:mergeLibDexDebug
> Task :app:mergeDebugJavaResource
> Task :app:mergeDebugJniLibFolders
> Task :app:validateSigningDebug
> Task :app:mergeProjectDexDebug
> Task :app:mergeDebugNativeLibs
> Task :app:stripDebugDebugSymbols
> Task :app:desugarDebugFileDependencies
> Task :app:mergeExtDexDebug
> Task :app:packageDebug
> Task :app:assembleDebug
Copy the codeIn the TaskManager, there are two main methods used to create a Task, they createTasksBeforeEvaluate method with createTasksForVariantScope method respectively. It is important to note that createTasksForVariantScope method is an abstract method, create the Tasks their specific task distributed to TaskManager subclasses of processing, One of the most common subclasses is ApplicationTaskManager, which is the Task manager used to create Tasks in Android applications.
Most of the tasks in the packaging process are under this directory:
com.android.build.gradle.internal.tasks
Let’s take a look at the implementation classes and meanings of each Task required in the assembleDebug packaging process, as shown in the following table:
| Task | Corresponding implementation class | role |
|---|---|---|
| preBuild | AppPreBuildTask | A pre-created task for application Variant checks |
| preDebugBuild | The difference with preBuild is that this task is used for some Vrariant checks in the Debug environment | |
| generateDebugBuildConfig | GenerateBuildConfig | Generate BuildConfig classes that are relevant to the build target |
| javaPreCompileDebug | JavaPreCompileTask | Used to execute necessary actions prior to Java compilation |
| mainApkListPersistenceDebug | MainApkListPersistence | Used to persist APK data |
| generateDebugResValues | GenerateResValues | Generate Res resource type values |
| createDebugCompatibleScreenManifests | CompatibleScreensManifest | Generates a list of (compatible screens) nodes with a given screen density and size list |
| extractDeepLinksDebug | ExtractDeepLinksTask | It is used to extract a series of DeepLink (DeepLink technology, the main application scenario is to directly call the Android native app through the Web page, and transfer the required parameters to the app directly in the form of Uri, saving the user’s registration cost) |
| compileDebugAidl | AidlCompile | Compile the AIDL file |
| compileDebugRenderscript | RenderscriptCompile | Compile the Renderscript file |
| generateDebugResources | The TaskManager. CreateAnchorTasks method through taskFactory. Register (taskName) sign up for a task | Empty task, anchor point |
| processDebugManifest | ProcessApplicationManifest | Handling manifest files |
| mergeDebugResources | MergeResources | Use AAPT2 to merge resource files |
| processDebugResources | ProcessAndroidResources | Used to process resources and generate R.class files |
| compileDebugJavaWithJavac | JavaCompileCreationAction (this is an Action, can be seen in the gradle source register an Action can be get from TaskFactory and the matching Task, therefore, a Task or Action, The Action of Task) | Used to perform compilation of Java source code |
| compileDebugSources | The TaskManager. CreateAnchorTasks method through taskFactory. Register (taskName) sign up for a task | Empty task, anchor point use |
| mergeDebugShaders | MergeSourceSetFolders.MergeShaderSourceFoldersCreationAction | Merge Shader files |
| compileDebugShaders | ShaderCompile | Compile Shaders |
| generateDebugAssets | The TaskManager. CreateAnchorTasks method through taskFactory. Register (taskName) sign up for a task | Empty task, anchor point |
| mergeDebugAssets | MergeSourceSetFolders.MergeAppAssetCreationAction | Merging Assets Files |
| processDebugJavaRes | ProcessJavaResConfigAction | Process Java Res resources |
| checkDebugDuplicateClasses | CheckDuplicateClassesTask | Used to detect project external dependencies to ensure that duplicate classes are not included |
| dexBuilderDebug | DexArchiveBuilderTask | Use to convert.class files into dex Archives, or dex archive, by adding a dex file to addFile |
| mergeLibDexDebug | DexMergingTask.DexMergingAction.MERGE_LIBRARY_PROJECT | Just merge the DEX files in the library project |
| mergeDebugJavaResource | MergeJavaResourceTask | Merge Java resources from multiple Moudles |
| mergeDebugJniLibFolders | MergeSourceSetFolders.MergeJniLibFoldersCreationAction | Merge JniLibs source folders with the appropriate priority |
| validateSigningDebug | ValidateSigningTask | Used to check whether a keystore file exists in the signature configuration of the current Variant. If the current keystore defaults to debug keystore, it will be created even if it does not exist |
| mergeProjectDexDebug | DexMergingTask.DexMergingAction.MERGE_PROJECT | Just merge the project’s DEX files |
| mergeDebugNativeLibs | MergeNativeLibsTask | Merge native libraries from multiple moudles |
| stripDebugDebugSymbols | StripDebugSymbolsTask | Remove Debug symbols from Native library. |
| desugarDebugFileDependencies | DexFileDependenciesTask | Handle the dependencies of Dex files |
| mergeExtDexDebug | DexMergingTask.DexMergingAction.MERGE_EXTERNAL_LIBS | DEX files for merging external libraries only |
| packageDebug | PackageApplication | Packaging APK |
| assembleDebug | Assemble | Empty task, anchor point use |
Currently, there are three main types of tasks in Gradle Plugin, as shown below:
- 1),
Incremental Task:Inheriting from the NewIncrementalTask incremental Task base class, you need to override the doTaskAction abstract method to implement incremental functionality. - 2),
The incremental Task:Inheriting from NonIncrementalTask, the non-incremental Task base class, overwrites the doTaskAction abstract method to implement the full update function. - 3),
Transform Task:We write every custom will Transform the call appExtension. RegisterTransform (new CustomTransform ()) registration method to save it to the Extension of the current transforms in the class List, when LibraryTaskManager/TaskManager call createPostCompilationTasks (responsible for creating the compiled for a given Variant task) method, Will take out the Extension of the corresponding tranforms traverse the list, and through the TransformManager. AddTransform method converts each Transform with the matching TransformTask instance, Internal concrete and the method is through the new TransformTask. CreationAction (…). Is created in the form of.
With a thorough understanding of the basic Tasks and Tasks involved in the packaging process, let’s take a closer look at the implementation of the most important Tasks.
Important Task source code analysis
1. Resource processing related tasks
1), processDebugManifest
Corresponding implementation classes for ProcessApplicationManifest processDebugManifest Task, it inherits the IncrementalTask, but does not implement isIncremental method, So we just need to look at its doFullTaskAction method.
Call link
processDebugManifest.dofFullTaskAction => ManifestHelperKt.mergeManifestsForApplication => ManifestMerge2.merge
Copy the codeMain process analysis
The Task function is primarily used to merge all the Mainfest (including Module and Flavor) using the MergingReport, ManifestMerger2, and XmlDocument instances.
Let’s take a direct look at the Merge process of the ManifestMerger2. Merge method to see what the specific merge looks like. The main steps are as follows:
1. Get the master Manifest information to do some necessary checks, where an instance of LoadedManifestInfo is returned.
// load the main manifest file to do some checking along the way.
LoadedManifestInfo loadedMainManifestInfo =
load(
new ManifestInfo(
mManifestFile.getName(),
mManifestFile,
mDocumentType,
Optional.absent() /* mainManifestPackageName*/),
selectors,
mergingReportBuilder);
Copy the code2. Execute system attribute injection in Manifest: Replace some properties defined in the main Manifest with properties defined in Gradle, For example package, version_code, version_name, min_sdk_versin, target_sdk_version, max_sdk_version, etc.
// perform system property injection
performSystemPropertiesInjection(mergingReportBuilder,
loadedMainManifestInfo.getXmlDocument());
Copy the code/**
* Perform {@link ManifestSystemProperty} injection.
* @param mergingReport to log actions and errors.
* @param xmlDocument the xml document to inject into.
*/
protected void performSystemPropertiesInjection(
@NonNull MergingReport.Builder mergingReport,
@NonNull XmlDocument xmlDocument) {
for (ManifestSystemProperty manifestSystemProperty : ManifestSystemProperty.values()) {
String propertyOverride = mSystemPropertyResolver.getValue(manifestSystemProperty);
if(propertyOverride ! =null) { manifestSystemProperty.addTo( mergingReport.getActionRecorder(), xmlDocument, propertyOverride); }}}Copy the codeCombine flavors and build the manifest file that corresponds to them.
for (File inputFile : mFlavorsAndBuildTypeFiles) {
mLogger.verbose("Merging flavors and build manifest %s \n", inputFile.getPath());
LoadedManifestInfo overlayDocument =
load(
new ManifestInfo(
null,
inputFile,
XmlDocument.Type.OVERLAY,
mainPackageAttribute.transform(it -> it.getValue())),
selectors,
mergingReportBuilder);
if(! mFeatureName.isEmpty()) { overlayDocument = removeDynamicFeatureManifestSplitAttributeIfSpecified( overlayDocument, mergingReportBuilder); }// check the package definition
Optional<XmlAttribute> packageAttribute =
overlayDocument.getXmlDocument().getPackage();
// if both files declare a package name, it should be the same.
if(loadedMainManifestInfo.getOriginalPackageName().isPresent() && packageAttribute.isPresent() && ! loadedMainManifestInfo.getOriginalPackageName().get().equals( packageAttribute.get().getValue())) {// 2. If the package definition is repeated, the following information is output
String message = mMergeType == MergeType.APPLICATION
? String.format(
"Overlay manifest:package attribute declared at %1$s value=(%2$s)\n"
+ "\thas a different value=(%3$s) "
+ "declared in main manifest at %4$s\n"
+ "\tSuggestion: remove the overlay declaration at %5$s "
+ "\tand place it in the build.gradle:\n"
+ "\t\tflavorName {\n"
+ "\t\t\tapplicationId = \"%2$s\"\n"
+ "\t\t}",
packageAttribute.get().printPosition(),
packageAttribute.get().getValue(),
mainPackageAttribute.get().getValue(),
mainPackageAttribute.get().printPosition(),
packageAttribute.get().getSourceFile().print(true))
: String.format(
"Overlay manifest:package attribute declared at %1$s value=(%2$s)\n"
+ "\thas a different value=(%3$s) "
+ "declared in main manifest at %4$s",
packageAttribute.get().printPosition(),
packageAttribute.get().getValue(),
mainPackageAttribute.get().getValue(),
mainPackageAttribute.get().printPosition());
mergingReportBuilder.addMessage(
overlayDocument.getXmlDocument().getSourceFile(),
MergingReport.Record.Severity.ERROR,
message);
returnmergingReportBuilder.build(); }... }Copy the code4. Merge the manifest files in the library
for (LoadedManifestInfo libraryDocument : loadedLibraryDocuments) {
mLogger.verbose("Merging library manifest " + libraryDocument.getLocation());
xmlDocumentOptional = merge(
xmlDocumentOptional, libraryDocument, mergingReportBuilder);
if(! xmlDocumentOptional.isPresent()) {returnmergingReportBuilder.build(); }}Copy the code5. Implement placeholder replacement in the manifest file
performPlaceHolderSubstitution(
loadedMainManifestInfo,
xmlDocumentOptional.get(),
mergingReportBuilder,
severity);
Copy the code6. Then replace some attributes in the final merged manifest, similar to Step 2.
7, save the manifest to build/intermediates/merged_manifests flavorName/AndroidManifest. XML, so far, has been to produce the final manifest file.
2), mergeDebugResources
MergeDebugResources corresponds to the MergeResources Task, which uses the AAPT2 combined resources.
Call link
MergeResources.doFullTaskAction => ResourceMerger.mergeData => MergedResourceWriter.end => mResourceCompiler.submitCompile => AaptV2CommandBuilder.makeCompileCommand
Copy the codePrincipal process analysis
MergeResources inherits from IncrementalTask, and for incremental Tasks we only need to look at the implementation of three methods:
isIncrementaldoFullTaskActiondoIncrementalTaskAction
1. First look at the isIncremental method.
The MergeResources Task supports increments and must have overridden the doIncrementalTaskAction method
protected boolean isIncremental() {
return true;
}
Copy the code2, and then to check doFullTaskAction method, internal obtained through getConfiguredResourceSets resourceSets, Build /generated/ RES/RS
List<ResourceSet> resourceSets = getConfiguredResourceSets(preprocessor);
Copy the code3. Create a ResourceMerger and fill it with resourceSets.
ResourceMerger merger = new ResourceMerger(minSdk.get());
Copy the code4, create ResourceCompilationService, it USES the aapt2.
/ / used makeAapt aapt2, then return ResourceCompilationService instance.
ResourceCompilationService resourceCompiler =
getResourceProcessor(
getAapt2FromMaven(),
workerExecutorFacade,
errorFormatMode,
flags,
processResources,
getLogger())) {
Copy the code5. Add resourceSet obtained in Step 2 to ResourceMerger.
for (ResourceSet resourceSet : resourceSets) {
resourceSet.loadFromFiles(new LoggerWrapper(getLogger()));
merger.addDataSet(resourceSet);
}
Copy the code6. Create MergedResourceWriter
MergedResourceWriter writer =
new MergedResourceWriter(
workerExecutorFacade,
destinationDir,
publicFile,
mergingLog,
preprocessor,
resourceCompiler,
getIncrementalFolder(),
dataBindingLayoutProcessor,
mergedNotCompiledResourcesOutputDirectory,
pseudoLocalesEnabled,
getCrunchPng());
Copy the code7. Call the resourcEmergen. mergeData method to merge the resources.
merger.mergeData(writer, false /*doCleanUp*/);
Copy the codeCall MergedResourceWriter’s start, ignoreItemInMerge, removeItem, addItem, and end methods, where item contains the resources to be processed, including XML and image resources. For each item file, a corresponding CompileResourceRequest instance is created and added to the mCompileResourceRequests ConcurrentLinkedQueue.
9, call mResourceCompiler submitCompile method processing resources.
// MergedResourceWriter.end()
mResourceCompiler.submitCompile(
new CompileResourceRequest(
fileToCompile,
request.getOutputDirectory(),
request.getInputDirectoryName(),
request.getInputFileIsFromDependency(),
pseudoLocalesEnabled,
crunchPng,
ImmutableMap.of(),
request.getInputFile()));
mCompiledFileMap.put(
fileToCompile.getAbsolutePath(),
mResourceCompiler.compileOutputFor(request).getAbsolutePath());
Copy the codeWill eventually be used in submitCompile AaptV2CommandBuilder. MakeCompileCommand method generates aapt2 command to deal with resources.
10. Finally, I won’t go over the doIncrementalTaskAction implementation here, because the incremental task implementation process is not very different from the full implementation, just using the modified file to get resourceSets.
2. The process of packaging a Class file into a Dex file
DexArchiveBuilderTask, the counterpart of DexArchiveBuilderTask, is used to convert.class files into dex Archives, which can add a dex file via addFile.
Call link
DexArchiveBuilderTask.doTaskAction => DexArchiveBuilderTaskDelegate.doProcess => DexArchiveBuilderTaskDelegate.processClassFromInput => DexArchiveBuilderTaskDelegate.convertToDexArchive -> DexArchiveBuilderTaskDelegate.launchProcessing -> DexArchiveBuilder.convert
Copy the codePrincipal process analysis
In DexArchiveBuilderTask, there are two ways to process classes: one is to process classes in the directory, and the other is to process classes in.jar.
So, why are these two ways?
Because the class files in.jar are generally dependent libraries that do not change, Gradle can implement a caching operation here.
ConvertJarToDexArchive processing JAR
When working with jars, Gradle creates a separate DEX file for each class file in the JAR and puts them back into the jar.
private fun convertJarToDexArchive(
jarInput: File,
outputDir: File,
bootclasspath: ClasspathServiceKey,
classpath: ClasspathServiceKey,
cacheInfo: D8DesugaringCacheInfo
): List<File> {
if(cacheInfo ! == DesugaringDontCache) { val cachedVersion = cacheHandler.getCachedVersionIfPresent( jarInput, cacheInfo.orderedD8DesugaringDependencies )if(cachedVersion ! =null) {
// If there is a cache, use the cached JAR package directly.
val outputFile = getOutputForJar(jarInput, outputDir, null)
Files.copy(
cachedVersion.toPath(),
outputFile.toPath(),
StandardCopyOption.REPLACE_EXISTING
)
// no need to try to cache an already cached version.
return listOf()
}
}
// If there is no cache, call the convertToDexArchive method to generate the dex.
return convertToDexArchive(
jarInput,
outputDir,
false,
bootclasspath,
classpath,
setOf(),
setOf()
)
}
Copy the code2. Use convertToDexArchive to process dir and subsequent processing of jar
An internal call to launchProcessing is made to handle dir, as shown below:
private fun launchProcessing(
dexConversionParameters: DexArchiveBuilderTaskDelegate.DexConversionParameters,
outStream: OutputStream,
errStream: OutputStream,
receiver: MessageReceiver
) {
val dexArchiveBuilder = dexConversionParameters.getDexArchiveBuilder(
outStream,
errStream,
receiver
)
val inputPath = dexConversionParameters.input.toPath()
val hasIncrementalInfo =
dexConversionParameters.input.isDirectory && dexConversionParameters.isIncremental
/ / if a class of new | | modified, for processing
fun toProcess(path: String): Boolean {
if(! dexConversionParameters.belongsToThisBucket(path))return false
if(! hasIncrementalInfo) {return true
}
val resolved = inputPath.resolve(path).toFile()
return resolved in dexConversionParameters.additionalPaths || resolved in dexConversionParameters.changedFiles
}
val bucketFilter = { name: String -> toProcess(name) }
loggerWrapper.verbose("Dexing '" + inputPath + "' to '" + dexConversionParameters.output + "'")
try {
ClassFileInputs.fromPath(inputPath).use { input ->
input.entries(bucketFilter).use { entries ->
/ / internal calls the dx | | d8 to generate the dex file
dexArchiveBuilder.convert(
entries,
Paths.get(URI(dexConversionParameters.output)),
dexConversionParameters.input.isDirectory
)
}
}
} catch (ex: DexArchiveBuilderException) {
throw DexArchiveBuilderException("Failed to process $inputPath", ex)
}
}
Copy the codeAs you can see, in DexArchiveBuilder there are two subclasses as follows:
D8DexArchiveBuilder:Call D8 to generate the DEX file.DxDexArchiveBuilder:Call DX to generate the DEX file.
Here we take D8DexArchiveBuilder as an example to illustrate how it calls D8 to generate DEX files. The source code is as follows:
@Override
public void convert(
@NonNull Stream<ClassFileEntry> input, @NonNull Path output, boolean isIncremental)
throws DexArchiveBuilderException {
// 1. Create an instance of D8 diagnostic information processor, which is used to issue diagnostic information of different levels. There are three types of diagnostic information in descending severity: Error, warning, info.
D8DiagnosticsHandler d8DiagnosticsHandler = new InterceptingDiagnosticsHandler();
try {
// create a D8 command builder instance.
D8Command.Builder builder = D8Command.builder(d8DiagnosticsHandler);
AtomicInteger entryCount = new AtomicInteger();
// 3, read the byte data of each class.
input.forEach(
entry -> {
builder.addClassProgramData(
readAllBytes(entry), D8DiagnosticsHandler.getOrigin(entry));
entryCount.incrementAndGet();
});
if (entryCount.get() == 0) {
// 3. If there is no data to traverse, return. Here AtomicInteger class is used to realize whether there is traversal of the data distinction processing.
return;
}
OutputMode outputMode =
isIncremental ? OutputMode.DexFilePerClassFile : OutputMode.DexIndexed;
// Set a number of configurations for the D8 command builder instance, such as build mode, minimum Sdk version, etc.
builder.setMode(compilationMode)
.setMinApiLevel(minSdkVersion)
.setIntermediate(true)
.setOutput(output, outputMode)
.setIncludeClassesChecksum(compilationMode == compilationMode.DEBUG);
if (desugaring) {
builder.addLibraryResourceProvider(bootClasspath.getOrderedProvider());
builder.addClasspathResourceProvider(classpath.getOrderedProvider());
if(libConfiguration ! =null) { builder.addSpecialLibraryConfiguration(libConfiguration); }}else {
builder.setDisableDesugaring(true);
}
// 5. Run the assembled D8 command using the D8 class run method in com.android.tools.r8 tool package.
D8.run(builder.build(), MoreExecutors.newDirectExecutorService());
} catch (Throwable e) {
throwgetExceptionToRethrow(e, d8DiagnosticsHandler); }}Copy the codeThe D8DexArchiveBuilder convert process can be summarized in five steps as follows:
- 1) Create an instance of D8 diagnostic information processor, which is used to issue diagnostic information of different levels. It can be divided into three categories, which are error, Warning and INFO in descending severity.
- 2) Create a D8 command builder instance.
- 3) Read the byte data of each class iteratively.
- 4) Set up a series of configurations for the D8 command builder instance, such as build mode, minimum Sdk version, etc.
- 5) Run the assembled D8 command using the D8 class run method in com.android.tools.r8 toolkit.
Eight, summary
Let’s go back to the initial Gradle plugin architecture diagram, as shown below:
The last
We can think about it from the bottom up and remember, what is the main flow of each layer? What are some of the key details involved? At this point, do you feel like you really understand how the Gradle plug-in architecture works?
Reference links:
Android Gradle Plugin V3.6.2 source code
Gradle V5.6.4 source code
3, Android Plugin DSL Reference
Gradle DSL Reference
5, designing – gradle – plugins
Android-training => gradle
One of 7, serial | understanding Gradle framework: the Plugin, the Extension, buildSrc
8, serial | understanding gradle frame # 2: dependence analysis
9, serial | understanding gradle frame # 3: artifacts
Android Gradle Plugin Plugin
Android Gradle Plugin Plugin
12. Gradle Cook his computer (analysis of source code of construction)
13. Gradle Cook up his computer (analysis of the source code of core entrusted object of construction life cycle)