“This is the first day of my participation in the First Challenge 2022. For details: First Challenge 2022.”

Currently, Android is building APK, the most commonly used is Gradle packaging. To understand the process of Android Apk packaging, it is necessary to understand the entire building process of Gradle Plugin. After understanding this, we can develop Gradle Plugin with ease.

Let’s take a look at the internal structure of APK:

Android APK package structure

Let’s look at the structure of a normal APK.

Build –outputs– APK — DEBUG: build–outputs– APK –debug: build–outputs– APK –debug: build–outputs– APK –debug

class.dex

Java code is converted to class files by javac and dex files by dx.

res

The processed binary resource file is saved.

esoources.arsc

Saves a mapping of the resource ID name and the value/path corresponding to the resource.

META-INF

Used to verify APK signatures, including three important files manifest. MT, cert. SF, cert.rsa.

  • Manifest.mf keeps a summary of all files
  • Cert.sf keeps a summary of each message in the manifest.mf
  • Cert. RSA contains the signature of the cert. SF file and the certificate used for the signature

AndroidManifest.xml

Global configuration file, here is the compiled binary file.

Ii. AppPlugin building process

Before analyzing, let’s compile the project, directly to the compile button in the upper left. You can see the following output:

:android-gradle-plugin-source:preBuild UP-TO-DATE
:android-gradle-plugin-source:preDebugBuild
:android-gradle-plugin-source:compileDebugAidl
:android-gradle-plugin-source:compileDebugRenderscript
:android-gradle-plugin-source:checkDebugManifest
:android-gradle-plugin-source:generateDebugBuildConfig
:android-gradle-plugin-source:prepareLintJar UP-TO-DATE
:android-gradle-plugin-source:generateDebugResValues
:android-gradle-plugin-source:generateDebugResources
:android-gradle-plugin-source:mergeDebugResources
:android-gradle-plugin-source:createDebugCompatibleScreenManifests
:android-gradle-plugin-source:processDebugManifest
:android-gradle-plugin-source:splitsDiscoveryTaskDebug
:android-gradle-plugin-source:processDebugResources
:android-gradle-plugin-source:generateDebugSources
.......
Copy the code

Gradle builds with Task by Task. Before analyzing the Task source code, let’s follow the process of building the Gradle Plugin.

To view the source code of the Android Gradle Plugin, we need to add the Android Grade Plugin dependency to the project as follows:

compileOnly 'com. Android. Tools. Build: gradle: 3.4.0'
Copy the code

Now you’re ready to look at the gradle source code, but where do you start?

To build a module as an Android project, you configure build.gradle in the root directory to import the following plug-ins:

apply plugin: 'com.android.application'
Copy the code

We talked about custom plug-ins again. You define an xxx.properties file that declares the entry class for the plug-in. Here, we will know the android gradle plugin entry class, as long as find the android. Application. The properties file is ok, then you can see the internal indicated the plug-in implementation class:

implementation-class=com.android.build.gradle.internal.plugins.AppPlugin
Copy the code

The entry is defined as AppPlugin, which inherits from BasePlugin:

public class AppPlugin extends AbstractAppPlugin {...// Apply the specified plugin
   @Override
    protected void pluginSpecificApply(@NonNull Project project) {}...// Get an extension class that provides a corresponding Android Extension
  @Override
    @NonNull
    protected Class<? extends AppExtension> getExtensionClass() {
        returnBaseAppModuleExtension.class; }}Copy the code

AppPlugin doesn’t do much, and most of the work is done in BasePlugin. The main apply method is in BasePlugin, where some preparatory work is done, as follows:

public final void apply(@NonNull Project project) {
        CrashReporting.runAction(
                () -> {
                    basePluginApply(project);
                    pluginSpecificApply(project);
                });
    }
Copy the code

The basePluginApply method is called in the Apply method as follows:

## BasePlugin.java
private void basePluginApply(@NonNull Project project) {
        // We run by default in headless mode, so the JVM doesn't steal focus.
        System.setProperty("java.awt.headless"."true");

        this.project = project;
        this.projectOptions = new ProjectOptions(project);
  	// Check gradle version numbers
        checkGradleVersion(project, getLogger(), projectOptions);
        DependencyResolutionChecks.registerDependencyCheck(project, projectOptions);

        project.getPluginManager().apply(AndroidBasePlugin.class);
	// Check the path
        checkPathForErrors();
  	// Check the module name
        checkModulesForErrors();

        PluginInitializer.initialize(project);
        ProfilerInitializer.init(project, projectOptions);
  	// Record method timethreadRecorder = ThreadRecorder.get(); .if(! projectOptions.get(BooleanOption.ENABLE_NEW_DSL_AND_API)) {// Configuration project
            threadRecorder.record(
                    ExecutionType.BASE_PLUGIN_PROJECT_CONFIGURE,
                    project.getPath(),
                    null.this::configureProject);
           / / configure the Extension
            threadRecorder.record(
                    ExecutionType.BASE_PLUGIN_PROJECT_BASE_EXTENSION_CREATION,
                    project.getPath(),
                    null.this::configureExtension);
           / / create the Tasks
            threadRecorder.record(
                    ExecutionType.BASE_PLUGIN_PROJECT_TASKS_CREATION,
                    project.getPath(),
                    null.this::createTasks);
        } else{... }}Copy the code

Threadrecoirder.recode () records the path of the last parameter and the point in time when it was executed, and does some plug-in checking and configuration initialization.

// Configure the project to set the build callback
this::configureProject
/ / configure the Extension
this::configureExtension
// Create a task
this::createTasks
Copy the code

2.1 configurePoroject Configuration items

This stage mainly did:

# #BasePlugin 
private void configureProject(a) {
        finalGradle gradle = project.getGradle(); ./ / AndroidSDK processing class
        sdkHandler = new SdkHandler(project, getLogger());
        if(! gradle.getStartParameter().isOffline() && projectOptions.get(BooleanOption.ENABLE_SDK_DOWNLOAD)) {// The configuration depends on the download processing
            SdkLibData sdkLibData = SdkLibData.download(getDownloader(), getSettingsController());
            sdkHandler.setSdkLibData(sdkLibData);
        }

   	/ / create AndroidBuilder
        AndroidBuilder androidBuilder =
                new AndroidBuilder(
                        project == project.getRootProject() ? project.getName() : project.getPath(),
                        creator,
                        new GradleProcessExecutor(project),
                        new GradleJavaProcessExecutor(project),
                        extraModelInfo.getSyncIssueHandler(),
                        extraModelInfo.getMessageReceiver(),
                        getLogger());
   	// Create the DataBindingBuilder instance.
        dataBindingBuilder = new DataBindingBuilder();
        dataBindingBuilder.setPrintMachineReadableOutput(
                SyncOptions.getErrorFormatMode(projectOptions) == ErrorFormatMode.MACHINE_PARSABLE);

        if(projectOptions.hasRemovedOptions()) { androidBuilder .getIssueReporter() .reportWarning(Type.GENERIC, projectOptions.getRemovedOptionsErrorMessage()); }...// Force the use of no less than the currently supported minimum plug-in version, otherwise an exception will be thrown.
        GradlePluginUtils.enforceMinimumVersionsOfPlugins(
                project, androidBuilder.getIssueReporter());

        // Apply the Java Plugin
        project.getPlugins().apply(JavaBasePlugin.class);

        DslScopeImpl dslScope =
                new DslScopeImpl(
                        extraModelInfo.getSyncIssueHandler(),
                        extraModelInfo.getDeprecationReporter(),
                        objectFactory);

        @NullableFileCache buildCache = BuildCacheUtils.createBuildCacheIfEnabled(project, projectOptions); .// Add a description to the Assemble task
     project.getTasks()
                .getByName("assemble")
                .setDescription(
                        "Assembles all variants of all applications and secondary packages.");
  
     // project clears the cache after execution
    gradle.addBuildListener(
                new BuildListener() {
                		.......
                    @Override
                    public void buildFinished(@NonNull BuildResult buildResult) {
                       
                        if(buildResult.getGradle().getParent() ! =null) {
                            return;
                        }
                        ModelBuilder.clearCaches();
                        sdkHandler.unload();
                        threadRecorder.record(
                                ExecutionType.BASE_PLUGIN_BUILD_FINISHED,
                                project.getPath(),
                                null,
                                () -> {
                                    WorkerActionServiceRegistry.INSTANCE
                                            .shutdownAllRegisteredServices(
                                                    ForkJoinPool.commonPool());
                                    Main.clearInternTables();
                                });
                      // Clear the dex cache when the task is completedDeprecationReporterImpl.Companion.clean(); }}); . }Copy the code

In this method, sdkHandle is created and the android.jar package is used when the code is compiled. An AndroidBuilder object is also created, which is used to merge manifest and create. Finally, gradle lifecycle listening is added and the dex cache is cleared after project execution.

2.2 configureExtension configureExtension

This is the phase of configuring extension, creating configurable objects in our Android block.

  1. To create theBuildType,ProductFlavor,SignIngConfigThree types of Containers, and then it’s passed increateExtensionMethods. Here,AppExtensionThat isbuild.gradleIn theAndroid {} DSL closuresAnd look atcreateExtensionMethods:
## AbstractAppPlugin.java 
protected BaseExtension createExtension(
            @NonNull Project project,
            @NonNull ProjectOptions projectOptions,
            @NonNull GlobalScope globalScope,
            @NonNull SdkHandler sdkHandler,
            @NonNull NamedDomainObjectContainer<BuildType> buildTypeContainer,
            @NonNull NamedDomainObjectContainer<ProductFlavor> productFlavorContainer,
            @NonNull NamedDomainObjectContainer<SigningConfig> signingConfigContainer,
            @NonNull NamedDomainObjectContainer<BaseVariantOutput> buildOutputs,
            @NonNull SourceSetManager sourceSetManager,
            @NonNull ExtraModelInfo extraModelInfo) {
        return project.getExtensions()
                .create(
                        "android",
                        getExtensionClass(),
                        project,
                        projectOptions,
                        globalScope,
                        sdkHandler,
                        buildTypeContainer,
                        productFlavorContainer,
                        signingConfigContainer,
                        buildOutputs,
                        sourceSetManager,
                        extraModelInfo,
                        isBaseApplication);
    }
Copy the code

We can also see that android configuration blocks are built from Gradle.

  1. Some management classes are created in turnvariantFactory,taskManagerandvariantManager. Where TaskManager is a management class that creates specific tasks,VariantFactoryIs a factory class for building variants, mainly generating objects for building variants.
  2. Configure thesigningConfigContainer,buildTypeContainerandproductFlavorContainerFor each configured callbackvariantManagerFor management.
## BasePlugin.java
// Map whenObjectAdded Callbacks to the singingConfig container.
signingConfigContainer.whenObjectAdded(variantManager::addSigningConfig);

        buildTypeContainer.whenObjectAdded(
                buildType -> {
                    if (!this.getClass().isAssignableFrom(DynamicFeaturePlugin.class)) {
                        SigningConfig signingConfig =
                                signingConfigContainer.findByName(BuilderConstants.DEBUG);
                        buildType.init(signingConfig);
                    } else {
                        // initialize it without the signingConfig for dynamic-features.
                        buildType.init();
                    }
                    variantManager.addBuildType(buildType);
                });
				// Map whenObjectAdded Callbacks to the productFlavor container.
        productFlavorContainer.whenObjectAdded(variantManager::addProductFlavor);
Copy the code
  1. Create the default Debug signature, and create both debug and Release buildTypes.
variantFactory.createDefaultComponents(
                buildTypeContainer, productFlavorContainer, signingConfigContainer);
Copy the code

The real implementation is in ApplicationVariantFactory:

 public void createDefaultComponents(
            @NonNull NamedDomainObjectContainer<BuildType> buildTypes,
            @NonNull NamedDomainObjectContainer<ProductFlavor> productFlavors,
            @NonNull NamedDomainObjectContainer<SigningConfig> signingConfigs) {
        // The signature configuration must be created first so that the build type "debug" can be initialized using the debug signature configuration.
        signingConfigs.create(DEBUG);
        buildTypes.create(DEBUG);
        buildTypes.create(RELEASE);
    }
Copy the code

2.3 createTasks Creates a task

In the createTask method of BasePlugin, there are two main steps to start building the required Task: creating a Task that does not depend on flavor and creating a Task that depends on configuration items.

## BasePlugin.java
private void createTasks(a) {
  	// Create Tasks before EVALUATE
        threadRecorder.record(
                ExecutionType.TASK_MANAGER_CREATE_TASKS,
                project.getPath(),
                null,
                () -> taskManager.createTasksBeforeEvaluate());

  	// Create Android Tasks
        project.afterEvaluate(
                CrashReporting.afterEvaluate(
                        p -> {
                            sourceSetManager.runBuildableArtifactsActions();

                            threadRecorder.record(
                                    ExecutionType.BASE_PLUGIN_CREATE_ANDROID_TASKS,
                                    project.getPath(),
                                    null.this::createAndroidTasks);
                        }));
    }
Copy the code
  1. createTasksBeforeEvaluate()

This method registers a series of tasks for the container, including uninstallAll, deviceCheck, connectedCheck, preBuild, extractProguardFiles, sourceSets, AssembleAndroidTest compileLint, lint, lintChecks cleanBuildCacheresolveConfigAttr, consumeConfigAttr.

  1. createAndroidTasks()

In this method, flavors related data is generated, and Task instances corresponding to flavors are created and registered in the Task.

## BaePlugin.java
@VisibleForTesting
    final void createAndroidTasks(a) {.../ / Project Path, CompileSdk, BuildToolsVersion, Splits, KotlinPluginVersion, FirebasePerformancePluginVersion written Project configuration
         ProcessProfileWriter.getProject(project.getPath())
                .setCompileSdk(extension.getCompileSdkVersion())
                .setBuildToolsVersion(extension.getBuildToolsRevision().toString())
                .setSplits(AnalyticsUtil.toProto(extension.getSplits()));

        String kotlinPluginVersion = getKotlinPluginVersion();
        if(kotlinPluginVersion ! =null) {
            ProcessProfileWriter.getProject(project.getPath())
                    .setKotlinPluginVersion(kotlinPluginVersion);
        }

        if (projectOptions.get(BooleanOption.INJECT_SDK_MAVEN_REPOS)) {
            sdkHandler.addLocalRepositories(project);
        }

      	// Create a task for the applicationList<VariantScope> variantScopes = variantManager.createAndroidTasks(); . }Copy the code

Let’s look at the createAndroidTasks method of variantManager:

##   VariantManager.java
public List<VariantScope> createAndroidTasks(a) {
        variantFactory.validateModel(this);
        variantFactory.preVariantWork(project);

        if (variantScopes.isEmpty()) {
          // Build flavor variantspopulateVariantDataList(); } taskManager.createTopLevelTestTasks(! productFlavors.isEmpty());for (final VariantScope variantScope : variantScopes) {
          // Create a task for variant data
            createTasksForVariantData(variantScope);
        }

        taskManager.createSourceSetArtifactReportTask(globalScope);

        taskManager.createReportTasks(variantScopes);

        return variantScopes;
    }
Copy the code

If the scopes are empty, the populateVariantDataList method will be called. In this method, the corresponding combination will be created based on the flavor and dimension and stored in flavorComboList. The last call createVariantDataForProductFlavors method.

## VariantManager.java
public void populateVariantDataList(a) {
  //List<String> flavorDimensionList = extension.getFlavorDimensionList(); .// Get the productFlavor array iteratively
    Iterable<CoreProductFlavor> flavorDsl =
                    Iterables.transform(
                            productFlavors.values(),
                            ProductFlavorData::getProductFlavor);
  
  // Create a flavor and dimension combination
    List<ProductFlavorCombo<CoreProductFlavor>> flavorComboList =
                    ProductFlavorCombo.createCombinations(
                            flavorDimensionList,
                            flavorDsl);

  // Create VariantData for each combination
            for (ProductFlavorCombo<CoreProductFlavor>  flavorCombo : flavorComboList) {
                //noinspection uncheckedcreateVariantDataForProductFlavors( (List<ProductFlavor>) (List) flavorCombo.getFlavorList()); }... }Copy the code

Look at next createVariantDataForProductFlavors, finally is the method called createVariantDataForProductFlavorsAndVariantType, In this method, the resulting VariantData is created and added to the variantScopes collection, where all the build variants are added to the scopes collection.

## VariantManager.java
private void createVariantDataForProductFlavorsAndVariantType(
            @NonNull List<ProductFlavor> productFlavorList, @NonNull VariantType variantType) {... BaseVariantData variantData = createVariantDataForVariantType( buildTypeData.getBuildType(), productFlavorList, variantType); addVariant(variantData); . }public void addVariant(BaseVariantData variantData) {
        variantScopes.add(variantData.getScope());
    }
Copy the code

Then look at createTasksForVariantData methods, create the variant data, will give each variantData create the corresponding task.

# #VariantManager
public void createTasksForVariantData(final VariantScope variantScope) {
        final BaseVariantData variantData = variantScope.getVariantData();
        final VariantType variantType = variantData.getType();
        final GradleVariantConfiguration variantConfig = variantScope.getVariantConfiguration();

  	/ / create assembleXXXTask
        taskManager.createAssembleTask(variantData);
        if (variantType.isBaseModule()) {
          // If the variantType is a Base moudle, the corresponding bundle Task is created
            taskManager.createBundleTask(variantData);
        }
   if (variantType.isTestComponent()) {
     		......
   } else {
     // Create a series of tasks based on variantDatataskManager.createTasksForVariantScope(variantScope); }}Copy the code

The AssembleTask method is created first. Look at the createAssembleTask method:

# #TaskManager  
public void createAssembleTask(@NonNull final BaseVariantData variantData) {
        final VariantScope scope = variantData.getScope();
  	// It is eventually registered in TaskContainer
        taskFactory.register(
          	// Get assemble XXX
                getAssembleTaskName(scope, "assemble"),
                null /*preConfigAction*/,
                task -> {
                    task.setDescription(
                            "Assembles main output for variant "
                                    + scope.getVariantConfiguration().getFullName());
                },
                taskProvider -> scope.getTaskContainer().setAssembleTask(taskProvider));
    }
Copy the code

And then back to createTasksForVariantData method, see createTasksForVariantScope method, it is an abstract method, Will perform to the ApplicationTaskManager createTasksForVariantScope method.

@Override
public void createTasksForVariantScope(@NonNull final VariantScope variantScope) {
        createAnchorTasks(variantScope);
        createCheckManifestTask(variantScope);   / / testing the manifest
        handleMicroApp(variantScope);
        createDependencyStreams(variantScope);
        createApplicationIdWriterTask(variantScope);    // application id 
        taskFactory.register(new MainApkListPersistence.CreationAction(variantScope));
        createBuildArtifactReportTask(variantScope);
        createMergeApkManifestsTask(variantScope);   / / merge the manifest
        createGenerateResValuesTask(variantScope);
        createRenderscriptTask(variantScope);
        createMergeResourcesTask(variantScope, true, ImmutableSet.of());   // Merge resource files
        createShaderTask(variantScope);
        createMergeAssetsTask(variantScope);
        createBuildConfigTask(variantScope);
        createApkProcessResTask(variantScope);    // Process resources
        createProcessJavaResTask(variantScope);
        createAidlTask(variantScope);      / / processing aidl
        createMergeJniLibFoldersTasks(variantScope);    / / merge jni
        createDataBindingTasksIfNecessary(variantScope, MergeType.MERGE);    / / processing databinding
        TaskProvider<? extends JavaCompile> javacTask = createJavacTask(variantScope);
        addJavacClassesStream(variantScope);
        setJavaCompilerTask(javacTask, variantScope);
        createPostCompilationTasks(variantScope);   // Handle Android Transform
        createValidateSigningTask(variantScope);
        taskFactory.register(new SigningConfigWriterTask.CreationAction(variantScope));
        createPackagingTask(variantScope, null /* buildInfoGeneratorTask */);   / / packaging apkcreateConnectedTestForVariant(variantScope); . }Copy the code

In this method, you can see that a list of Tasks are created suitable for application build, and that the gradle Plugin build process is completed.

Let’s examine these Tasks through assembleDebug’s packaging process.

3. App packaging process

3.1 Analysis of packaging process

Before we begin, let’s review the packaging process of APK. We can take a look at the figure on the Android website:

The diagram provided on the official website is relatively brief and can only be seen as a general logic. Let’s look at the details of the packaging process in more detail.

It can be summarized as the following seven steps:

  1. throughaaptPackage the RES resource file and generateR.java,resources.arscandresfile
  2. To deal with.aidlFile, generate the correspondingJavaInterface file
  3. throughJava CompiercompileR.java, Java interface file, Java source file, generate. Class file
  4. throughdexCommand,.classFiles and third-party libraries.classFile processing generationclasses.dex
  5. throughapkbuilderTools toaaptAndroid Gradle plugin 3.0.0 uses AAPT2 instead of AAPTresources.arscandresFiles,assetsFiles andclasses.dexPackage together to generate APK
  6. Debug and release the above APK with the jarsigner signature tool
  7. throughzipalignTool will be signed afterapkAlign

Now that we have an overview of the packaging process, what does APK packaging look like in Task latitude?

3.2 Task Latitude Analysis packaging process

You can use the following command to package a Debug Apk package and see which tasks are involved in the package.

./gradlew assembleDebug --console=plain
Copy the code

You can see the following output:

> Task :app:preBuild UP-TO-DATE > Task :app:preDebugBuild UP-TO-DATE > Task :app:compileDebugAidl NO-SOURCE > Task :app:checkDebugManifest UP-TO-DATE > Task :app:generateDebugBuildConfig UP-TO-DATE > Task :app:compileDebugRenderscript NO-SOURCE > Task :app:mainApkListPersistenceDebug UP-TO-DATE > Task :app:generateDebugResValues UP-TO-DATE > Task :app:generateDebugResources UP-TO-DATE > Task :app:createDebugCompatibleScreenManifests UP-TO-DATE > Task :app:processDebugManifest UP-TO-DATE > Task :app:mergeDebugResources UP-TO-DATE > Task :app:processDebugResources UP-TO-DATE > Task :app:compileDebugKotlin UP-TO-DATE > Task :app:prepareLintJar UP-TO-DATE > Task :app:generateDebugSources UP-TO-DATE > Task :app:javaPreCompileDebug UP-TO-DATE > Task :app:compileDebugJavaWithJavac UP-TO-DATE > Task :app:compileDebugSources UP-TO-DATE > Task :app:mergeDebugShaders UP-TO-DATE > Task :app:compileDebugShaders UP-TO-DATE > Task :app:generateDebugAssets UP-TO-DATE > Task :app:mergeDebugAssets UP-TO-DATE >  Task :app:validateSigningDebug UP-TO-DATE > Task :app:signingConfigWriterDebug UP-TO-DATE > Task :app:checkDebugDuplicateClasses UP-TO-DATE > Task :app:transformClassesWithDexBuilderForDebug UP-TO-DATE > Task :app:transformDexArchiveWithExternalLibsDexMergerForDebug UP-TO-DATE > Task :app:transformDexArchiveWithDexMergerForDebug UP-TO-DATE > Task :app:mergeDebugJniLibFolders UP-TO-DATE > Task :app:processDebugJavaRes NO-SOURCE > Task :app:transformResourcesWithMergeJavaResForDebug UP-TO-DATE > Task :app:transformNativeLibsWithMergeJniLibsForDebug UP-TO-DATE > Task :app:packageDebug UP-TO-DATE > Task :app:assembleDebug UP-TO-DATECopy the code

As we know from the previous App Pugin build process analysis, the implementation of tasks can be found in TaskManager, and there are two main methods to create tasks. Are the Taskmanager. CreateTasksBeforeEvaluate and ApplicationTaskManager. CreateTasksForVariantScope ().

AssemebleDebug package process assemebleDebug package process assemebleDebug package process assemebleDebug package process

Task Corresponding implementation class role
preBuild Empty task, only used as an anchor point
preDebugBuild An empty task is only used as an anchor point for the variant
compileDebugAidl AidlCompile Processing aidl
compileDebugRenderscript RenderscriptCompile Handling renderscript
checkDebugManifest CheckManifest Check whether manifest exists
generateDebugBuildConfig GenerateBuildConfig Generate BuildConfig Java
prepareLintJar PrepareLintJar Copy the Lint JAR package to the specified location
generateDebugResValues GenerateResValues Generate resvalues, generated. XML
generateDebugResources Empty task, anchor point
mergeDebugResources MergeResources Merging resource files
createDebugCompatibleScreenManifests CompatibleScreensManifest Compatible-screens is generated in the manifest file, specifying screen adaptation
processDebugManifest ProcessApplicationManifest Merging manifest files
splitsDiscoveryTaskDebug SplitsDiscovery Generate split-list.json for APK subcontracting
processDebugResources LinkApplicationAndroidResourcesTask Aapt packages resources
generateDebugSources Empty task, anchor point
javaPreCompileDebug JavaPreCompileTask Generate annotationProcessors. Json file
compileDebugJavaWithJavac AndroidJavaCompile Compiling Java files
compileDebugNdk NdkCompile Compile the NDK
compileDebugSources Empty task, anchor point use
mergeDebugShaders MergeSourceSetFolders Merge shader files
compileDebugShaders ShaderCompile Compile shaders
generateDebugAssets Empty task, anchor point
mergeDebugAssets MergeSourceSetFolders Merging Assets Files
transformClassesWithDexBuilderForDebug DexArchiveBuilderTransform The class packaging dex
transformDexArchiveWithExternalLibsDexMergerForDebug ExternalLibsMergerTransform Package the dex of the three-party library, so that the dex increment does not need to merge, saving time
transformDexArchiveWithDexMergerForDebug DexMergerTransform Pack the final dex
mergeDebugJniLibFolders MergeSouceSetFolders Merge jni lib files
transformNativeLibsWithMergeJniLibsForDebug MergeJavaResourcesTransform Merge jnilibs
transformNativeLibsWithStripDebugSymbolForDebug StripDebugSymbolTransform Remove debug symbols from native lib
processDebugJavaRes ProcessJavaResConfigAction Java res
transformResourcesWithMergeJavaResForDebug MergeJavaResourcesTransform Merge the Java res
validateSigningDebug ValidateSigningTask Verify the signature
packageDebug PackageApplication Packaging apk
assembleDebug Empty task, anchor point

There are three types of tasks in gradle Plugin: non-incremental tasks, incremental tasks, and transform tasks.

  1. The incremental task

    Look at the @taskAction annotation

  2. Incremental task

    First look at whether the isIncremental method supports increments, and then look at the doFullTaskAction method. If you support increments, look at the doIncrementalTaskAction method.

  3. transform task

    Look directly at the implementation of the transform method

We’ll take a closer look at the implementation of several of these important tasks.

Gradle: When it is difficult to find the implementation class for a task, you can add it to build.gradle and print it in the command window.

gradle.taskGraph.whenReady {
    it.allTasks.each{ task ->
        println "${task.name} : ${task.class.name - "_Decorated"}"}}Copy the code

3.3 Implementation analysis of key tasks

3.3.1 compileDebugAidl(compile.aidl files)

The implementation class for compileDebugAidl is’ AidlCompile, which converts. Aidl files into Java interface files that the compiler can process using the AIDL tool.

Call link: AidlCompile.doFullTaskAction -> workers.submit(AidlCompileRunnable.class, new AidlCompileParams(dir, processor)) -> DirectoryWalker.walk() -> action.call() -> AidlProcessor.call()

Since it is an incremental Task, let’s go straight to the doFullTaskAction method:

@TaskAction
    public void doFullTaskAction(a) throws IOException {... AidlProcessor processor =new AidlProcessor(
                            aidl,
                            target.getPath(IAndroidTarget.ANDROID_AIDL),
                            fullImportList,
                            sourceOutputDir,
                            packagedDir,
                            packageWhitelist,
                            new DepFileProcessor(),
                            processExecutor,
                            new LoggedProcessOutputHandler(new LoggerWrapper(getLogger())));

      		// A Processor object is generated and passed to AidlCompileRunnable
      		// Execute asynchronously in this thread
            for (File dir : sourceFolders) {
                workers.submit(AidlCompileRunnable.class, new AidlCompileParams(dir, processor));
            }
            workers.close();
    }
Copy the code

3.3.2 generateDebugBuildConfig (Generate BuildConfig file)

The createBuildConfigTask method called in AppplicationTaskManager generates a BuildConfig file. Look at the createBuildCoonfigTask method, which creates the GenerateBuildConfig object. Look at the TaskAction tag:

## GenerateBuildConfig.java
    @TaskAction
    void generate(a) throws IOException {
        
        File destinationDir = getSourceOutputDir();
        FileUtils.cleanOutputDir(destinationDir);

        BuildConfigGenerator generator = new BuildConfigGenerator(
                getSourceOutputDir(),
                getBuildConfigPackageName());

        // Add default attributes, including DEBUG, APPLICATION_ID, FLAVOR, VERSION_CODE, VERSION_NAME
        generator
                .addField(
                        "boolean"."DEBUG",
                        isDebuggable() ? "Boolean.parseBoolean(\"true\")" : "false")
                .addField("String"."APPLICATION_ID".'"' + appPackageName.get() + '"')
                .addField("String"."BUILD_TYPE".'"' + getBuildTypeName() + '"')
                .addField("String"."FLAVOR".'"' + getFlavorName() + '"')
                .addField("int"."VERSION_CODE", Integer.toString(getVersionCode()))
                .addField(
                        "String"."VERSION_NAME".'"' + Strings.nullToEmpty(getVersionName()) + '"')
                .addItems(getItems());   // Add custom attributes

        List<String> flavors = getFlavorNamesWithDimensionNames();
        int count = flavors.size();
        if (count > 1) {
            for (int i = 0; i < count; i += 2) {
                generator.addField(
                        "String"."FLAVOR_" + flavors.get(i + 1), '"' + flavors.get(i) + '"'); }}// Internally call javaWriter to generate the file
        generator.generate();
    }
Copy the code

We created BuildConfigGenerator first, added DEBUG, APPLICATION_ID, FLAVOR, VERSION_CODE, VERSION_NAME, and custom properties, Finally, call JavaWriter to generate the BuildConfig.java file.

3.3.3 mergeDebugResources (Merge resource files)

The corresponding implementation class for mergeDebugResources is MergeResources.

Put together the resources in the RES directory and compile and package them using AAPT2 (Android Resource packaging tool). Aapt2 divide the task of resource compilation into compilation and connection. They will generate flat files and compile resource files into binary files.

Call link: MergeResources.doFullTaskAction() -> merger.mergeData() -> DataMerger.mergeData() -> MergedResourceWriter.end() -> ResourceCompiler.submitCompile -> AaptV2CommandBuilder.makeCompileCommand()

Going back to the source, you can see MergeResources supports incremental compilation, so let’s take a look at its doFullTaskAction() method:

  1. All resourcesets are retrieved, which includes all res resources in the project, and traversed into the ResourceMerger collection

    List<ResourceSet> resourceSets = getConfiguredResourceSets(preprocessor); .for (ResourceSet resourceSet : resourceSets) {
            resourceSet.loadFromFiles(getILogger());
            merger.addDataSet(resourceSet);
        }
Copy the code

  2. Call ResourceMerger to consolidate resources

    merger.mergeData(writer, false /*doCleanUp*/);
Copy the code

  3. Call the mergeData method of the DataMerger, and call the MergedResourceWriter’s start(), addItem(), and end() methods. Add to ValuesResMap and CompileResourceRequests containers, respectively:

    @Override
   public void addItem(@NonNull final ResourceMergerItem item) throws ConsumerException {...if (type == ResourceFile.FileType.XML_VALUES) {
           // Add the XML file
           mValuesResMap.put(item.getQualifiers(), item);
       } else{...// Collect new processing requests
           mCompileResourceRequests.add(
                       newCompileResourceRequest(file, getRootFolder(), folderName)); }}Copy the code

  4. Finally, take a look at MergedResourceWriter’s end method and its parent class implementation, which calls the postWriteAction method, iterates through the resource file, creates the Request object, and finally adds it to the ResourceCompiler.

    ##  MergeResourceWriter
@Override
    protected void postWriteAction(a) throws ConsumerException {...// now write the values files.
        for (String key : mValuesResMap.keySet()) {
        	......
        	CompileResourceRequest request =
                            newCompileResourceRequest( outFile, getRootFolder(), folderName, pseudoLocalesEnabled, crunchPng, blame ! =null? blame : ImmutableMap.of()); . mResourceCompiler.submitCompile(request); }}Copy the code

    Back in MergedResourceWriter’s end method, we see that the resource will be processed in the end method.

    mResourceCompiler.submitCompile(
                            new CompileResourceRequest(
                                    fileToCompile,
                                    request.getOutputDirectory(),
                                    request.getInputDirectoryName(),
                                    pseudoLocalesEnabled,
                                    crunchPng,
                                    ImmutableMap.of(),
                                    request.getInputFile()));
Copy the code

    Final call AaptV2CommandBuilder. MakeCompileCommand () method to generate aapt2 command to deal with resources, processing after generation XXX. XML. Flat format.

    When retrieving resourceSets, the modified file is used, and the image-to-WebP plug-in can also be placed in front of this task.

3.3.4 processDebugResources (Package resource files)

ProcessDebugResources corresponding implementation class is LinkApplicationAndroidResourcesTask.

Call link: LinkApplicationAndroidResourcesTask.doFullTaskAction() -> AaptSplitInvoker.run() -> invokeAaptForSplit() -> AndroidBuilder.processResources() ->aapt.link()

Will call AaptV2CommandBuilder. MakeLinkCommand () method of packaging resources and generate R.j and ava documents. Ap_ resources.

3.3.5 processDebugManifest (Merging manifest files)

ProcessDebugManifest corresponding implementation class is ProcessApplicationManifest

Finally merge the Androidmanifest.xml file.

3.3.6 transformClassesWithDexBuilderForDebug pack to dex (class)

TransformClassesWithDexBuilderForDebug implementation class is TransformTask, and truly achieve. Class files into. Dex file is DexArchiveBuilderTransform.

Call link: DexArchiveBuilderTransform.transform -> convertToDexArchive -> launchProcessing -> dexArchiveBuilder.convert -> DxDexArchiveBuilder.dex -> CfTranslator.translate

Take a look at its transform method:

public void transform(@NonNull TransformInvocation transformInvocation)
            throws TransformException, IOException, InterruptedException {...for (DirectoryInput dirInput : input.getDirectoryInputs()) {
                    logger.verbose("Dir input %s", dirInput.getFile().toString());
                  // convertToDexArchive will be called
                    convertToDexArchive(
                            transformInvocation.getContext(),
                            dirInput,
                            outputProvider,
                            isIncremental,
                            bootclasspathServiceKey,
                            classpathServiceKey,
                            additionalPaths);
                }
						
                for (JarInput jarInput : input.getJarInputs()) {
                    logger.verbose("Jar input %s", jarInput.getFile().toString()); .// the third party will call processJarInputList<File> dexArchives = processJarInput( transformInvocation.getContext(), isIncremental, jarInput, outputProvider, bootclasspathServiceKey, classpathServiceKey, additionalPaths, cacheInfo); . }Copy the code

The convertToDexArchive method is also eventually called inside the processJarInput method, mainly for subsequent dir and JAR processing. The launchProcessing method is called inside.

private static void launchProcessing(
            @NonNull DexConversionParameters dexConversionParameters,
            @NonNull OutputStream outStream,
            @NonNull OutputStream errStream,
            @NonNull MessageReceiver receiver)
            throws IOException, URISyntaxException {...// is an abstract class, and will be processed according to the d8 tool or dx tool to complete the transformation from.class code to.dex code
    dexArchiveBuilder.convert(
                    entries,
                    Paths.get(newURI(dexConversionParameters.output)), dexConversionParameters.isDirectoryBased()); . }Copy the code

DexArchiveBuilder. Convert there are two subtypes of implementation, D8DexArchiveBuilder and DxDexArchiveBuilder, were called to play dex d8 and dx.

Take a look at the final result above:

summary

With that in mind, let’s recall what happened in the gradle Plugin build and what tasks were involved in the APK packaging process.

reference

【Android training manual 】 common technology – talk about Android packaging

Android Gradle Plugin – Android Gradle Plugin main process analysis

Complete the Android skill tree — from the AGP build process to the APK packaging process

An in-depth exploration of Gradle automatic construction technology

Android Gradle Plugin Plugin