Statement:

Baidu APP technology [original: Xuduokai] Link: juejin.cn/post/684490…

preface

Every Android programmer works with Gradle, directly or indirectly, whether they are aware of Gradle or not. Every time you create a new project with Android Studio, AS automatically creates a generic directory structure, which you can then develop by adding dependencies to your app’s build.gradle, clicking Sync Now in the upper right corner, and writing code. Click the small green arrow Run to Run the code, and everything is fine except occasionally Sync Now will fail and change to Try Again. If the problem is not solved after several tries Again, open the browser, paste the corresponding error, find the solution, copy and paste the solution, Click Try Again, problem solved, life is still good.

If you want to understand why problems occur and why doing so solves them, you need to step away from the common buttons of AS and explore the secret behind it — Gradle, the automated build tool.

This article will introduce:

  1. Why do you need an automated build tool?
  2. What are the Android projects created by default
  3. Dependency management
  4. Packaging process

By reading this article, you can get a general idea of how Gradle works. You can search for relevant content and solve common compilation errors more quickly

Why do you need an automated build tool?

The following commands are only examples to illustrate the problem. For details about how to use these commands, refer to the related command manuals

As we know, an APK package is actually a ZIP package containing code and resources. So we can write a Shell script called Assemble. Sh that anyone can execute to get the APK package, perfect:

  1. To convert the.java file to a. Class file, run the javac xxx.java command
  2. Android also converts.class files into.dex files: dx xxx.class
  3. Package into APK: zip xxx.apk [Code and resources to be packaged]

In Android, the code references resources through the R.java file, so you need to continue adding commands and require that the command be executed before the javac command. In actual development, it is impossible for us to implement all functions by ourselves, and we may rely on excellent open source libraries. The modified pseudo-code is as follows:

  1. Generate r.java: aapt [resource file]
  2. To convert the.java file to a. Class file, run the javac xxx.java r.java-classpath xxx.jar command
  3. Android also converts.class files into.dex files: dx xxx.class r.class xx.jar
  4. Package into APK: zip xxx.apk [Code and resources to be packaged]

Everything seems to be under control, really? Let’s take a look at the actual packaging process for Android APK:

Isn’t a Shell script too big to think about implementing such a complex process? Don’t worry, the implementation will also encounter the following problems:

  • For multiple projects, each project requires a copy of the above Shell script
  • For a single project, each time a feature is added, a piece of code needs to be inserted into the original process. As the requirements increase, the script becomes difficult to maintain
  • How do you manage imported external dependencies? How to open debug and Release packages? How to play multi-channel package?

Gradle is one of the automated build tools that can simplify this process by using conventions such as placing code, resources, etc., in a specified directory and using build scripts to quickly produce the final build product.

In contrast to the simple example above, every Project is called a Project in Gradle. Every Task that needs to be performed, such as generating R files, compiling Java files, etc., is called a Task in Gradle. DependsOn (TaskB) to implement TaskB and TaskA. Gradle also provides doFirst and doLast to execute some code before and after each Task.

At this point, we know why we need an automated build tool:

  • Prevent manual build intervention
  • Create repeatable builds
  • And most importantly: improve programming efficiency and focus on requirements development

What are the Android projects created by default

Every time you create a new project with Android Studio, AS automatically creates a directory structure like the one shown in the image above. The image briefly explains what each directory or file is for.

Gradle and idea

.gradle and.idea store the gradle and AS caches for the current project.

One of the most common applications is that after clicking sync, AS generates.iml files for each project. They work with.gradle and.idea to provide common functions such AS code hints. So if your code flares red and you’re sure your dependencies are ok, try the following steps to clear the AS cache:

  • Delete. Idea Delete. Gradle files
  • The command line executes./gradlew clean
  • Select File -> Invalidate caches/restart
  • Sync

Gradle/wrapper with gradlew gradlew. Bat

When we first configure the Android environment, we need to install Java and install AS, but we don’t need to install Gradle because of Gradle/Wrapper.

When the Gradlew script is executed, it ensures that each project uses the Gradle version it expects. The secret lies in this piece of gradlew code

exec "$JAVACMD" "${JVM_OPTS[@]}" -classpath "$CLASSPATH" org.gradle.wrapper.GradleWrapperMain "$@"

Copy the code

Instead of starting Gradle directly, Gradlew starts gradle-wrapper.jar, which determines that if there is no Gradle environment, Download the appropriate distributionUrl from gradle-wrapper.properties and start Gradle.

Since Gradle allows command line startup with additional parameters to customize Gradle’s runtime environment, Baidu app customizable gradle-wrapper.jar, Debug /release packages can be used to specify different gradle running memory for computers with different memory sizes.

setting.gradle

Gradle includes the include method, which specifies which projects to compile. For each Project that builds, Gradle creates a Project object

The root directory build. Gradle

First, buildScript blocks: Gradle executes from the top down by default, and will execute first wherever buildScript blocks are located

Maven repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories You can also add your own Maven repository via maven methods. Note that it should not be assumed that components will always be pulled from a particular repository. If Gradle requests a repository timeout, it will automatically request other repositories.

Dependencies: Represents the dependencies that Gradle needs to execute. Such as need Android Gradle Plugin plug-in packaging apk package for us, you will need to add: the classpath ‘com. Android. View the build: Gradle: 3.4.0’

Finally, allprojects and repositories: configuration for allprojects in allprojects, repositories for project declaration dependencies

app build.gradle

First, you can see the Apply plugin: ‘com.android.application’, when applied, it will create a series of tasks for us, such as assembleDebug and assembleRelease, and execute these tasks to get the final APK.

Android code blocks are apis provided by plug-ins that allow us to modify the behavior of tasks.

The contents of the dependencies code block determine which components the Project depends on. Different dependency declarations have different results, which will be analyzed in the next section.

Dependency management

Depend on the configuration

In the Android Gradle Plugin 3.0 era, Google uses implementation and API options instead of compile. Since the interface has changed, Google has simply renamed the other configuration items to make it easier for people to understand their configuration. Note that the old interface was not removed immediately, but will be removed in the next major release. The following is the official Chinese explanation of each configuration item:

For example, suppose A depends on B, and B depends on C. If B uses A implementation dependency on C, A can’t call C’s code if B uses an API dependency on C, A can call C’s code if B uses A compileOnly dependency on C, If B has A runtimeOnly dependency on C, A and B cannot call C’s code, but C’s code is packaged into APK

In fact, every component has its own compileClasspath and runtimeClasspath and when a component is compiled, Gradle puts it in compileClasspath and when a component is packed, Gradle will place it in runtimeClasspath

Different dependency configuration items, just put declared dependencies into different classpath of different components, go back to the implementation example and put C into B’s compileClasspath and runtimeClasspath, Put it into A’s runtimeClasspath, so that if A calls C’s code, javAC will report an error during A’s compile phase, but eventually C will be packaged into APK

The same applies to API, compileOnly and runtimeOnly

Source code and binary

When you want to rely on a source code project you just say: Implementation Project (‘:demo: myLibrary ‘)

And we know for sure that the dependencies in MyLibrary will be packaged correctly into APK

When we rely on binary need to write like this: implementation ‘androidx. Appcompat: appcompat: 1.0.2’

When executing dependencies (only runtimeClasspath of the release package) : ./gradlew :app:dependencies –configuration releaseRuntimeClasspath > dependencies.txt

When you output a dependency graph, you see that instead of just relying on an AppCompat component (which shows only partial dependencies), you also include the component’s own dependencies, and dependencies of dependencies until the component itself has no dependencies, a feature called dependency passing

releaseRuntimeClasspath - Resolved configuration for runtime for variant: Release \ - androidx appcompat: appcompat: 1.0.2 + - androidx. Annotation: the annotation: 1.0.0 + - androidx. Core: the core: 1.0.1 | + -- -- -- androidx. Annotation: the annotation: 1.0.0. | + -- -- -- androidx collection: collection: 1.0.0 | | \ -- -- -- Androidx. Annotation: the annotation: 1.0.0. | + -- -- -- androidx lifecycle: lifecycle - runtime: 2.0.0 | | + -- -- -- Androidx. Lifecycle: lifecycle - common: 2.0.0 | | | \ - androidx annotation: the annotation: 1.0.0 | | + -- -- -- Androidx. Arch. The core: the core - common: 2.0.0 | | | \ - androidx annotation: the annotation: 1.0.0 | | \ -- -- -- Androidx. Annotation: the annotation: 1.0.0 | \ - androidx versionedparcelable: versionedparcelable: 1.0.0 | + -- -- -- Androidx. Annotation: the annotation: 1.0.0 | \ - androidx collection: collection: 1.0.0 + - (*) Androidx. Collection: collection: 1.0.0 (*). + - androidx cursoradapter: cursoradapter: 1.0.0Copy the code

How does Gradle determine these dependencies? When a component is uploaded using the Maven specification, not only the binary of the component is uploaded, but also a pom.xml file in which the dependency information is stored.

Because it may be necessary to go over the wall to view the public Maven server, the following is to show you the server background built by Baidu App itself, so as to understand how the uploaded binary is stored in the server structure

This is the background of Maven server built by Baidu App. Click one item to view details:

There are binary AAR uploads, poM files, and readMe files that we customize at upload time

After looking at the remote POM file, let’s see how the binary is stored locally when downloaded

Here is a simple POM file:

Runtime will not compile, but compile will package, and compile will compile and package

Two practical examples: 1. Suppose A depends on B, and B depends on C

Class Foo inherits Bar from C; class Foo inherits Bar from C; class Foo inherits Bar from C; If A depends on B and B depends on C, BC is binary and B’s POM depends on C. In Gradle 4.4, A can still call C’s code. This problem was fixed after Gradle 5.0

Rely on the conflict

What is dependency conflict:

Suppose ABC is source code, D is binary, A declaration depends on B, A declaration depends on C, B declaration depends on D 1.0, C declaration depends on D 1.1, in this case, D has dependency conflict, need to determine whether to use 1.0 or 1.1

How to resolve dependency conflicts:

  1. When compiling, B relies on VERSION 1.0 of D and C relies on version 1.1 of D, but ultimately D is packaged into APK with version 1.1, because the highest version is chosen by default due to version conflicts
  2. Gradle provides a set of rules for resolving dependency conflicts, such as disallow dependency passing, exclude to remove a dependency, and replace one component with another
  3. Baidu app adds a rule on this basis: If the version number of the final application is higher than the version number defined in version.properties, an error will be reported

Note:

  1. Assuming D releases 1.2, but neither B nor C releases a new version based on D 1.2, the final package is D 1.1, so all components are packaged into the APK package with the version defined in version.properties
  2. Suppose D’s MavenId is changed from D to E, and C releases binary based on E, while B remains the same. In actual packaging, class repeat errors will be reported. The reason is that B’s POM file still relies on D, so B needs to re-issue a binary based on E renamed after D

With this in mind for the packaging process, let’s actually look at what other tasks are actually executed when the packaging Task is executed. The environment configuration is as follows: Gradle 5.1.1 Android Gradle Plugin 3.1.2 org.gradle. Parallel =true Enable parallel compilation release package minifyEnabled true

Execute the command to get the following output

Gradlew assembleRelease --dry-runCopy the code

There are many tasks. Next, I will introduce a few key tasks for you. If you are not introduced, you can find the corresponding implementation class and see how it is implemented.

preBuild

Description: Do some pre-compile checks One example: some people may experience the following error

"Android dependency "+ display+ "is set to compileOnly/provided which is not supported"

Copy the code

The reason for this is the compileClasspath and runtimeClasspath we talked about earlier. When a component is compileClasspath 1.1.1 but runtimeClasspath 1.1.2 due to different dependency profiles, preBuild will detect this and report an error to compileClasspath 1.1.2

compileReleaseAidl

Description: Internally uses AidlProcessor to call the call method using aiDL under build-tool to perform compilation.

Generate and merge

These tasks allow us to dynamically generate some code throughout the compilation project. The generated resources need to be merged with existing resources, and we need to pay attention to the possibility of overwriting existing resources, so I won’t go into details.

process

The first step: We have Java source files under APP project, Java source files generated by AIDL, generate, R.Java, etc., as well as a series of binary files such as jar packages of source code sub-project and remote AAR decompression. The source files are what javac needs to compile, and the binaries.jar/.class are the set of files that javac should look for if a class name or other symbol is not found in the existing source file. The corresponding Javac parameter is -classpath

This parameter is an application to compileClasspath. If your source file references a class whose JAR package is not in compileClasspath, Javac will report an error and the symbol will not be found

The second step: After the source files are compiled into class files, Google provides a Transform mechanism that allows us to modify the binary before packaging, Such as: in the app: transformClassesWithXXXForRelease SKIPPED the Transform is our custom. By: app: transformClassesAndResourcesWithProguardForRelease SKIPPED can see Proguard is implemented through the Transform mechanism, The image here shows a.class file and a.jar/.class file. The first is obviously a javac compilation, and the second is runtimeClasspath, which is the binary that needs to be packaged. So I think you understand how compileClasspath and runtimeClasspath affect the packaging process

Step 3: After the Transform has processed all the class files, the next step is to convert the.class file to the.dex file. It is important to note that Javac can only find problems with the source code, not with binaries that were not compiled. During dex conversion, serious code problems can be found, such as Class duplication or a Class whose name remains the same but is changed from Class to Interface.

Step 4: Package the previous and resources. The corresponding class is PackageApplication, and once you get the Task, you can customize the packaged content

conclusion

Gradle life cycle, Plugin development, Transform mechanism, etc. But if you can have a general understanding of the whole compilation tool chain, you can know when you need to go in which direction to solve the problem, is the value of this article.