Original Xuduokai Baidu App technology

preface

Every Android programmer works with Gradle, directly or indirectly, whether they are aware of Gradle or not. Every time you create a new project with Android Studio, AS automatically creates a generic directory structure, which you can then develop by adding dependencies to your app’s build.gradle, clicking Sync Now in the upper right corner, and writing code. Click the small green arrow Run to Run the code, and everything is fine except occasionally Sync Now will fail and change to Try Again. If the problem is not solved after several tries Again, open the browser, paste the corresponding error, find the solution, copy and paste the solution, Click Try Again, problem solved, life is still good.

If you want to understand why problems occur and why doing so solves them, you need to step away from the common buttons of AS and explore the secret behind it — Gradle, the automated build tool.

This article will introduce:

  1. Why do you need an automated build tool?
  2. What are the Android projects created by default
  3. Dependency management
  4. Packaging process

By reading this article, you can get a general idea of how Gradle works. You can search for relevant content and solve common compilation errors more quickly

Why do you need an automated build tool?

The following commands are only examples to illustrate the problem. For details about how to use these commands, refer to the related command manuals

As we know, an APK package is actually a ZIP package containing code and resources. So we can write a Shell script called Assemble. Sh that anyone can execute to get the APK package, perfect:

  1. To convert the.java file to a. Class file, run the javac xxx.java command
  2. Android also converts.class files into.dex files: dx xxx.class
  3. Package into APK: zip xxx.apk [Code and resources to be packaged]

In Android, the code references resources through the R.java file, so you need to continue adding commands and require that the command be executed before the javac command. In actual development, it is impossible for us to implement all functions by ourselves, and we may rely on excellent open source libraries. The modified pseudo-code is as follows:

  1. Generate r.java: aapt [resource file]
  2. To convert the.java file to a. Class file, run the javac xxx.java r.java-classpath xxx.jar command
  3. Android also converts.class files into.dex files: dx xxx.class r.class xx.jar
  4. Package into APK: zip xxx.apk [Code and resources to be packaged]

Everything seems to be under control, really? Let’s take a look at the actual packaging process for Android APK:

  • For multiple projects, each project requires a copy of the above Shell script
  • For a single project, each time a feature is added, a piece of code needs to be inserted into the original process. As the requirements increase, the script becomes difficult to maintain
  • How do you manage imported external dependencies? How to open debug and Release packages? How to play multi-channel package?

Gradle is one of the automated build tools that can simplify this process by using conventions such as placing code, resources, etc., in a specified directory and using build scripts to quickly produce the final build product.

In contrast to the simple example above, every Project is called a Project in Gradle. Every Task that needs to be performed, such as generating R files, compiling Java files, etc., is called a Task in Gradle. DependsOn (TaskB) to implement TaskB and TaskA. Gradle also provides doFirst and doLast to execute some code before and after each Task.

At this point, we know why we need an automated build tool:

  • Prevent manual build intervention
  • Create repeatable builds
  • And most importantly: improve programming efficiency and focus on requirements development

What are the Android projects created by default

Gradle and idea

.gradle and.idea store the gradle and AS caches for the current project.

One of the most common applications is that after clicking sync, AS generates.iml files for each project. They work with.gradle and.idea to provide common functions such AS code hints. So if your code flares red and you’re sure your dependencies are ok, try the following steps to clear the AS cache:

  • Delete. Idea Delete. Gradle files
  • The command line executes./gradlew clean
  • Select File -> Invalidate caches/restart
  • Sync

Gradle/wrapper with gradlew gradlew. Bat

When we first configure the Android environment, we need to install Java and install AS, but we don’t need to install Gradle because of Gradle/Wrapper.

When the Gradlew script is executed, it ensures that each project uses the Gradle version it expects. The secret lies in this piece of gradlew code

exec "$JAVACMD" "${JVM_OPTS[@]}" -classpath "$CLASSPATH" org.gradle.wrapper.GradleWrapperMain "$@"
Copy the code

Instead of starting Gradle directly, Gradlew starts gradle-wrapper.jar, which determines that if there is no Gradle environment, Download the appropriate distributionUrl from gradle-wrapper.properties and start Gradle.

Since Gradle allows command line startup with additional parameters to customize Gradle’s runtime environment, Baidu app customizable gradle-wrapper.jar, Debug /release packages can be used to specify different gradle running memory for computers with different memory sizes.

setting.gradle

The root directory build. Gradle

Maven repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories repositories You can also add your own Maven repository via maven methods. Note that it should not be assumed that components will always be pulled from a particular repository. If Gradle requests a repository timeout, it will automatically request other repositories.

Dependencies: Represents the dependencies that Gradle needs to execute. Such as need Android Gradle Plugin plug-in packaging apk package for us, you will need to add: the classpath ‘com. Android. View the build: Gradle: 3.4.0’

Finally, allprojects and repositories: configuration for allprojects in allprojects, repositories for project declaration dependencies

app build.gradle

Android code blocks are apis provided by plug-ins that allow us to modify the behavior of tasks.

The contents of the dependencies code block determine which components the Project depends on. Different dependency declarations have different results, which will be analyzed in the next section.

Dependency management

Depend on the configuration

In the Android Gradle Plugin 3.0 era, Google uses implementation and API options instead of compile. Since the interface has changed, Google has simply renamed the other configuration items to make it easier for people to understand their configuration. Note that the old interface was not removed immediately, but will be removed in the next major release. The following is the official Chinese explanation of each configuration item:











In fact, every component has its own compileClasspath and runtimeClasspath and when a component is compiled, Gradle puts it in compileClasspath and when a component is packed, Gradle will place it in runtimeClasspath

Different dependency configuration items, just put declared dependencies into different classpath of different components, go back to the implementation example and put C into B’s compileClasspath and runtimeClasspath, Put it into A’s runtimeClasspath, so that if A calls C’s code, javAC will report an error during A’s compile phase, but eventually C will be packaged into APK

The same applies to API, compileOnly and runtimeOnly

Source code and binary

When you want to rely on a source code project you just say: Implementation Project (‘:demo: myLibrary ‘)

And we know for sure that the dependencies in MyLibrary will be packaged correctly into APK

When we rely on binary need to write like this: implementation ‘androidx. Appcompat: appcompat: 1.0.2’

When executing dependencies (only runtimeClasspath of the release package) : ./gradlew :app:dependencies –configuration releaseRuntimeClasspath > dependencies.txt

When you output a dependency graph, you see that instead of just relying on an AppCompat component (which shows only partial dependencies), you also include the component’s own dependencies, and dependencies of dependencies until the component itself has no dependencies, a feature called dependency passing

releaseRuntimeClasspath - Resolved configuration for runtime forvariant: Release \ - androidx appcompat: appcompat: 1.0.2 + - androidx. Annotation: the annotation: 1.0.0 + - androidx. Core: the core: 1.0.1 | + -- -- -- androidx. Annotation: the annotation: 1.0.0. | + -- -- -- androidx collection: collection: 1.0.0 | | \ -- -- -- Androidx. Annotation: the annotation: 1.0.0. | + -- -- -- androidx lifecycle: lifecycle - runtime: 2.0.0 | | + -- -- -- Androidx. Lifecycle: lifecycle - common: 2.0.0 | | | \ - androidx annotation: the annotation: 1.0.0 | | + -- -- -- Androidx. Arch. The core: the core - common: 2.0.0 | | | \ - androidx annotation: the annotation: 1.0.0 | | \ -- -- -- Androidx. Annotation: the annotation: 1.0.0 | \ - androidx versionedparcelable: versionedparcelable: 1.0.0 | + -- -- -- Androidx. Annotation: the annotation: 1.0.0 | \ - androidx collection: collection: 1.0.0 + - (*) Androidx. Collection: collection: 1.0.0 (*). + - androidx cursoradapter: cursoradapter: 1.0.0Copy the code

How does Gradle determine these dependencies? When a component is uploaded using the Maven specification, not only the binary of the component is uploaded, but also a pom.xml file in which the dependency information is stored.

Because it may be necessary to go over the wall to view the public Maven server, the following is to show you the server background built by Baidu App itself, so as to understand how the uploaded binary is stored in the server structure

After looking at the remote POM file, let’s see how the binary is stored locally when downloaded

Two practical examples: 1. Suppose A depends on B, and B depends on C

















Rely on the conflict

What is dependency conflict:

How to resolve dependency conflicts:

  1. When compiling, B relies on VERSION 1.0 of D and C relies on version 1.1 of D, but ultimately D is packaged into APK with version 1.1, because the highest version is chosen by default due to version conflicts
  2. Gradle provides a set of rules for resolving dependency conflicts, such as disallow dependency passing, exclude to remove a dependency, and replace one component with another
  3. Baidu app adds a rule on this basis: If the version number of the final application is higher than the version number defined in version.properties, an error will be reported

Note:

  1. Assuming D releases 1.2, but neither B nor C releases a new version based on D 1.2, the final package is D 1.1, so all components are packaged into the APK package with the version defined in version.properties
  2. Suppose D’s MavenId is changed from D to E, and C releases binary based on E, while B remains the same. In actual packaging, class repeat errors will be reported. The reason is that B’s POM file still relies on D, so B needs to re-issue a binary based on E renamed after D

With this in mind for the packaging process, let’s actually look at what other tasks are actually executed when the packaging Task is executed. The environment configuration is as follows: Gradle 5.1.1 Android Gradle Plugin 3.1.2 org.gradle. Parallel =true Enable parallel compilation release package minifyEnabled true

Execute the command to get the following output

# --dry-run indicates that each Task is not actually executed

gradlew assembleRelease --dry-run
Copy the code

preBuild

Description: Do some pre-compile checks One example: some people may experience the following error

"Android dependency "+ display+ "is set to compileOnly/provided which is not supported"
Copy the code

The reason for this is the compileClasspath and runtimeClasspath we talked about earlier. When a component is compileClasspath 1.1.1 but runtimeClasspath 1.1.2 due to different dependency profiles, preBuild will detect this and report an error to compileClasspath 1.1.2

compileReleaseAidl

Description: Internally uses AidlProcessor to call the call method using aiDL under build-tool to perform compilation.

Generate and merge

These tasks allow us to dynamically generate some code throughout the compilation project. The generated resources need to be merged with existing resources, and we need to pay attention to the possibility of overwriting existing resources, so I won’t go into details.

process

This parameter is an application to compileClasspath. If your source file references a class whose JAR package is not in compileClasspath, Javac will report an error and the symbol will not be found

The second step: After the source files are compiled into class files, Google provides a Transform mechanism that allows us to modify the binary before packaging, Such as: in the app: transformClassesWithXXXForRelease SKIPPED the Transform is our custom. By: app: transformClassesAndResourcesWithProguardForRelease SKIPPED can see Proguard is implemented through the Transform mechanism, The image here shows a.class file and a.jar/.class file. The first is obviously a javac compilation, and the second is runtimeClasspath, which is the binary that needs to be packaged. So I think you understand how compileClasspath and runtimeClasspath affect the packaging process

Step 3: After the Transform has processed all the class files, the next step is to convert the.class file to the.dex file. It is important to note that Javac can only find problems with the source code, not with binaries that were not compiled. During dex conversion, serious code problems can be found, such as Class duplication or a Class whose name remains the same but is changed from Class to Interface.

Step 4: Package the previous and resources. The corresponding class is PackageApplication, and once you get the Task, you can customize the packaged content

conclusion

Gradle life cycle, Plugin development, Transform mechanism, etc. But if you can have a general understanding of the whole compilation tool chain, you can know when you need to go in which direction to solve the problem, is the value of this article.

The resources

Docs.gradle.org/current/use…

Developer.android.com/studio/buil…

Github.com/gradle/grad…

Android.googlesource.com/platform/to…

Pay special attention to

Welcome to join baidu App mobile R&D team. We advocate pragmatic, free and open technology culture. There is a perfect platform of basic ability here, and there are great talents in various fields, and there are more opportunities for growth. Join us and you get more than a job, you get a chance to change the world.

At present, there are Android/iOS/FE positions at all levels, you can check the relevant positions through baidu recruitment website, or directly send your resume to [email protected]