Introduction to Incremental compilation
Incremental compilation is as opposed to full compilation. Incremental compilation means that when a part of the source program is changed, the work of progressive recompilation is limited to the modified part and the content of the related part, and does not need to compile all the code. Incremental compilation can greatly shorten compilation time and improve compilation efficiency for software development, especially during debugging.
Full compilation means that when the user source program is partially modified, the recompilation of the code involves the entire source code, not just the partial modification and related parts. In other words, no matter what changes are made, the full compilation will be a completely new and complete compilation, not based on the previous compilation.
In general, in software development, full compilation is used to build and release versions, which can be time consuming and resource consuming. And in the debugging stage of the program, generally adopt incremental compilation, so that the location and solution of the problem than save time and effort. In Android development, as the amount of project code expands, the compilation time becomes longer and longer, which slows down the development efficiency. Therefore, Android officially introduced incremental updates such as Instant Run and Apply Changes.
Instant Run profile
Instant Run is an incremental build feature for Android Studio 2.0. To use Instant Run, you need to set minSdkVersion to 15 or above in your build.gradle file, and for maximum performance, you need to set minSdkVersion to 15 or above in your build.gradle file. You can set minSdkVersion to 21 or higher.
In a previous test run on Android Studio 3.0, Gradle 2.14.1, compiling a simple Demo project was reduced from 10 seconds to around 2 or 3 seconds. By default, Instant Run is disabled. To enable Instant Run, you can enable Instant Run in Settings, as shown.
Apply Changes
In Android Studio 3.5 and above, Apply Changes is officially available. To use Apply Changes, you need to meet the following two conditions:
- Apk must be a debug package;
- Run on Android 8.0 and above
When we run the project in Android Studio, we will see three buttons in the menu bar, one for restarting the application, as shown below.
As shown in the figure above, the left-to-right buttons represent Run, Apply Changes, and Apply Code Changes respectively.
- Run: Deploys all changes and restarts the application.
- Apply Changes: Attempts to Apply the changed resources and code and just restart the Activity without restarting the entire application.
- Apply Code Changes: Attempts to Apply Code Changes without restarting the operation. If only Code Changes are made, you can use this button to make the Code take effect.
However, because Apply Changes only works on Android 8.0 or later, the actual speed increase is not significant.
Freeline
Packages, in addition to the official alibaba client team also developed a for the Android platform based on dynamic replacement of incremental compilation tool, it can make full use of the cache file, within a few seconds quickly to changes in the code to compile and deploy to the device, effectively reduce the daily development of a large number of recompiling and installation of the time-consuming.
Performance: Internally adopt the idea of multi-project, multi-task and concurrency similar to Facebook’s open source tool Buck: Port scanning, code scanning, concurrent compilation, concurrent DX, concurrent merge DEX and other strategies have obvious acceleration effects on multi-core machines. In addition, corresponding cache strategies are made at the level of class, DEX and resources to achieve real incremental development. In addition, part of the acceleration component DX of Buck is introduced and optimized. DexMerger, in terms of resource compilation, deeply transforms Aapt resource compilation process. When resources change, incremental package compilation can be completed at the second level, in which the incremental package only contains the smallest change set (within 10Kb ~ hundreds of Kb), and it can also be used for dynamic resource/code replacement online in the later stage. Several times faster than the current schemes such as Instant-run, Buck and Layoutcast.
However, Freeline also has some problems that can not be ignored. The first is the lack of support for Kotlin, which is fatal now that Kotlin has been officially announced by Google as the preferred language for Android development. In addition, resources with ids cannot be deleted. Otherwise, errors may occur during resource compilation.
Another potential problem is that Freeline sacrifices some of its correctness to ensure faster compilation. For example, when a public static constant is changed, only the corresponding class file is compiled. Other classes that reference the constant are not compiled. Constant inline optimization can cause these classes to run with the same old values, and the changes won’t take effect.
Android builds the packaging process
For details of how Android goes from source code to package installation, please refer to the Android official diagram, which includes compilation, linking, and signing.
The Android compilation and packaging process is divided into the following stages:
- R file generation: R file records the ID of each resource, and then participates in the Java compilation process. R file is generated by Android Asset Package Tool (AAPT).
- Java (Kotlin) source code: We know that sometimes in app development there will be cross-process communication, then you can define the interface in the way of AIDL, aiDL tools can generate corresponding Java files from AIDL files. R files, aiDL-related Java files, and Java files in SRC are then compiled to generate.class files.
- Dex generation: the compiled class is packaged into a dex file by the dex tool. Freeline, the Android incremental packaging tool, uses the dex extracted from Buck, and freeline’s data is 40% faster than the original dex tool
Resource file compilation
- Aapt (Android Asset Package Tool) : AAPT packages and archives resource files in apps.
The following figure illustrates the complete steps of Android compile time and run time.
Incremental compilation principle
Android incremental compilation is divided into code incremental and resource incremental. The early Instant Run scheme of Android is not incremental in resources, but pushes the resources of the whole application into resource packages to the mobile phone, so the efficiency is very low.
The code to compile
After Google supports multidex (that is, a typical 65535 problem), There will be multiple dex files after Android packaging. When loading classes at runtime, it will search from a dexList in turn and return if it is found. Using this principle, incremental code can be packaged into dex files and inserted into the front of the dexList. This completes the class substitution.
There are two main concerns with incremental compilation of code, namely, getting the change file and compiling it, and compiling the dependent code. For incremental compilation of code, you can refer to the incremental compilation scheme of QQ Music: QQ Music Android compilation speed road
Compile resource
Resource compilation is similar to code increment in that the changed resource is collected and then compiled. Android resource compilation mainly uses AAPT or AAPT2. Generally speaking, aapT2 is now used for compilation of packaged resources, because aAPT tools do not support compilation of individual resources.
Aapt2 (Android Resource Packaging Tools) are resource building tools that Android Studio and Android Gradle plug-ins use to compile and package applications. Aapt2 parse resources, index them, and compile them into binary formats optimized for the Android platform.
When aapT2 is used for resource packaging and compilation, it is divided into two steps: compile and link. In the compilation stage, it is responsible for compiling single or multiple resources into binary files. The link stage is responsible for merging all binaries and then packaging.
Android development to now, the front of a lot of incremental compilation scheme has been invalid, now the market is more mature QQ music team dynamic compilation scheme and Savitar Android incremental compilation scheme
Savitar is a good Android compiler
As projects go through multiple iterations, they run into all sorts of problems, and slow compilations are a problem every mature Android team has to deal with. In the previous technology sharing of the Android retail team, the entire Android project has 25 business modules, 45W+ lines of source code (Java + Kotlin) and several build Flavors. The average incremental build time for requirement development is two minutes. With Gradle configuration and APK installation, it takes almost three minutes to verify a single line of code (MacBook Pro 13-inch, 2016, I5-8G). This situation greatly reduces the team’s development efficiency.
Before Savitar was born, we tried some of the established solutions in the community, such as BUCK, Freeline, InstantRun, etc. However, there are more or less some problems.
For example, FaceBook’s BUCK framework has its own powerful build system and incremental build caching mechanism, which can effectively improve the speed of compilation. However, its use and configuration are too complex, causing a lot of intrusion into the project. Some Android features, such as Databinding and Kotlin, are not supported.
The second is Alibaba’s open source Freeline, which is known for its extremely fast deployment, but for us the fatal flaw is that it doesn’t support Kotlin.
InstantRun is Google’s recommended acceleration method and has the most comprehensive support, but since we are a multi-process project and InstantRun does some preparation tasks at compile time, the acceleration is not obvious in practice.
Plan implementation
Savitar is a great incremental compilation improvement solution for Android team, it can effectively reduce the compilation time of module modification, including the accompanying IDE plug-in, easy to use, with some of the following significant features.
- Support Java, Kotlin incremental compilation
- Supports incremental compilation of resource files such as layout, values, assets and images
- Provides GUI interface plug-ins
- Based on Git debugging and multi-branch management, branches can be changed according to the actual situation
The structure design
As shown in the figure, Savitar as a whole is divided into four parts:
- GUI plug-in part: user-oriented GUI interface, including automatic update of runnable Jar (hereinafter referred to as Runner), various inspection tasks, compilation script call execution
- Runner part: A Jar package containing Savitar core logic code, complete the task of obtaining changes, script generation, compilation and execution
- Engineering support: a Gradle plugin to get the engineering information and insert the product loading code
- External dependencies: External dependencies required to complete the process
The following diagram illustrates the complete process of Savitar from the code modification to the completion of the modification product load run.
- Obtaining change information: Obtaining code and resource changes is a prerequisite for the entire process
- Obtain project information: Obtain dependency information, directory information, and Git information of the current project to prepare for subsequent compilation
- Compile and generate products: compile code and resources, and generate Dex products and Apk products
- Restart the loading product: complete the loading and running of the compiled product, complete the whole acceleration process
For details on how Savitar works from getting change information to loading, see Savitar for Android incremental compilation.
How to use
To make it easier for developers to implement incremental updates with Savitar, Savitar has developed an IDE plug-in that can complete the compilation and packaging process with a one-click trigger. First, open Android Studio, then search for Savitar and install it by selecting [Preference] -> [Plugin], as shown below.
Answers to Common Documents
Is Kotlinx supported and how?
As shown below, there is the following code:
import kotlinx.android.synthetic.main.activity_main.*
class MainActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
buttonCpu.setOnClickListener {
... // Click the event}}}Copy the codeFor the above code, must have used Kotlin Android students will not be unfamiliar, using Kotlinx features, can be in the.kt code using Xml defined component Id directly obtain View instance for operation, greatly reduce UI development costs.
However, the import in the above code is not a common form, and such syntax will fail to find import errors if compiled directly using standard Kotlinc. In this case, you need to use the Kotlin compiler plug-in. When Kotlin is compiled, you can pass in the Jar address and parameters of the Kotlinx plug-in to complete the compilation of files containing Kotlinx syntax.
sh kotlinc
-Xplugin=lib/android-extensions-compiler.jar
-P plugin:org.jetbrains.kotlin.android:package=${package_name}
-P plugin:org.jetbrains.kotlin.android:variant='${flovar}; ${resource_package}'
Copy the codeDocumentation references the Kotlin compiler plug-in
Reference: QQ Music Android compilation speed road Android incremental compilation improvement scheme Savitar