Do compile optimization, said at the time a plan, is to compile time amend the all modules depend on to aar, then the compiler will change every time module into source code, compile do then modify the module at the same time to upload to aar, so that you can always do the least module to participate in the source code to compile, only by raising the compilation speed.
Of course, it is easy to say, but not so easy to do. Finally, a friend developed the above description into an open source solution, which is very worthy of learning and reference.
1. Background description
As the size of the project grows, so does the compilation speed, sometimes requiring several minutes of compilation time for a change.
Based on this common situation, RocketX was introduced to improve the speed of full compilation by dynamically changing project dependencies in the compilation process and dynamically replacing modules with AAR, so that only modified modules can be compiled and no other modules can be compiled without changing any of the original project code.
2. Effect display
2.1. Test project introduction
The target project has a total of 3W+ classes and resource files, which were fully compiled in about 4min (MBP 8 generation I7 16g in 18 years was used in the test).
The effect of RocketX’s full growth (averaged 3 times for each operation).
As shown in the figure below, APP relies on BM business module, while BM business module relies on the top-level Base/Comm module.
dependencies
• When the Base/Comm module is changed, all modules at the bottom must be compiled. Because the APP/BMXXX module may use the interface or variable in the Base module, and I don’t know whether it is changed to. (Then it’s very slow.)
• When bmDiscover is changed, only the APP module and bmDiscover module are required for compilation. (Faster)
• RX (RocketX) basically controls the compilation speed of any module in the 30s, because only app and modified modules are compiled, and other modules are aar packages and not involved in compilation.
Top level module speed increased by 300%+
3. Problem analysis and module building
3.1 Analysis of ideas and problems
It is necessary to dynamically modify the unmodified Module dependencies to the corresponding AAR dependencies in the form of Gradle Plugin. If the Module changes, it will degenerate into project dependencies. In this way, only the modified Module and APP modules will be compiled each time.
Change the Implement/API moduleB to Implement/API aarB.
You need to build local Maven to store aArs for unmodified Modules. (flatDir can also be used instead for faster speeds)
The compilation process starts and needs to find which module has been modified.
We need to iterate through the dependencies of each module for replacement. How to obtain module dependencies? Can all module dependencies be retrieved at once, or can they be called individually by module? Does modifying one module dependency block subsequent module dependency callbacks?
After each module transforms into an AAR, the child dependencies (network dependencies, AAR) of its own are given to the Parent Module (how to find all parent Modules). Or directly to the App Module? Is there any risk of app to Module dependency breaking? There needs to be a technical solution.
The hook compilation process is required to replace the modified AAR in Loacal Maven after completion.
Provide AS status bar button to enable on/off function, speed up compilation or let developers use the already accustomed triangular run button.
3.2. Module construction
According to the above analysis, although there are many problems, the whole project can be roughly divided into the following parts:
4. Problem solving and implementation
4.1, implement the entrance in the DynamicAddDependencyMethods tryInvokeMethod method to realize source. It is a methodMissing function for a dynamic language.
tryInvokeMethod
public DynamicInvokeResult tryInvokeMethod(String name, Object... Arguments) {// omit some code... return DynamicInvokeResult.found(this.dependencyAdder.add(configuration, normalizedArgs.get(0), (Closure)null)); }Copy the codeDependencyAdder Implementation is a DirectDependencyAdder.
private class DirectDependencyAdder implements DependencyAdder<Dependency> { private DirectDependencyAdder() { } public Dependency add(Configuration configuration, Object dependencyNotation, @Nullable Closure configureAction) { return DefaultDependencyHandler.this.doAdd(configuration, dependencyNotation, configureAction); }}
Copy the codeFinally in DefaultDependencyHandler. This. DoAdd added, while DefaultDependencyHandler can get in the project.
DependencyHandler getDependencies();
Copy the codeBased on the above analysis, adding the corresponding AAR/JAR can be done with the following code.
fun addAarDependencyToProject(aarName: String, configName: String, project: Project) {// add aar dependent on API /implementation/ XXX (name: 'libaccount-2.0.0', ext: 'aar'), source code using linkedMap if (! File(FileUtil.getLocalMavenCacheDir() + aarName + ".aar").exists()) return val map = linkedMapOf<String, String>() map.put("name", aarName) map.put("ext", "aar") // TODO: 2021/11/5 change depend on Behind here need to be modified into / / project dependencies. The add (configName, "Com. ${project. The name} : ${project. The name} : 1.0") project. The dependencies. The add (configName, map)}Copy the code4.2 localMave uses flatDir in preference by specifying a cache directory to generate aar/ JAR packages, which are replaced by lookups when dependent changes are made.
fun flatDirs() { val map = mutableMapOf<String, File>() map.put("dirs", File(getLocalMavenCacheDir())) appProject.rootProject.allprojects { it.repositories.flatDir(map) }}
Copy the code4.3 When the compilation process starts, it is necessary to find out which module has been modified.
Use lastModifyTime to traverse the files of the entire project.
With each module as a granularity, recursively traverse the current module file, and integrate the lastModifyTime of each file to calculate a unique identifier countTime.
Compare countTime with the last one. If it is the same, it will be changed. We need to synchronize the calculated countTime to the local cache.
The overall time of 3W files is 1.2s, which is acceptable.
4.4 Obtaining module dependency relationship.
Find the time to generate the dependency graph for the entire project and generate the dependency graph parser here with the following code.
project.gradle.addListener(DependencyResolutionListener listener)
Copy the code4.5 Module dependency Project is replaced with AAR technical solution
The traversal order of each Module dependency replacement is unordered, so the technical solution needs to support unordered replacement.
The current scheme used is: if the current module A is not changed, it needs to replace A with A. ar through localMaven, and give A. ar and the child dependency of A to the parent Module of the first layer. (You may wonder what to do if the parent Module is also an AAR. In fact, there is no problem with this part. I won’t elaborate on it here because it is too long.)
Why should parent be given to app? A simple example is shown below. If B. ar is not given to MODULE A, the interface of A using module B is missing, which will result in compilation failure.
Give the technical solution demonstration of the overall project replacement:
4.5 hook compilation process, replace the MODIFIED AAR in Loacal Maven after completion.
If I hit the triangle Run, The command to execute is app:assembleDebug, need to add uploadLocalMavenTask after assembleDebug, finalizedBy to run our task to synchronize the modified AAR
4.6. Provide AS status bar button, small rocket button, one spitfire and one no spitfire, representing enable/disable, and a clean RockectX cache.
5 a little surprise a day
5.1 Found that click the run button, execute the command is APP :assembleDebug, each sub-module in output did not package aar.
Solution: Gradle is executed by the bundle Flavor{Flavor}Flavor{BuildType}Aar task. Then you just need to find and inject the corresponding task of each module into app:assembleDebug and run it.
5.2. Repeated problems of multiple JAR packages were found after operation.
Implementation fileTree(dir: “libs”, include: [“*.jar”]) implementation fileTree(dir: “libs”, include: [“*.jar”]) It can be determined by the following code:
// There are two kinds of dependencies: No need to add to parent, // implementation RootProject. files("libs/xxx.jar")// implementation fileTree(dir: "libs", include: ["*.jar"])childDepency.files is DefaultConfigurableFileCollection || childDepency.files is DefaultConfigurableFileTreeCopy the code5.3 It is found that aar/ JAR has multiple dependency modes.
implementation (name: ‘libXXX’, ext: ‘aar’)
implementation files(“libXXX.aar”)
Solution: Using the first and the second to merge with the AAR leads to class duplication problems.
5.4 discover aar’s new posture dependencies.
configurations.maybeCreate("default")artifacts.add("default", file('lib-xx.aar'))
Copy the codeDefault Config is actually the holder of the final output AAR of the Module. Default Config can hold a list of AAR. So manually adding an AAR to default Config is equivalent to packaging the current Module.
Solution: by childProject. Configurations. MaybeCreate (” default “). Find all artifacts added aar, release localmaven separately.
5.5. Discover that Android Module can be packaged as jar.
Solution: Find the task named JAR and inject uploadLocalMaven Task after the JAR task.
5.6. Arouter has a bug, transform didn’t clear old cache via outputProvider.deleteall ().
The arouter problem has been resolved and the code has been merged. However, no new plugin version was released to mavenCentral, so we solved it for Arouter first.
Github.com/alibaba/ARo…