1. Details of APT development component framework
In a previous article on the more advanced Android Startup task scheduling library, I introduced the development of componentized startup task scheduling tools to implement componentized scanning and annotation discovery using APT. However, the previous article did not go into the details of how APT is loaded into the specified generated class. There are some details here, so LET me elaborate.
1.1 How to Avoid generated class conflicts
First of all, When we use the annotation ISchedulerJob JobHunter generated implementation class is the realization of all classes will be unified into me. Shouheng. Startup. Hunter, the package name below (JobHunter’s implementation class using reflection load all the way ISchedulerJob instance). The problem then is how to avoid class conflicts when multiple components generate classes under the package. Because, while each Module is packaged as an AAR, there is no need to worry about class conflicts in the current AAR, if there are classes with the same class name under the same package in multiple AArs, the classes in other AArs will be overwritten (only one will be in effect).
One way to solve this problem is to use annotations in gradle files to indicate the current module name and then create classes with the module name suffix to avoid class collisions.
javaCompileOptions {
annotationProcessorOptions {
arguments = [STARTUP_MODULE_NAME: project.getName()]
}
}
Copy the code1.2 How to dynamically load all classes under the same APK package
Methods according to the above configuration, different module will be in the same package me. Shouheng. Startup. The hunter to generate different class below. So how do we find all the generated classes under the package name when we start the application? Here I refer to ARouter’s scheme, that is, to obtain the currently started APK at startup, parse the APK information, traverse the dex, and search for classes under the specified package name. The core code is as follows:
Extract APK, obtain dex path,
public static List<String> getSourcePaths(Context context) throws PackageManager.NameNotFoundException, IOException {
ApplicationInfo applicationInfo = context.getPackageManager().getApplicationInfo(context.getPackageName(), 0);
File sourceApk = new File(applicationInfo.sourceDir);
List<String> sourcePaths = new ArrayList<>();
sourcePaths.add(applicationInfo.sourceDir); //add the default apk path
//the prefix of extracted file, ie: test.classes
String extractedFilePrefix = sourceApk.getName() + EXTRACTED_NAME_EXT;
// If the VM already supports MultiDex, do not go to the Secondary Folder to load classesx.zip, it is not already there
// Whether there is multidex.version in sp is not accurate, because users who upgrade from a lower version include this sp configuration
if(! isVMMultidexCapable()) {//the total dex numbers
int totalDexNumber = getMultiDexPreferences(context).getInt(KEY_DEX_NUMBER, 1);
File dexDir = new File(applicationInfo.dataDir, SECONDARY_FOLDER_NAME);
for (int secondaryNumber = 2; secondaryNumber <= totalDexNumber; secondaryNumber++) {
//for each dex file, ie: test.classes2.zip, test.classes3.zip...
String fileName = extractedFilePrefix + secondaryNumber + EXTRACTED_SUFFIX;
File extractedFile = new File(dexDir, fileName);
if (extractedFile.isFile()) {
sourcePaths.add(extractedFile.getAbsolutePath());
//we ignore the verify zip part
} else {
throw new IOException("Missing extracted secondary dex file '" + extractedFile.getPath() + "'"); }}}return sourcePaths;
}
Copy the codeRead dex to get all of the following classes,
public static Set<String> getFileNameByPackageName(
Context context, final String packageName, Executor executor
) throws PackageManager.NameNotFoundException, IOException, InterruptedException {
final Set<String> classNames = new HashSet<>();
List<String> paths = getSourcePaths(context);
final CountDownLatch parserCtl = new CountDownLatch(paths.size());
for (final String path : paths) {
executor.execute(new Runnable() {
@Override
public void run(a) {
DexFile dexfile = null;
try {
if (path.endsWith(EXTRACTED_SUFFIX)) {
//NOT use new DexFile(path), because it will throw "permission error in /data/dalvik-cache"
dexfile = DexFile.loadDex(path, path + ".tmp".0);
} else {
dexfile = new DexFile(path);
}
// Iterate through all classes in dex
Enumeration<String> dexEntries = dexfile.entries();
while (dexEntries.hasMoreElements()) {
String className = dexEntries.nextElement();
if(className.startsWith(packageName)) { classNames.add(className); }}}catch (Throwable ignore) {
Log.e("AndroidStartup"."Scan map file in dex files made error.", ignore);
} finally {
if (null! = dexfile) {try {
dexfile.close();
} catch(Throwable ignore) { } } parserCtl.countDown(); }}}); } parserCtl.await();return classNames;
}
Copy the code2. Problems and solutions of the above schemes
2.1 Resolving a problem with APK loading classes
Following the above scenario solves the problem of getting classes from the same package name, but it causes other problems. After adopting the above scheme, there will be obvious pause effect when I start the APP. Although it is brief, the difference between before and after use can be felt by the naked eye.
The reason for this problem is that the above dynamic parsing and loading process takes some time, obviously the larger the package, the longer the parsing time, and the longer the pause time. This is clearly unacceptable to us. So are there other solutions to this problem? I can only think of two solutions,
Solution 1: Use a cache and then load from the cache after the first parsing;
Solution 2: pile insertion, fundamentally solve the problem, the problem is pile insertion is more complicated, affecting compilation;
2.2 Solution 1: Use SharedPreferences caching
The idea here is to read all the classes by parsing APK and then store them in SharedPreferences. However, it is stored according to whether the App is in debug mode and the version information of the App. That is, for the Release version, the stored classes correspond to the App version each time, and only need to be parsed again when the App version is updated. So, according to the SharedPreferences cache scheme, the first time to start a version of the application needs to parse, there will be a lag effect, followed by the cache.
private fun gatherByPackageScan(context: Context): List<JobHunter> {
val hunters = mutableListOf<JobHunter>()
val hunterImplClasses: Set<String>
if (debuggable || PackageUtils.isNewVersion(context, logger)) {
// Read all classes under the package name
hunterImplClasses = ClassUtils.getFileNameByPackageName(
context, "me.shouheng.startup.hunter", executor? :DefaultExecutor.INSTANCE)if (hunterImplClasses.isNotEmpty()) {
// Store the class name in SharedPreferences
context.getSharedPreferences(STARTUP_SP_CACHE_KEY, Context.MODE_PRIVATE)
.edit().putStringSet(STARTUP_SP_KEY_HUNTERS, hunterImplClasses).apply();
}
PackageUtils.updateVersion(context)
} else {
hunterImplClasses = HashSet(
context.getSharedPreferences(STARTUP_SP_CACHE_KEY, Context.MODE_PRIVATE)
.getStringSet(STARTUP_SP_KEY_HUNTERS, setOf())
)
}
hunterImplClasses.forEach {
val hunterImplClass = Class.forName(it)
hunters.add(hunterImplClass.newInstance() as JobHunter)
}
return hunters
}
Copy the code2.3 Solution 2: Dynamic scanning through ASM pile insertion
The idea of piling should be fairly clear, because the tarnsForm phase of Gradle compilation can dynamically scan classes, so we can easily achieve the goal of finding all classes under the specified package name. Once you find these classes, you just need to insert them into the specified class, and then insert the corresponding method according to the flag bits instead of parsing the APK as described above.
1. Idea of pile insertion
Calling scanAnnotations() takes the gatherHunters() logic first, and there is a registerByPlugin. Our piling logic is to insert addHunter() code into the gatherHunters() method. The addHunter() method reflects all instances of JobHunter’s implementation classes based on the class name passed in.
/** Scan annotations for job by [ISchedulerJob]. */
fun scanAnnotations(context: Context) {
try {
if (jobHunters == null) {
gatherHunters()
if(registerByPlugin) { logger? .i("Gathered hunters by startup-register plugin.");
} else{ jobHunters = gatherByPackageScan(context) } } jobHunters? .forEach { jobHunter ->valjobs = jobHunter.hunt() jobs? .let {this.jobs.addAll(it)
}
}
} catch (e: Exception) {
e.printStackTrace()
}
}
private fun gatherHunters(a) {
registerByPlugin = false
jobHunters = mutableListOf()
// addHunter()
}
private fun addHunter(className: String) {
registerByPlugin = true
if(! TextUtils.isEmpty(className)) {try {
val clazz = Class.forName(className)
val obj = clazz.getConstructor().newInstance()
if (obj is JobHunter) {
(jobHunters as? MutableList)? .add(obj) }else{ logger? .i("Register failed, class name: $className should implements one " +
"of me/shouheng/startup/JobHunter.")}}catch(e: java.lang.Exception) { logger? .e("Register class error: $className", e)
}
}
}
Copy the code2. Pile insertion
First, define a plug-in and register a Transform,
class RegisterPlugin : Plugin<Project> {
override fun apply(project: Project) {
val hasApp = project.plugins.hasPlugin(AppPlugin::class.java)
if (hasApp) {
Logger.make(project)
Logger.i("Project enable startup-register plugin")
val android = project.extensions.getByType(AppExtension::class.java)
val transform = RegisterTransform(project)
android.registerTransform(transform)
}
}
}
Copy the codeThen, the local directory and AAR are scanned in a custom Transform,
class RegisterTransform(private val project: Project): Transform() {
// ...
override fun transform(
context: Context? , inputs:MutableCollection<TransformInput>? , referencedInputs:MutableCollection<TransformInput>? , outputProvider:TransformOutputProvider? , isIncremental:Boolean
) {
Logger.i("Start scan register info in jar file.")
val startTime = System.currentTimeMillis()
if(! isIncremental) outputProvider? .deleteAll() inputs?.forEach { input -> scanJarInputs(input.jarInputs, outputProvider) scanDirectoryInputs(input.directoryInputs, outputProvider) } Logger.i("Scan finish, current cost time " + (System.currentTimeMillis() - startTime) + "ms")
insertImplClasses()
Logger.i("Generate code finish, current cost time: " + (System.currentTimeMillis() - startTime) + "ms")}}Copy the codeAnd use the custom ClassVisitor to get all JobHunter implementations,
class ScanClassVisitor(api: Int, cv: ClassVisitor): ClassVisitor(api, cv) {
override fun visit(version: Int, access: Int, name: String? , signature:String? , superName:String? , interfaces:Array<String>? {
super.visit(version, access, name, signature, superName, interfaces) interfaces? .any { it == JON_HUNTER_FULL_PATH }? .let {if(it && name ! =null && !hunterImplClasses.contains(name)) {
hunterImplClasses.add(name)
}
}
}
}
Copy the codeAfter reading all the classes, use the custom MethodVisitor to implement the above method insert and call,
class GeneratorMethodVisitor(
api: Int,
mv: MethodVisitor,
private val classes: List<String>
): MethodVisitor(api, mv) {
override fun visitInsn(opcode: Int) {
if ((opcode >= Opcodes.IRETURN && opcode <= Opcodes.RETURN)) {
classes.forEach {
val name = it.replace("/".".")
mv.visitVarInsn(Opcodes.ALOAD, 0)
mv.visitLdcInsn(name) // Class name
// generate invoke register method into AndroidStartupBuilder.gatherHunters()
mv.visitMethodInsn(Opcodes.INVOKESPECIAL, GENERATE_TO_CLASS_NAME,
ADD_HUNTER_METHOD_NAME, "(Ljava/lang/String;) V".false)}}super.visitInsn(opcode)
}
override fun visitMaxs(maxStack: Int, maxLocals: Int) {
super.visitMaxs(maxStack+4, maxLocals)
}
}
Copy the codeThat completes the code for piling. When using, we only need to add our custom plug-in to realize the custom scan in the component.
3. Troubleshooting method of ASM pile insertion
R8 failed to compile the code after piling because of compilation problems. R8 failed to execute the code after piling because of compilation problems. R8 failed to execute the code after piling.
Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete, position: Lme/shouheng/startup/AndroidStartupBuilder; gatherHunters()V, origin: /Users/wangshouheng/Desktop/repo/github/AndroidStartup/sample/app/build/intermediates/transforms/StartupRegister/debug/83.jar:me/shouheng/startup/AndroidStartupBuilder.class
... 36 more
Suppressed: java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.android.tools.r8.utils.Z: java.lang.ArrayIndexOutOfBoundsException: -1
at com.android.tools.r8.D8.d(D8.java:143)...38 more
Caused by: java.util.concurrent.ExecutionException: com.android.tools.r8.utils.Z: java.lang.ArrayIndexOutOfBoundsException: -1
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:552)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:513)
at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:86)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at com.android.tools.r8.utils.a1.a(SourceFile:11)
at com.android.tools.r8.utils.a1.a(SourceFile:40)
at com.android.tools.r8.utils.a1.a(SourceFile:38)
at com.android.tools.r8.utils.a1.a(SourceFile:37)
at com.android.tools.r8.ir.conversion.O.a(SourceFile:339)
at com.android.tools.r8.D8.d(D8.java:36)...38 more
Copy the codeAt this point, you can use the following two solutions:
Option 1: use JD-GUI to open the jar with the error, because the class parse error, so can not get the full stack. The plan doesn’t work!
Scheme 2: extract the jar, the compiled class files, command javap – c – p AndroidStartupBuilder. Class here – c required output parameters decompiled results, – p is decompiled content contains private fields and methods of the parameters. Follow the above instructions to output the decompilation results. We can find the true location of the error by reading the decompilation results. The plan works!
ASM plug-ins cannot read Kotlin bytecode or Kotlin bytecode compilation failure: Kotlin Code can be viewed by using the AS Tools->Kotlin->Show Kotlin Byte Code. For simple ASM staking code, you can also directly use the Kotlin bytecode form and manually translate it into ASM code.
3, summarize
If APT is not used in componentized application scenarios, there is no need to implement the above caching and piling logic, such as ButterKnife. You only need to call the corresponding method to complete automatic loading of APT related codes. However, in a componentized scenario, pegging and caching seem to be a necessary step in order to implement the scanning of classes under the specified package name. But to be exact, we don’t need to peg, just need the Transform phase to scan the class, store it, and load it dynamically when the application starts. So, what other options do we have besides staking? Of course there are, such as reading the class and writing it to a JSON file, and then reading the JSON file and parsing the generated class on startup. However, the performance of this method will be slightly less than that of piling.