Android platform dynamic loading principle, the essence of Java or related knowledge to achieve. In the Java language, however, it is the bytecode generation and classloaders that allow developers to programmatically manipulate classes. This paper mainly focuses on the two aspects of technology, launched on the Android platform application analysis.
Read on to learn about Android plugins, hotfixes, modularity, AOP, Java class loading, and more.
Analysis of dynamic loading technology
First, Java basic knowledge
1. Analysis of vm class loading
The Java VM loads the Class data from the Class file to the memory, verifies, converts, parses, and initializes the data, and finally forms a Java type directly used by the VM, completing the Class loading mechanism.
The loading, linking, and initialization of the types are done during the program’s runtime, which costs performance at the initial loading, but it is this mechanism that gives Java a high degree of flexibility for dynamic extension. For example, you can write an interface-oriented application and decide the actual implementation class at runtime, which can be loaded locally or dynamically from the network.
The full life cycle of a class from being loaded into the virtual machine memory to being unloaded from the memory includes: Loading, Verification, Preparation, Resolution, Initialization, Using, and UnLoading stages Validation, preparation, and parsing are collectively called Linking. The loading stage is mainly analyzed here:
* ‘loading’ is a stage in the ‘class loading’ * process that does three main things: 1) Get the binary byte stream that defines the class through its fully qualified name. 2) Convert the static storage structure represented by the byte stream into the runtime data structure of the method area. 3) Generate the corresponding Class object of this Class in memory, as the method area of the various data access entry of this Class.
The virtual machine specifies that loading a class requires the ability to “get this binary stream by the fully qualified name of the class” at the loading stage, but does not specify where and how to get it. For example:
- Read from a ZIP file, such as JAR or dex.
- Network flow, such as Applet.
- Runtime computational generation, such as dynamic proxies.
- There are other file generation, database read and so on.
- .
In this way, developers can control how the byte stream is obtained through a custom ClassLoader or hook system ClassLoader.
In both explicit (class.forname) and implicit (e.g., new) loading methods, the loadClass method of the classloader is called to complete the actual loading of the Class.
Class loaders have a parent delegation model, which ensures the security of Java programs, but is not mandatory by the virtual machine. We can also customize the classloading mechanism to break this mechanism. Such as Android plug-in loading, hot repair and OSGI modular dynamic loading class loading technology application.
2. Aspect Oriented Programming (AOP)
Java object-oriented language, usually used is OOP (object-oriented programming), and then in a lot of technical research and development (such as Android dynamic loading, performance monitoring, logging, etc.) slowly come into contact with a new name ** “AOP” **, After Google, only to find that this is a very special, very powerful, very advanced practical programming way. Baidu Encyclopedia explains as follows:
AOP is an abbreviation of Aspect Oriented Programming, which means: Aspect Oriented Programming. It is a technique to achieve unified maintenance of program functions by means of precompilation and dynamic proxy at runtime. AOP is a continuation of OOP, a hot topic in software development, an important content in Spring framework, and a derivative paradigm of functional programming. Using AOP can isolate each part of the business logic, so that the coupling degree between each part of the business logic is reduced, the reusability of the program is improved, and the efficiency of development is improved.
In fact, AOP programming pattern has existed for many years, and there are many applications in Android project development, there are many excellent and popular tools such as APT, AspectJ, Javassist, etc., for our AOP development to provide convenient implementation.
Different libraries are used at different stages, and here is a diagram from the network to illustrate the differences among the three.
APT
We have all heard and used Annotation Processing Too (APT). The concept is easy to understand. It is mainly used at compile time, and it is also a popular and common technology.
After parsing the annotations at compile time, the code is combined with Square’s open source javapoet project to generate Java source code with custom logic. There are many open source libraries in use such as ButterKnife, EventBus, etc.
AspectJ
AspectJ supports compile-time and load-time code injection, and a special compiler is used to generate Class files that comply with the Java bytecode specification. More details can be found here.
Javassist
Javassist is an open source class library for analyzing, editing, and creating Java bytecode. Allow developers the freedom to add new methods to an already compiled class, or modify existing methods, allowing developers to ignore the details and structure of the modified class itself.
In RePlugin, a 360 open source plug-in project, Javassist technology was used to reduce Hook points to Android and to decouple development layer code logic.
Two, plug-in technology analysis
The Android project was developed early, and probably originally just to solve the problem of 65535 and package size, companies were looking at dynamic loading on the Android platform, which is now called plug-in solutions.
From 14 years dynamic-load-APk open source project, to now (17 years) RePlugin and VirtualAPK, analysis of the implementation principle, personal understanding, in fact, it is not very difficult, nothing more than using static proxy, Dynamic proxy, reflection, HOOK and ClassLoader (dex) merge, Or parent delegated destruction) and other related technologies to the sandbox implementation of the four components of Android.
Of course, the real difficulty lies in the adaptation of tens of millions of Android models and Rom, and the compatibility of complete Android functions, there is no denying the great contribution of the plug-in in the development of the Android industry (thank you for your open source bigshot 🙏).
There are so many good open source libraries out there, do we need to learn about them? The answer is no.
Personally think, learning plug-in is an Android development, contact system source code, solid Android knowledge, improve development skills a good way. It is recommended to study the first open source DroidPlugin project of 15 years ago. Although it is somewhat less complex and compatible than RePlugin and VirtualAPK, which are open source this year, the latter will actually have the same idea as the former. Are the pit thoughts; After startActivity, use certain technical means to replace targeActivity at a certain point, wait for AMS to call back, find a suitable point, change back to the original activity). Personally, the latter is an upgrade and optimization of the DroidPlugin from different angles.
Three, hot fix
In the middle stage of android technology development, there are many large apps that need to solve online problems urgently, so the industry began to study online hotfix technology.
In order to solve the 65535 problem, Google officially released Multidex solution, which uses the Application class loading and initialization process to dynamically load multiple dex when attachContext is used. The Intant Run quick code deployment solution (hot swap) was released in Android Studio 2.0.
This not only provides a class reference scheme for plug-in, but also provides a feasible scheme of Java layer for hot modification (this article will not expand the implementation analysis of Native layer). Representative projects such as Meituan’s thermal renewal program Robust.
Here is a simple analysis of the implementation principle of Google Intant Run
Reading the official documentation and source code, we can see that the official code quick section defines three concepts:
- Hot swap: Code changes are applied and projected onto the APP without restarting the APP or rebuilding the current activity. Scenario: Applies to most simple changes (including some method implementation changes, or variable value changes)
- Warm plug: The activity needs to be restarted to see the desired changes. Scenario: Typically, a code change involves a resource file, or RES.
- Cold plug: The app needs to be restarted (but still does not need to be reinstalled) Scenarios: any structural changes, such as inheritance rules, method signatures, etc.
Search Intant Run Server source code and you will find the following code update logic:
private int handlePatches(List<ApplicationPatch> changes,
boolean hasResources, int updateMode) {
if (hasResources) {
FileManager.startUpdate();
}
for (ApplicationPatch change : changes) {
String path = change.getPath();
if (path.endsWith(".dex")) { handleColdSwapPatch(change); // Cold swap Boolean canHotSwap =false;
for (ApplicationPatch c : changes) {
if (c.getPath().equals("classes.dex.3")) {
canHotSwap = true;
break; }}if (!canHotSwap) {
updateMode = 3;
}
} else if (path.equals("classes.dex.3")) { updateMode = handleHotSwapPatch(updateMode, change); // Hot plug}else if(isResourcePath(path)) { updateMode = handleResourcePatch(updateMode, change, path); }}if (hasResources) {
FileManager.finishUpdate(true);
}
return updateMode;
}
Copy the codeThe main analysis involves hot swap and cold swap of code updates. Analysis of the code process is mainly through Intant Run for us to generate Apk, and then follow the trail.
Let’s first look at what the case APK Application does, and analyze how the case APK launches our real App.
protected void attachBaseContext(Context context) {
if(! AppInfo.usingApkSplits) { String apkFile = context.getApplicationInfo().sourceDir; long apkModified = apkFile ! = null ? new File(apkFile) .lastModified() : 0L; createResources(apkModified); setupClassLoaders(context, context.getCacheDir().getPath(), apkModified); } createRealApplication();if(this.realApplication ! = null) { try { Method attachBaseContext = ContextWrapper.class .getDeclaredMethod("attachBaseContext",
new Class[] { Context.class });
attachBaseContext.setAccessible(true); attachBaseContext.invoke(this.realApplication, new Object[] { context }); } catch (Exception e) { } } } private void createResources(long apkModified) { File file = FileManager.getExternalResourceFile(); this.externalResourcePath = (file ! = null ? file.getPath() : null);if(file ! = null) { try { long resourceModified = file.lastModified();if ((apkModified == 0L) || (resourceModified <= apkModified)) {
if (Log.isLoggable("InstantRun", 2)) {
Log.v("InstantRun"."Ignoring resource file, older than APK");
}
this.externalResourcePath = null;
}
} catch (Throwable t) {
Log.e("InstantRun"."Failed to check patch timestamps", t);
}
}
}
private static void setupClassLoaders(Context context, String codeCacheDir,
long apkModified) {
List dexList = FileManager.getDexList(context, apkModified);
Class server = Server.class;
Class patcher = MonkeyPatcher.class;
if(! dexList.isEmpty()) { ClassLoader classLoader = BootstrapApplication.class .getClassLoader(); String nativeLibraryPath; try { nativeLibraryPath = (String) classLoader.getClass() .getMethod("getLdLibraryPath", new Class[0])
.invoke(classLoader, new Object[0]);
if (Log.isLoggable("InstantRun", 2)) {
Log.v("InstantRun"."Native library path: "+ nativeLibraryPath); } } catch (Throwable t) { nativeLibraryPath = FileManager.getNativeLibraryFolder() .getPath(); } IncrementalClassLoader.inject(classLoader, nativeLibraryPath, codeCacheDir, dexList); }}Copy the codeYou can see here that at the end there’s going to be an IncrementalClassLoader, and the current PathClassLoader delegate to IncrementalClassLoader to load dex. In code cold deployment scenarios, when the APK process restarts, IncrementalClassLoader will load the deployed change code first. Analyze the source code to see its structure diagram as follows:
The createRealApplication method creates the real Application and starts the real Dex in a similar way to the pluginization idea above, which is also a good example of a shell dynamically loading code.
private void createRealApplication() {
if(AppInfo.applicationClass ! = null) {if (Log.isLoggable("InstantRun", 2)) {
Log.v("InstantRun"."About to create real application of class name = "
+ AppInfo.applicationClass);
}
try {
Class realClass = (Class) Class
.forName(AppInfo.applicationClass);
if (Log.isLoggable("InstantRun", 2)) {
Log.v("InstantRun"."Created delegate app class successfully : "
+ realClass + " with class loader "
+ realClass.getClassLoader());
}
Constructor constructor = realClass
.getConstructor(new Class[0]);
this.realApplication = ((Application) constructor
.newInstance(new Object[0]));
if (Log.isLoggable("InstantRun", 2)) {
Log.v("InstantRun"."Created real app instance successfully :"+ this.realApplication); } } catch (Exception e) { throw new IllegalStateException(e); }}else{ this.realApplication = new Application(); }}Copy the codeAfter the actual Application is created, it is time to analyze how the hot update of the changed code takes effect.
public void onCreate() {
if(! AppInfo.usingApkSplits) { MonkeyPatcher.monkeyPatchApplication(this, this, this.realApplication, this.externalResourcePath); MonkeyPatcher.monkeyPatchExistingResources(this, this.externalResourcePath, null); }else{MonkeyPatcher. MonkeyPatchApplication (this, this, / / reflection ActivityThread replacement Application enclosing realApplication, null); }if(AppInfo.applicationId ! = null) { try { boolean foundPackage =false;
int pid = Process.myPid();
ActivityManager manager = (ActivityManager) getSystemService("activity");
List processes = manager
.getRunningAppProcesses();
boolean startServer = false;
if((processes ! = null) && (processes.size() > 1)) {for (ActivityManager.RunningAppProcessInfo processInfo : processes) {
if (AppInfo.applicationId
.equals(processInfo.processName)) {
foundPackage = true;
if (processInfo.pid == pid) {
startServer = true;
break; }}}if((! startServer) && (! foundPackage)) { startServer =true; }}else {
startServer = true;
}
if(startServer) { Server.create(AppInfo.applicationId, this); } } catch (Throwable t) { Server.create(AppInfo.applicationId, this); }}if(this.realApplication ! = null) { this.realApplication.onCreate(); }} public static void monkeyPatchApplication(Context Context, application bootstrap, Application realApplication, String externalResourceFile) { try { Class activityThread = Class .forName("android.app.ActivityThread");
Object currentActivityThread = getActivityThread(context,
activityThread);
Field mInitialApplication = activityThread
.getDeclaredField("mInitialApplication");
mInitialApplication.setAccessible(true);
Application initialApplication = (Application) mInitialApplication
.get(currentActivityThread);
if((realApplication ! = null) && (initialApplication == bootstrap)) { mInitialApplication.set(currentActivityThread, realApplication); }if(realApplication ! = null) { Field mAllApplications = activityThread .getDeclaredField("mAllApplications");
mAllApplications.setAccessible(true);
List allApplications = (List) mAllApplications
.get(currentActivityThread);
for (int i = 0; i < allApplications.size(); i++) {
if(allApplications.get(i) == bootstrap) { allApplications.set(i, realApplication); }}} Class loadedApkClass; try { loadedApkClass = Class.forName("android.app.LoadedApk");
} catch (ClassNotFoundException e) {
loadedApkClass = Class
.forName("android.app.ActivityThread$PackageInfo");
}
Field mApplication = loadedApkClass
.getDeclaredField("mApplication");
mApplication.setAccessible(true);
Field mResDir = loadedApkClass.getDeclaredField("mResDir");
mResDir.setAccessible(true);
Field mLoadedApk = null;
try {
mLoadedApk = Application.class.getDeclaredField("mLoadedApk");
} catch (NoSuchFieldException e) {
}
for (String fieldName : new String[] { "mPackages"."mResourcePackages" }) {
Field field = activityThread.getDeclaredField(fieldName);
field.setAccessible(true);
Object value = field.get(currentActivityThread);
for (Map.Entry> entry : ((Map>) value).entrySet()) {
Object loadedApk = ((WeakReference) entry.getValue()).get();
if(loadedApk ! = null) {if (mApplication.get(loadedApk) == bootstrap) {
if(realApplication ! = null) { mApplication.set(loadedApk, realApplication); }if(externalResourceFile ! = null) { mResDir.set(loadedApk, externalResourceFile); // Resource file replacement}if((realApplication ! = null) && (mLoadedApk ! = null)) { mLoadedApk.set(realApplication, loadedApk); } } } } } } catch (Throwable e) { throw new IllegalStateException(e); }}Copy the codeBefore calling realApplication, the program replaces all instances of Application in its own process, including:
- MInitialApplication for ActivityThread is realApplication
- All applications in mAllApplications are realApplications
- ActivityThread mPackages, the application of mLoaderApk mResourcePackages realApplication.
MonkeyPatchExistingResources method to substitute resource problems, a new AssetManager newAssetManager object, Then replace all the mAssets members of the current Resource and resource-. Theme with the newAssetManager object.
Here APK has completed the startup operation. Next, focus on analyzing the magic handleHotSwapPatch hot deployment. Without restarting the Activity or restarting the process, it can take effect immediately.
private int handleHotSwapPatch(int updateMode, ApplicationPatch patch) {
try {
String dexFile = FileManager.writeTempDexFile(patch.getBytes());
if (dexFile == null) {
returnupdateMode; //code update mode } String nativeLibraryPath = FileManager.getNativeLibraryFolder() .getPath(); DexClassLoader dexClassLoader = new DexClassLoader(dexFile, this.mApplication.getCacheDir().getPath(), nativeLibraryPath, getClass().getClassLoader()); // Load dex Class<? > aClass = Class.forName("com.android.tools.fd.runtime.AppPatchesLoaderImpl".true,
dexClassLoader);
try {
PatchesLoader loader = (PatchesLoader) aClass.newInstance();
String[] getPatchedClasses = (String[]) aClass
.getDeclaredMethod("getPatchedClasses", new Class[0])
.invoke(loader, new Object[0]);
if(! loader.load()) { updateMode = 3; } } catch (Exception e) { Log.e("InstantRun"."Couldn't apply code changes", e);
e.printStackTrace();
updateMode = 3;
}
} catch (Throwable e) {
Log.e("InstantRun"."Couldn't apply code changes", e);
updateMode = 3;
}
return updateMode;
}
Copy the codeThe handleHotSwapPatch function is to invoke the load method of the AppPatchesLoaderImpl class.
public boolean load() {
try {
for(String className : getPatchedClasses()) { ClassLoader cl = getClass().getClassLoader(); Class<? > aClass = cl.loadClass(className +"$override"); Object o = aClass.newInstance(); Class<? > originalClass = cl.loadClass(className); Field changeField = originalClass.getDeclaredField("$change");
changeField.setAccessible(true);
Object previous = changeField.get(null);
if(previous ! = null) { Field isObsolete = previous.getClass().getDeclaredField("$obsolete");
if(isObsolete ! = null) { isObsolete.set(null, Boolean.valueOf(true));
}
}
changeField.set(null, o);
if((Log.logging ! = null) && (Log.logging.isLoggable(Level.FINE))) { Log.logging.log(Level.FINE, String.format("patched %s",
new Object[] { className }));
}
}
} catch (Exception e) {
if(Log.logging ! = null) { Log.logging.log(Level.SEVERE, String.format("Exception while patching %s",
new Object[] { "foo.bar" }), e);
}
return false;
}
return true;
}
Copy the codeThe load method mainly loads the Patch class name +
Decompile the original class file, you will find that each class will have multiple *”IncrementalChange localIncrementalChange = $change”* members, and the following code will be added to each method to intercept patch execution logic.
public void onCreate(Bundle paramBundle) {
IncrementalChange localIncrementalChange = $change;
if (localIncrementalChange ! = null) {// The patch class will be used to assign the value and then follow the new code logiclocalIncrementalChange.access$dispatch(
"onCreate.(Landroid/os/Bundle;) V", new Object[] { this, paramBundle }); // Dispatch to the corresponding new method logic via the IncrementalChange interfacereturn; } super.onCreate(paramBundle); . }Copy the codeThis way, the new method logic in the new class can be invoked instantly without a restart. Interested students can further look at nuptBoyzhb students decompiled intant Run source.
Iv. Modularity
Android technology has developed to mature device, android customer degree project has developed to a certain scale, in order to facilitate the low coupling development of each business line, speed up the compilation speed, according to the module business upgrade and test and other aspects of the efficient, each have put forward modular solutions.
When it comes to modularity, there is also the concept of “componentization”, which is directly considered as a concept by some bloggers on the Internet. In my opinion, there are essential differences between the two.
According to my personal understanding, the target object of componentization is code. In order to understand the decoupling of functional module code and increase reuse efforts, component modules are extracted from code functional modules to form the “componentization” of the project. The target object of modularity is developers, and it is more based on the line of business to decouple the code calls between the lines of business, so that a line of business can be highly cohesive, and the work from development to online can be completed independently, without being affected by other module development. It can also be said that componentization is only a subset of the concept in a modular project.
In the process of modularization, the development goals need to be met, which is defined by the Ali Atlas open source project goals:
- During the project period, the independent development and debugging functions of the project can be realized, and the engineering module can be independent.
- At run time, complete component life cycle mapping, class isolation and other mechanisms are implemented.
- During the operation and maintenance period, it provides rapid incremental update and repair capabilities and rapid upgrades.
In the Java world, however, there is already the OSGI specification, which is a dynamic modularity specification based on the Java language. On Android modules, you can also customize the ClassLoader to dynamically load the module code by referring to OSGI’s decouple method.
Those who are interested can study Atlas, the Dynamic Bundle framework project of Alibaba’s mobile Taobao R&D team.
Five, the summary
This paper mainly from the macro perspective, from the Java dynamic loading basic knowledge, to the recent more popular “modular” for the basic concept and technical basis of the review, the purpose is to Java dynamic loading technology in the Android platform application and other related technology tree sorting and combing. I hope to have a macro understanding of the plug-in and modularized dynamic loading technology related to Android platform, and have some in-depth research in a certain direction.
Welcome to reprint, please indicate the source: Changxing E station Canking.win