Research and implementation of Android hot update technology

Research and implementation of Android hot update technology

—— Essential introduction ——

Recently time is really a little nervous! However, I still rushed to finish this article, there is no way, Yang Yang’s mysterious award is very tempting for me!

Use Kotlin to develop Android
Use Kotlin to develop Android projects -Kibo

Due to the length of this article, there may be mistakes, please forgive me.

The first part focuses on the current hot update scheme research, and the second part is to implement their own hot update framework.

The research and implementation of Android hot update technology

1. Food that goes into the stomach

2. Dreams in your heart

3. Read books into your mind

Without further ado, let’s first understand some concepts related to hot updates.

——— Concept explanation ——–

Hot update related concepts

• Componentization —- is to divide an app into multiple modules, and each Module is a component (Module). In the development process, we can make these components depend on each other or debug some components separately, but when the final release is to unify these components into an APK, which is componentization development. That’s pretty much what I’ve done before. For details, see the Android componentization scheme
• Pluginization — The whole APP is divided into many modules, including a host and multiple plug-ins, each module is an APK (each modular module is a lib), and the host APK and plug-in APK are separated or combined when packaged. In development, there are often many requirements piled into the project, beyond 65535, plug-in is a solution.

How can APP projects integrate seamlessly with plugins

Here’s a picture to help you understand:

• Hot update— When a class or plugin is updated with a smaller granularity, we call it a hot fix, usually to fix a bug!! Such as updating a bug method or making urgent changes to a lib package or even a class. In 2016, Google’s Android Studio launched Instant Run and put forward three terms.
  1. “Hot Deployment” – Simple changes within the method without restarting the app and Activity.
  2. “Warm Deployment” – The app does not need to restart, but the activity does, for example, modify resources.
  3. “Cold deployment” – App needs to be restarted, such as inheritance changes or method signature changes.
• Incremental updating, and hot update difference is one of the biggest, but the people should be well understood, the android application, some of the big games, in particular, several G is a dime a dozen, but each update is not going to download the latest version, just download a few megabytes of incremental package can complete the update, and this technology is used by the incremental updating. The implementation process looks something like this: we have a large application installed on our phone, and after downloading the incremental package, the APK and the incremental package on our phone are merged to form a new package, and then we install it again. The installation process may be visible, or the application itself has sufficient permissions to be installed directly in the background.

The Android Studio update today is an incremental update. The patch pack is only 51M, or more than 1gb if you download the new version.

And what exactly is hot update?

In some cases, when an App is released and a critical bug is discovered that needs to be urgently fixed, all sides of the company are busy repackaging the App, testing it, switching packages to various App markets and channels, prompting users to upgrade, downloading users, and overwriting the install. Sometimes just a single line of code needs to be changed and repackaged and redistributed at great cost. Releasing releases all the time will drive users crazy!! (Well, apes go crazy, too.)

This raises a question: is there a way to fix urgent bugs on the fly as a patch, without redistributing the App and requiring users to re-download and overwrite the installation?

This loading, which requires the replacement of new classes and resource files at runtime, can be considered a hot operation. Before hot updates, it was possible to dynamically load classes through reflective annotations, reflective calls, and reflective injection. The hot update framework is to solve such a problem.

In a sense, hot updating is about doing one thing, replacing. It’s modularity when you’re replacing something that’s a chunk of content, and when you’re replacing a method, it’s called hot update, and when you’re replacing a class, you’re heating up plug-ins, and again, all hot update solutions, in a sense, are hot plug-ins, because hot update solutions are doing this outside of your app. It’s as simple as that. Either replacing a class or a method is doing the same thing. This substitution, which counts as a hook operation, is an invasive operation at any code level.

Technology that changes the behavior of an app
Bug fixes for published apps

Ok, now that we know what hot updates are, let’s take a look at some of the mature schemes in use for hot updates.

This section describes the hot update solution

I will first talk about how to use several schemes, say the principle, and finally talk about how to achieve a hot update scheme of their own!

–Dexposed & AndFix & (HotFix)SopHix

Dexposed (Ali heat update program one)

“Dexposed”
“Xposed”
Xposed source code analysis – overview

Dexposed in the AOP principle from Xposed. In Dalvik VIRTUAL machine, it is mainly realized by changing the definition of a method object method in Dalvik Virtual machine, by changing the type of this method to Native and linking the implementation of this method to a general native Dispatch method. This Dispatch method implements AOP through JNI callbacks to a unified processing method on the Java side, and finally calls before, after functions in the unified processing method. In the Art virtual machine, it is also implemented by changing an ArtMethod entry function.

Unfortunately, android 4.4 and later versions have replaced Dalvik with Art, so to hook android 4.4 and later versions must be adapted to the mechanism of Art VIRTUAL machine. Currently, dexposed_L for Art is officially only beta, so it’s best not to use it in an official online product.

“Here”

1. A jar package named PatchLoader is introduced. This library implements a hot update framework, and the host APK (possibly buggy live version) packages this JAR package into THE APK when it is released.

2. The patch APK (bug-fixed online version) only needs this JAR package at compile time, but does not include this JAR package when packaged into APK, so as to avoid conflicts when the patch APK is integrated into the host APK;

3. – Patch APK will rely on dexposedbridge.jar and PatchLoader.jar as provided;

4. The patch APK is downloaded from the server by online downloading. The patch APK is integrated into the host APK, and the function in the patch APK is used to replace the original function, so as to realize the online bug repair function.

AndFix (Alige Update Solution 2)

AndFix is an online hotfix framework for Android apps. Using this framework, we were able to fix bugs in our App online without having to repeat the release. AndFix stands for “Android hot-fix.” Supports Android 2.3 to 6.0, and supports ARM and X86 devices. Perfect support for Dalvik and ART Runtime. The AndFix patch file is an.apatch file. It’s distributed from your server to your client to fix bugs in your App.

AndFix update implementation process (do not blame the ugliness of the picture do not blame the ﹏⊙) :

1. First add dependencies

   ` compile 'com. Alipay. Euler: andfix: 0.3.1 @ aar' `Copy the code

2. Then add the following code to application.onCreate ()

   `patchManager = new PatchManager(context); ` `patchManager.init(appversion); //current version` `patchManager.loadPatch(); `Copy the code

3. You can use this to get the AppVersion. Each appVersion change causes all patches to be deleted, and if the AppVersion does not change, all the saved patches will be loaded.

   `String appversion= getPackageManager().getPackageInfo(getPackageName(), 0).versionName; `Copy the code

4. Then call PatchManager’s addPatch method to load the new patch where necessary, such as after downloading the patch file.

5. After that, the patching process begins by generating an APK file, then changing the code to generate another APK after fixing bugs. Use the official tool apkpatch to generate apatch file in.apatch format. You need to provide the original apk, the repaired apk, and a signature file.

6. Upload the Apatch file to the phone via network transfer or ADB push, and the patch will be loaded when you run addPatch.

How AndFix works:

1. First load the PATCH file through the JarFile of the VM, and then read the patch. MF file to get the name of the PATCH class

2. Use DexFile to read the dex file in the patch file, obtain the patch method according to the annotation, then obtain thunder and method name according to the annotation, use classLoader to obtain the Class, and then obtain the bug method according to the reflection.

3. The jni layer fixes bugs by replacing properties of bug method objects with C++ Pointers.

PatchManager

public PatchManager(Context context) { mContext = context; mAndFixManager = new AndFixManager(mContext); // Initialize AndFixManager mPatchDir = new File(McOntext.getfilesdir (), DIR); MPatchs = new ConcurrentSkipListSet< patch >(); MLoaders = new ConcurrentHashMap<String, ClassLoader>(); // Initialize the class loader to store the corresponding class.Copy the code

mAndFixManager = new AndFixManager(mContext);

public AndFixManager(Context context) { mContext = context; mSupport = Compat.isSupport(); AndFix if (mSupport) {mSecurityChecker = new SecurityChecker(mContext); MOptDir = new File(McOntext.getfilesdir (), DIR); // Initialize the patch folder. mOptDir.exists() && ! mOptDir.mkdirs()) {// make directory fail mSupport = false; Log.e(TAG, "opt dir create error."); } else if (! mOptDir.isDirectory()) {// not directory mOptDir.delete(); // Delete mSupport = false; }}}......Copy the code

mPatchManager.init(appversion)
Init (String appVersion)

public void init(String appVersion) { if (! mPatchDir.exists() && ! mPatchDir.mkdirs()) {// make directory fail Log.e(TAG, "patch dir create error."); return; } else if (! mPatchDir.isDirectory()) {// not directory mPatchDir.delete(); return; } SharedPreferences sp = mContext.getSharedPreferences(SP_NAME, Context.MODE_PRIVATE); String ver = sp.getString(SP_VERSION, null); // Store information about patch files. if (ver == null || ! ver.equalsIgnoreCase(appVersion)) { cleanPatch(); Sp.edit ().putString(SP_VERSION, appVersion).commit(); // Delete patch file sp.edit().putString(SP_VERSION, appVersion).commit(); } else {initPatchs(); / / initializes the patch list, the local patch file is loaded into the memory}} / * * * * * * * * * * * * * omitted initialization, delete, load, specific methods to realize * * * * * * * * * * * * * * * * * /Copy the code

Init is used to save, delete, and load the patch file.

How is the patch file loaded? In fact, the patch file is essentially a JAR package, which can be read by using JarFile:

public Patch(File file) throws IOException { mFile = file; init(); } @SuppressWarnings("deprecation") private void init() throws IOException { JarFile jarFile = null; InputStream inputStream = null; try { jarFile = new JarFile(mFile); JarEntry Entry = jarfile.getJarentry (ENTRY_NAME); InputStream = jarfile.getinputStream (entry); inputStream = jarfile.getinputStream (entry); Manifest manifest = new Manifest(inputStream); Attributes main = manifest.getMainAttributes(); mName = main.getValue(PATCH_NAME); Patch. MF patch-name mTime = new Date(main.getValue(CREATED_TIME)); Created-time mClassesMap = new HashMap<String, List<String>>(); Attributes.Name attrName; String name; List<String> strings; for (Iterator<? > it = main.keySet().iterator(); it.hasNext();) { attrName = (Attributes.Name) it.next(); name = attrName.toString(); // check whether the suffix of name is -classes and add the value of name to the set. If (name.endswith (CLASSES)) {strings = arrays.asList (main.getValue(attrName).split(",")); if (name.equalsIgnoreCase(PATCH_CLASSES)) { mClassesMap.put(mName, strings); } else { mClassesMap.put( name.trim().substring(0, name.length() - 8),// remove // "-Classes" strings); } } } } finally { if (jarFile ! = null) { jarFile.close(); } if (inputStream ! = null) { inputStream.close(); }}}Copy the code

patchManager.loadPatch()

public void loadPatch() { mLoaders.put("*", mContext.getClassLoader()); // wildcard Set<String> patchNames; List<String> classes; for (Patch patch : mPatchs) { patchNames = patch.getPatchNames(); for (String patchName : patchNames) { classes = patch.getClasses(patchName); List mandFixManager.fix (patch.getfile (), McOntext.getclassloader (), classes); // Fix bug methods}}}Copy the code

mAndFixManager.fix(patch.getFile(), mContext.getClassLoader(),classes)

public synchronized void fix(File file, ClassLoader classLoader, List<String> classes) { if (! mSupport) { return; } // Check the signature of the patch file if (! mSecurityChecker.verifyApk(file)) {// security check fail return; } /****** omits the code ********/ // load the patch file dex final DexFile DexFile = dexfile. loadDex(file.getabsolutePath (), optfile.getAbsolutePath(), Context.MODE_PRIVATE); if (saveFingerprint) { mSecurityChecker.saveOptSig(optfile); } ClassLoader patchClassLoader = new ClassLoader(classLoader) { @Override protected Class<? > findClass(String className) throws ClassNotFoundException {// Override findClass <? > clazz = dexFile.loadClass(className, this); if (clazz == null && className.startsWith("com.alipay.euler.andfix")) { return Class.forName(className); // annotation's class // not found} if (clazz == null) {throw new ClassNotFoundException(className); } return clazz; }}; Enumeration<String> entrys = dexFile.entries(); Class<? > clazz = null; while (entrys.hasMoreElements()) { String entry = entrys.nextElement(); if (classes ! = null && ! classes.contains(entry)) { continue; // skip, not need fix } clazz = dexFile.loadClass(entry, patchClassLoader); // Get the bug class file if (clazz! = null) { fixClass(clazz, classLoader); // next code } } } catch (IOException e) { Log.e(TAG, "pacth", e); } } private void fixClass(Class<? > clazz, ClassLoader classLoader) { Method[] methods = clazz.getDeclaredMethods(); MethodReplace methodReplace; String clz; String meth; for (Method method : Methods) {// Get the annotation for this method, because the methods in the generated patch class for the buggy method are all annotated method.getannotation (methodreplace.class); if (methodReplace == null) continue; clz = methodReplace.clazz(); Meth = methodreplace.method (); meth = methodreplace.method (); // Get method in the annotation if (! isEmpty(clz) && ! isEmpty(meth)) { replaceMethod(classLoader, clz, meth, method); //next code } } } private void replaceMethod(ClassLoader classLoader, String clz, String meth, Method method) { try { String key = clz + "@" + classLoader.toString(); Class<? > clazz = mFixedClass.get(key); Fix if (clazz == null) {class not load class <? > clzz = classLoader.loadClass(clz); // initialize target class clazz = AndFix.initTargetClass(clzz); Class} if (clazz! = null) {// initialize class OK mFixedClass.put(key, clazz); Method src = clazz.getDeclaredMethod(meth, method.getParameterTypes()); // Get the method of the buggy class according to reflection (buggy apk) andfix.addreplacemethod (SRC, method); }} catch (Exception e) {log. e(TAG, "replaceMethod", e); } } public static void addReplaceMethod(Method src, Method dest) { try { replaceMethod(src, dest); // Call the native method initFields(dest.getDeclaringClass()); } catch (Throwable e) { Log.e(TAG, "addReplaceMethod", e); } } private static native void replaceMethod(Method dest, Method src);Copy the code

MethodReplace = @methodrePlace = @methodrePlace = @methodrePlace = @methodreplace = @methodreplace = @methodreplace

ReplaceMethod (Method dest,Method SRC) is a native Method.

extern void dalvik_replaceMethod(JNIEnv* env, jobject src, jobject dest); //Dalvik extern void art_replaceMethod(JNIEnv* env, jobject src, jobject dest); //ArtCopy the code

It can also be seen from the notes in the source code that since the Android 4.4 version is no longer using Dalvik VIRTUAL machine, but Art virtual machine, different processing needs to be done for different mobile phone systems.

First look at the implementation of Dalvik replacement method:

extern void __attribute__ ((visibility ("hidden"))) dalvik_replaceMethod( JNIEnv* env, jobject src, jobject dest) { jobject clazz = env->CallObjectMethod(dest, jClassMethod); ClassObject* clz = (ClassObject*) dvmDecodeIndirectRef_fnPtr( dvmThreadSelf_fnPtr(), clazz); clz->status = CLASS_INITIALIZED; Method* meth = (Method*) env->FromReflectedMethod(src); Method* target = (Method*) env->FromReflectedMethod(dest); LOGD("dalvikMethod: %s", meth->name); meth->jniArgInfo = 0x80000000; meth->accessFlags |= ACC_NATIVE; Int argsSize = dvmComputeMethodArgsSize_fnPtr(meth); // Set Method to Native int argsSize = dvmComputeMethodArgsSize_fnPtr(meth); if (! dvmIsStaticMethod(meth)) argsSize++; meth->registersSize = meth->insSize = argsSize; meth->insns = (void*) target; meth->nativeFunc = dalvik_dispatcher; // Replace method implementation with native method}Copy the code

Art replacement method implementation:

Extern void __attribute__ ((visibility ("hidden"))) art_replaceMethod(JNIEnv* env, jobject SRC, jobject dest) { if (apilevel > 22) { replace_6_0(env, src, dest); } else if (apilevel > 21) { replace_5_1(env, src, dest); } else { replace_5_0(env, src, dest); }} // Take 5.0 as an example:  void replace_5_0(JNIEnv* env, jobject src, jobject dest) { art::mirror::ArtMethod* smeth = (art::mirror::ArtMethod*) env->FromReflectedMethod(src); art::mirror::ArtMethod* dmeth = (art::mirror::ArtMethod*) env->FromReflectedMethod(dest); dmeth->declaring_class_->class_loader_ = smeth->declaring_class_->class_loader_; //for plugin classloader dmeth->declaring_class_->clinit_thread_id_ = smeth->declaring_class_->clinit_thread_id_; dmeth->declaring_class_->status_ = (void *)((int)smeth->declaring_class_->status_-1); // Put some parameters declaring_class_ = dmeth->declaring_class_; smeth->access_flags_ = dmeth->access_flags_; smeth->frame_size_in_bytes_ = dmeth->frame_size_in_bytes_; smeth->dex_cache_initialized_static_storage_ = dmeth->dex_cache_initialized_static_storage_; smeth->dex_cache_resolved_types_ = dmeth->dex_cache_resolved_types_; smeth->dex_cache_resolved_methods_ = dmeth->dex_cache_resolved_methods_; smeth->vmap_table_ = dmeth->vmap_table_; smeth->core_spill_mask_ = dmeth->core_spill_mask_; smeth->fp_spill_mask_ = dmeth->fp_spill_mask_; smeth->mapping_table_ = dmeth->mapping_table_; smeth->code_item_offset_ = dmeth->code_item_offset_; smeth->entry_point_from_compiled_code_ = dmeth->entry_point_from_compiled_code_; smeth->entry_point_from_interpreter_ = dmeth->entry_point_from_interpreter_; smeth->native_method_ = dmeth->native_method_; // replace smeth->method_index_ = dmeth->method_index_; smeth->method_dex_index_ = dmeth->method_dex_index_; LOGD("replace_5_0: %d , %d", smeth->entry_point_from_compiled_code_, dmeth->entry_point_from_compiled_code_); }Copy the code

In fact, this substitution can be viewed as a three-step process

1. Open the link library to get the operation handle, get the internal functions of the Native layer, and get the ClassObject object

2. Change the attribute of the access permission to public

3. Get a pointer to the old and new methods, and the new method points to the target method to achieve method replacement.

If we want to know exactly what methods have been replaced in the patch pack, we can just go to the patch file and see that all methods with @replacemethod annotations are basically methods that need to be replaced.

I’ve been learning C++ lately, and it’s clear how powerful a language that can control the underlying language is, but android can call C++ from JNI, so there’s nothing funny about that!

Ok, now we have analyzed the implementation process and principle of AndFix. Its advantage is that patches can be applied without restarting. Unfortunately, it still has many defects, which directly lead ali to abandon it again.

1. Not all method fixes are supported
   
   

2. Does not support YunOS

3. New classes and new fields cannot be added

4. You need to use the pre-hardened APK to make a patch, but the patch file is easy to decompile, that is, the modified class source code is easy to leak.

5. Using a hardened platform may invalidate the hot patch feature (see this issue in 360 hardened, I have not verified).

Sophix– Ali’s ultimate thermal fix

SopHix

Barba proved once again that I am the best, no one is better than me!! Because I support everything and have no flaws. It’s perfect!

Method level substitution

Start by creating an app:

*

    <meta-data
        android:name="com.taobao.android.hotfix.IDSECRET"
        android:value="24582808-1" />
    <meta-data
        android:name="com.taobao.android.hotfix.APPSECRET"
        android:value="da283640306b464ff68ce3b13e036a6e" />
    <meta-data
        android:name="com.taobao.android.hotfix.RSASECRET"
        android:value="MIIEvAIBA**********" />
Copy the code

Add maven repository address:

repositories {
    maven {
       url "http://maven.aliyun.com/nexus/content/repositories/releases"
    }
}
Copy the code

Add gradle coordinate version dependencies:

The compile 'com. Aliyun. Ams: alicloud - android - hotfix: 3.1.0'Copy the code

The project structure is also simple:

MainActivity

public class MainActivity extends AppCompatActivity { @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); ((TextView)findViewById(R.id.tv)).setText(String.valueOf(BuildConfig.VERSION_NAME)); findViewById(R.id.btn_click).setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Intent intent; intent = new Intent(MainActivity.this,SecondActivity.class); startActivity(intent); }}); }}Copy the code

Basically, there is a text box showing the current version and a button to jump to SecondActivity

SecondActivity

public class SecondActivity extends AppCompatActivity { @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_second); String s = null; findViewById(R.id.btn).setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Toast.makeText(secondActivity. this, "Popup error!" , Toast.LENGTH_SHORT).show(); }}); }}Copy the code

It is also very simple, only one button, after the button is clicked, a Toast will pop up.

In this way, our online app has been completed (it is a little rough), let’s take a look at the effect (please forgive me for my poor screen recording technology at the first time, I will do better and better in the future).

Then our users started using it and found a bug! “The popup is wrong!” , the user can no matter other, change good to me immediately!

At this time, the development of ER estimated that the heart of thousands of grass mud horse in the pentium, pray pray Buddha online do not have a problem, just online has a problem, “where is my test ER!!” The most violent method is to pop up “popup content is working!” in SecondActivity Toast. One line of code! Bingo!

Without a hot update, we might have had to make a temporary release or even release a new version, but now that we have The Sophix, we don’t need to bother.

First we went to download the patch packaging tool.

Old package: < Required > select the baseline package path (the problematic APK).

New package: < Required > Select a new package path (fixed the problem APK).

Log: The log output window is displayed.

Advanced: Expand advanced options

Settings: Configure other information.

GO! : The patch generation starts.

So first let’s add the old package and the new package, configure it and see what happens!

Strong cooling starts only after the patch is restarted.

Time depends, because the content of the project itself is relatively small, so the generation of patches is relatively fast, just wait a moment. If the project is large, it may take a longer time

So let’s see what’s actually generated? The patch generation directory is displayed

There is a pit here, I use my ZTE mobile phone to find that I have been getting the wrong package name when using the patch debugging tool, and then I borrowed someone else’s Huawei mobile phone to test and then IT is ok. I ended up recording it in a simulator.

Let’s download the debugging tool first to see the effect. First, connect the application (the pit is here, some mobile phones may prompt the package name error, but in fact there is no error. Although the official website provides a solution, it is still not solved, so we have to use the simulator).

Then there are two ways to load the patch package, one is to scan the QR code, and the other is to load the local patch JAR package, it is really difficult to operate on the simulator!! Finally I gave in and borrowed my classmate’s phone to scan the QR code and load the patch package… And then you get a log prompt

From the log prompt in the picture, we can see that the patch package was downloaded first, and then the patch was completed. We were asked to restart the APP, so we restarted. Of course, we should see the patch version 1.1 and Toast pop up normally.

Of course, at present, we still load the patch package on the debugging tool. After we release the patch package, we can directly open the app without the debugging tool to realize the patch, so as to complete the bug repair!

That looks really is very simple actually implements the hot fix, main we generate patch work is to provide tools to implement, ali actually we can tell, Sophix and AndFix like mentioned earlier, different places is the patch file has given tool can generate a key, and support more. Other updates such as the SO library and library and resource files can be seen in the official documentation.

The Sophix is primarily an update to the Alibaichuan HotFix. What is HotFix?

So Ali dad is always progressing, knowing there is a problem with the technology to solve the problem, this is not, from Dexposed–>AndFix >HotFix >Sophix, the technology is more and more mature.

Here are some hot update solutions for another large plant

Qzone Super patch & wechat Tinker Tencent hot update scheme

Babajia hot update technology has been developing, as the Internet giant Tencent how willing to lag behind, so is also hot pursuit of dry up!

Qzone super patch (Tencent hot update scheme 1) & DEX loading principle

DEX load



How does a class loader load a class

There are two types of loaders in Android: PathClassLoader and DexClassLoader.

public class DexClassLoader extends BaseDexClassLoader { public DexClassLoader(String dexPath, String optimizedDirectory, String libraryPath, ClassLoader parent) { super(dexPath, new File(optimizedDirectory), libraryPath, parent); } } public class PathClassLoader extends BaseDexClassLoader { public PathClassLoader(String dexPath, ClassLoader parent) { super(dexPath, null, null, parent); } public PathClassLoader(String dexPath, String libraryPath, ClassLoader parent) { super(dexPath, null, libraryPath, parent); }}Copy the code

The difference between the two is:

1. DexClassLoader: can load JAR/APK /dex. Can load uninstalled APK from SD card.

2. PathClassLoader: To pass in the Path of apK in the system, only apK files that have been installed can be loaded.

BaseDexClassLoader
PathClassLoader
optimizedDirectory
BaseDexClassLoader

public BaseDexClassLoader(String dexPath, File optimizedDirectory,
        String libraryPath, ClassLoader parent) {
    super(parent);
    this.originalPath = dexPath;
    this.pathList = new DexPathList(this, dexPath, libraryPath, optimizedDirectory);
}
Copy the code

DexPathList

Public DexPathList(ClassLoader definingContext, String dexPath, String libraryPath, File optimizedDirectory) {... this.dexElements = makeDexElements(splitDexPath(dexPath), optimizedDirectory); } private static Element[] makeDexElements(ArrayList<File> files, File optimizedDirectory) { ArrayList<Element> elements = new ArrayList<Element>(); for (File file : files) { ZipFile zip = null; DexFile dex = null; String name = file.getName(); if (name.endsWith(DEX_SUFFIX)) { dex = loadDexFile(file, optimizedDirectory); } else if (name.endsWith(APK_SUFFIX) || name.endsWith(JAR_SUFFIX) || name.endsWith(ZIP_SUFFIX)) { zip = new ZipFile(file); }... if ((zip ! = null) || (dex ! = null)) { elements.add(new Element(file, zip, dex)); } } return elements.toArray(new Element[elements.size()]); } private static DexFile loadDexFile(File file, File optimizedDirectory) throws IOException { if (optimizedDirectory == null) { return new DexFile(file); } else { String optimizedPath = optimizedPathFor(file, optimizedDirectory); return DexFile.loadDex(file.getPath(), optimizedPath, 0); } } //** //* Converts a dex/jar file path and an output directory to an //* output file path for an associated optimized  dex file. // private static String optimizedPathFor(File path, File optimizedDirectory) { String fileName = path.getName(); if (! fileName.endsWith(DEX_SUFFIX)) { int lastDot = fileName.lastIndexOf("."); if (lastDot < 0) { fileName += DEX_SUFFIX; } else { StringBuilder sb = new StringBuilder(lastDot + 4); sb.append(fileName, 0, lastDot); sb.append(DEX_SUFFIX); fileName = sb.toString(); } } File result = new File(optimizedDirectory, fileName); return result.getPath(); }Copy the code

OptimizedDirectory is used to cache the dex file we need to load and create a DexFile object. If it is null, the DexFile object will be created using the original path of the dex file.

OptimizedDirectory must be an internal storage path, regardless of the type of dynamic loading, the loaded executable must be stored in the internal storage. DexClassLoader can specify its own optimizedDirectory, so it can load the external dex, because this dex will be copied to the optimizedDirectory of the internal path; PathClassLoader does not have optimizedDirectory, so it can only load internal dex, which is mostly in the apK already installed in the system.

This is just an instance of the class loader, where a DexFile instance is created to hold the dex file, which we assume is used to load the class.

In Android, the ClassLoader uses the loadClass method to load the required classes

public Class<? > loadClass(String className) throws ClassNotFoundException { return loadClass(className, false); } protected Class<? > loadClass(String className, boolean resolve) throws ClassNotFoundException { Class<? > clazz = findLoadedClass(className); if (clazz == null) { ClassNotFoundException suppressed = null; try { clazz = parent.loadClass(className, false); } catch (ClassNotFoundException e) { suppressed = e; } if (clazz == null) { try { clazz = findClass(className); } catch (ClassNotFoundException e) { e.addSuppressed(suppressed); throw e; } } } return clazz; }Copy the code

The loadClass method calls the findClass method, and BaseDexClassLoader overloads the findClass method

@Override protected Class<? > findClass(String name) throws ClassNotFoundException { Class clazz = pathList.findClass(name); if (clazz == null) { throw new ClassNotFoundException(name); } return clazz; }Copy the code

The result is a call to DexPathList’s findClass

public Class findClass(String name) { for (Element element : dexElements) { DexFile dex = element.dexFile; if (dex ! = null) { Class clazz = dex.loadClassBinaryName(name, definingContext); if (clazz ! = null) { return clazz; } } } return null; }Copy the code

So I’m going to iterate through all the DexFile instances, which means I’m going to iterate through all the dex files that I’ve loaded, and then I’m going to call loadClassBinaryName to see if I can load the class THAT I want.

public Class loadClassBinaryName(String name, ClassLoader loader) {
    return defineClass(name, loader, mCookie);
}
private native static Class defineClass(String name, ClassLoader loader, int cookie);
Copy the code

A classloader can contain multiple dex, where the objects in this set are all dex files, and then call to traverse all dex from scratch. If the desired class is found in dex, In other words, if there are multiple dex and the class to be found is found in the previous dex, it will not continue to search for the class in other dex.

dex.loadClassBinaryName(name, definingContext)

If you want to load a class, you call the findClass method of the ClassLoader, find the class in the dex, and load it into memory

This gives you a general idea of how super patches work, and what super patches do is make the class loader only find the classes that we’ve fixed!

In layman’s terms, we want to change the contents of dexElements, which is a member variable in the PathClassLoader class, by adding a dex and placing it first.

Because Qzone super patch is not open source, here is just to tell you the implementation principle of class loading mechanism, the specific implementation process should be like this (figure is probably the most intuitive) :

Obtain the application PathdexClassloader > PathList > DexElements by reflection, and obtain the DexClassloader > PathList > DexElements of the dex patch. Then, two DexElements are combined through combinArray method, patch DexElements are put in the front, and then combined DexElements are used as DexElements in PathdexClassloader. In this way, the patch dex can be loaded preferentially during loading, from which our patch class can be loaded, which can basically ensure stability and compatibility

Advantage:

1. Instead of synthesizing the whole package (compared to wechat Tinker, next topic), the product is smaller and more flexible

2. Can achieve class replacement, high compatibility. (Some phones don’t work)

Inadequate:

1. It does not take effect immediately and must be restarted to take effect.

2. In order to fix this process, two Dex must be added to the application! There is only one class in Dalvikhack. dex, which has little impact on performance. However, for patch.dex, it takes a lot of time to load when a certain number of classes are repaired.

3. In ART mode, if a class changes its structure, it will be out of memory. To solve this problem, it is necessary to load all relevant calling classes, parent classes, and subclasses into patch.dex, which leads to the abnormal size of the patch package and further increases the time consuming when the application is started and loaded.

Wechat Tinker (Tencent Hot Update Scheme II)

For wechat, hot updates are implemented using a “highly available” patch framework that meets at least the following conditions:

1. Stability and compatibility; Wechat needs to run on hundreds of millions of devices, and even a 1 percent exception from the patch framework will affect tens of thousands of users. Ensuring the stability and compatibility of the patch framework is our top priority;

2. Performance; Wechat also has very strict performance requirements. First of all, the patch framework cannot affect the performance of applications, based on the fact that users will not use patches in most cases. Secondly, patch packages should be as few as possible, which is related to user traffic and patch success rate.

3. Ease of use. Once these two core issues are resolved, we want the patch framework to be easy to use and fully supported, even at the feature release level.

Have a look at the Qzone super patch above (mainly to explain the loading principle of dex, but this is also important for the analysis of Tinker). Is there a way to develop transparently without the pitfalls of the QZone solution? There must be. For example, we can use the new Dex completely, so that there will not be the problem of Art address confusion, and there is no need to insert stakes in Dalvik. Of course, given the size of the patch pack, we couldn’t just put the new Dex in it. However, we can put the differences between the old and new Dex into the patch package, and at the simplest, we can adopt the BsDiff algorithm.

Tinker is wechat’s official Hot patch solution for Android. It supports dynamic delivery of code, So library and resources, So that apps can be updated without the need for reinstallation.

Tinker is more like incremental update of APP. Through the difference algorithm on the server side, the difference package between the old and new dex is calculated and pushed to the client side for synthesis. The traditional difference algorithm is BsDiff, but Tinker’s genius is that it developed its own DexDiff algorithm based on the Dex file format, which we’ll talk about later.

If our app wants to integrate Tinker hot updates, we can directly create our own app on Tencent’s Bugly and access it. I’m creating an application here, but I’m using the official example directly for integration. Because the official integration steps are very detailed, and the corresponding set of tutorials, we should be very convenient to use. Let’s try this first:

First create an app, get the AppID and AppKey, then download BuglyHotfixEasyDemo on GitHub (I won’t create a new project here, which feels unnecessary) with the following directory structure:

BugClass is a class with an error:

Public class BugClass {public String bug() {// This code will report null pointer exception // String STR = null; // Log.e("BugClass", "get string length:" + str.length()); return "This is a bug class"; }}Copy the code

LoadBugClass is to get the string returned from BugClass

** @return Returns the bug String */ public static String getBugString() {BugClass bugClass = new BugClass(); return bugClass.bug(); }}Copy the code

On the other hand, MainActivity has many buttons, one of which is toggle type. Click on the button to pop up Toast, and the displayed content is the string returned above.

/ * * * * * * * * * * to omit N lines of code * * * * * * * * * * * * * * * * / / according to the application before the patch package later to test whether the patch package success. * * application before the patch package, prompt "This is a bug class" * patch application package, Prompt "The bug has fixed" * / public void testToast () {Toast. MakeText (this, LoadBugClass getBugString (), Toast.LENGTH_SHORT).show(); } @override public void onClick(View v) {switch (v.getid ()) {case r.i.btnShowtoast: break; /*********** again omit N lines of code ************/Copy the code

From the project structure is also very simple that an example, multi-channel packaging we will not try, to a simple basic packaging implementation!

Display effect (after clicking the display effect button, there is still a bug package, so the display is bug class) :

Tinker hot update integration implementation

1. Compile the benchmark package

1. Configure the tinkerId of the base package

Gradle = tinker-support.gradle = tinker-support.gradle = tinker-support.gradle

The tinkerId is ideally a unique identifier, such as git version number, versionName, and so on. If you want to test hot updates, you need to report the baseline version online.

It is emphasized here that the baseline version is configured with a unique tinkerId, and the premise for this baseline version to apply patches is to integrate the overheated update SDK and start reporting to the Internet, so that we will correspond this tinkerId to a target version in the background. For example, tinkerId = “bugly_1.0.0” corresponds to a target version of 1.0.0, and patches based on this version will match the target version.

1. Build base package (original package with bugs)

Run assembleRelease to compile the base package. The following files are generated in the build/baseApk directory. You can configure the path and file name

We can see tinkerId.

2. Bug fixes for the baseline version

BugClass returns “The bug has fixed”;

3. Build patch packs based on baseline releases

Modify the apK path, mapping file path, and resId file path to be repaired

*/ def baseApkDir = "app-0813-20-54-50" tinkerId = "1.0.1-patch"Copy the code

Executing the task that builds the fix pack produces a complete APK for bug fixes

If you want to build patches for different build environments, just execute the task generated by the TinkerSupport plugin, such as buildTinkerPatchRelease, to build patches for release build environments. Note: If the TinkerSupport plugin version is earlier than 1.0.4, you need to use tinkerPatchRelease to generate the patch package.

Outputs /patch in build/outputs/patch directory:

Three files are generated: unSignedApk, signedApk, and signedWith7ZipApk.

UnSignedApk simply compresses the files in tinker_result into a compressed package.

SignedApk will sign unSignedApk using Jarsigner.

SignedWith7ZipApk is mainly used to decompress signedApk and then do sevenZip compression.

4. Upload patch packages to the platform

Time for a miracle!! Upload patches to the platform and issued to edit the rules, click publish new patches, upload the generated patch in front of the bag, the platform will automatically be matched to the target version for you, can choose distributed range (development equipment, full amount, custom), fill in the complete note, click issued to patch to take effect immediately, so that you can receive our strategy, in the middle of the client The SDK will automatically install the patch packages locally for you.

The client received the policy and needed to download the patch update.

Ok, now we have a brief look at the effect of Bugly hot update, which applies wechat’s Tinker scheme. In fact, it is not difficult to see that Bugly and Ali’s Sophix are both a strategy to deliver patch packages.

There are two major schools of hot update technology, one is the Native genre of Ali, namely AndFix and Sophix, and the other is the Java genre of Tencent’s Qzone super patch. Finally, wechat chose to continue to follow its own Java genre (its own path is to go black!). , but wechat is not rigid conventions, but the pursuit of perfection! This has to mention the DexDiff algorithm mentioned above:

DexDiff algorithm:

We mentioned the loading process of dex above. We all know that dex file is a bytecode file running in Dalvik, similar to the class file running in JVM. When decompiling, APK will contain one or more *. Dex files, which store the code we wrote. In general, we will also use tools to convert to JARS, and then decompile some tools to view them (dex2jar).

Jar files we should all know, similar to the class file compression package, under normal circumstances, we can see a class file directly unzip. Dex file we can not extract the internal class file, that must be because of its format, we will not analyze the specific format here, let’s take a look at the basic steps of DexDiff (details of the source code will be covered) :

1. First of all, there are such things as bugdex, bugfixeddex and Patchdex.

2. Second, figure out how much each part of the BugFixeddex (referring to a specific part of the DEX structure) occupies;

3. Then, compare each part of the Bugdex and BugFixeddex, compare each part, and note the differences (what was removed, what was added, and in what form it was recorded and stored).

4. Finally, the different saved records are written to the patch

According to the above, hot update of Dex in Tinker is mainly divided into three parts: 1. Generation of patch package; 2. 2. Dex after patch package delivery; Iii. Loading process after generating full Dex.

“Tinker source transfer”

I downloaded the latest version, version 1.8.1. Source code we pick focus to see, mainly to find the above said three parts:

1. Generation of patch package;

We called buildTinkerPatchRelease in Tinker-support to build the patch

tinkerPatch()

 protected void tinkerPatch() {
    Logger.d("-----------------------Tinker patch begin-----------------------");

    Logger.d(config.toString());
    try {
        //gen patch
        ApkDecoder decoder = new ApkDecoder(config);
        decoder.onAllPatchesStart();
        decoder.patch(config.mOldApkFile, config.mNewApkFile);
        decoder.onAllPatchesEnd();

        //gen meta file and version file
        PatchInfo info = new PatchInfo(config);
        info.gen();

        //build patch
        PatchBuilder builder = new PatchBuilder(config);
        builder.buildPatch();

    } catch (Throwable e) {
        e.printStackTrace();
        goToError();
    }

    Logger.d("Tinker patch done, total time cost: %fs", diffTimeFromBegin());
    Logger.d("Tinker patch done, you can go to file to find the output %s", config.mOutFolder);
    Logger.d("-----------------------Tinker patch end-------------------------");
}
Copy the code

This is actually the process of generating patch, invoke com. Tencent. Tinker. Build. The decoder. ApkDecoder in patch (File oldFile, File newFile) methods:

 public boolean patch(File oldFile, File newFile) throws Exception {
    writeToLogFile(oldFile, newFile);
    //check manifest change first
    manifestDecoder.patch(oldFile, newFile);

    unzipApkFiles(oldFile, newFile);

    Files.walkFileTree(mNewApkDir.toPath(), new ApkFilesVisitor(config, mNewApkDir.toPath(), mOldApkDir.toPath(), dexPatchDecoder, soPatchDecoder, resPatchDecoder));

    //get all duplicate resource file
    for (File duplicateRes : resDuplicateFiles) {
    //            resPatchDecoder.patch(duplicateRes, null);
        Logger.e("Warning: res file %s is also match at dex or library pattern, " + "we treat it as unchanged in the new resource_out.zip", getRelativePathStringToOldFile(duplicateRes));
    }

    soPatchDecoder.onAllPatchesEnd();
    dexPatchDecoder.onAllPatchesEnd();
    manifestDecoder.onAllPatchesEnd();
    resPatchDecoder.onAllPatchesEnd();

    //clean resources
    dexPatchDecoder.clean();
    soPatchDecoder.clean();
    resPatchDecoder.clean();
    return true;
}
Copy the code

From the source we can see that the first is to detect the manifest file, to see if there is any change, if the manifest component is found to be new, then throw an exception, because Tinker currently does not support the addition of four components.

After passing the detection, decompress the APK file, traverse the old and new APK, and hand it to the ApkFilesVisitor for processing.

In the visitFile method of ApkFilesVisitor, for dex type file, call dexDecoder for patch operation; We mainly analyze dexDecoder, so omit the operation codes of so type and RES type:

@Override public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException { Path relativePath = newApkPath.relativize(file); Path oldPath = oldApkPath.resolve(relativePath); File oldFile = null; //is a new file? ! if (oldPath.toFile().exists()) { oldFile = oldPath.toFile(); } String patternKey = relativePath.toString().replace("\\", "/"); if (Utils.checkFileInPattern(config.mDexFilePattern, patternKey)) { //also treat duplicate file as unchanged if (Utils.checkFileInPattern(config.mResFilePattern, patternKey) && oldFile ! = null) { resDuplicateFiles.add(oldFile); } try { dexDecoder.patch(oldFile, file.toFile()); } catch (Exception e) { // e.printStackTrace(); throw new RuntimeException(e); } return FileVisitResult.CONTINUE; } if (Utils.checkFileInPattern(config.mSoFilePattern, PatternKey)) {// Also treat duplicate file as unchanged /***** **************/} if (utils.checkFileinpattern (config.mresFilepattern, patternKey)) {/***** omit so parse, For Res files, use resDecoder to operate patch **************/} return filevisitResult.continue; }Copy the code

Patch (final File oldFile, final File newFile);

@SuppressWarnings("NewApi") @Override public boolean patch(final File oldFile, final File newFile) throws IOException, TinkerPatchException { final String dexName = getRelativeDexName(oldFile, newFile); / > > > > > > > > > > > > > > > > > > > > > > omit N line < < < < < < < < < < < < < < < < < < < < < < / try { excludedClassModifiedChecker.checkIfExcludedClassWasModifiedInNewDex(oldFile, newFile); } / > > > > > > > > > > > > > > > > > > > > > > omit N line < < < < < < < < < < < < < < < < < < < < < < / / / If corresponding new dex was completely does, just return false. // don't process 0 length dex if (newFile == null || ! newFile.exists() || newFile.length() == 0) { return false; } File dexDiffOut = getOutputPath(newFile).toFile(); final String newMd5 = getRawOrWrappedDexMD5(newFile); //new add file if (oldFile == null || ! oldFile.exists() || oldFile.length() == 0) { hasDexChanged = true; copyNewDexAndLogToDexMeta(newFile, newMd5, dexDiffOut); return true; } / > > > > > > > > > > > > > > > > > > > > > > omit N line < < < < < < < < < < < < < < < < < < < < < < / RelatedInfo RelatedInfo = new RelatedInfo (); relatedInfo.oldMd5 = oldMd5; relatedInfo.newMd5 = newMd5; // collect current old dex file and corresponding new dex file for further processing. oldAndNewDexFilePairList.add(new AbstractMap.SimpleEntry<>(oldFile, newFile)); dexNameToRelatedInfoMap.put(dexName, relatedInfo); return true; }Copy the code

It can be seen from the source code that the input dex file is checked first to see whether any classes that are not allowed to be modified are modified. For example, loader-related classes are not allowed to be modified. In this case, an exception will be thrown.

If the dex is new, the dex is directly copied to the result file.

If the dex is modified, the added and deleted classes are collected. OldAndNewDexFilePairList saves the mapping between the old and new dex for later analysis.

Simply add the new dex file to the addedDexFiles. Call the UniqueDexDiffDecoder. Patch:

@Override
public boolean patch(File oldFile, File newFile) throws IOException, TinkerPatchException {
    boolean added = super.patch(oldFile, newFile);
    if (added) {
        String name = newFile.getName();
        if (addedDexFiles.contains(name)) {
            throw new TinkerPatchException("illegal dex name, dex name should be unique, dex:" + name);
        } else {
            addedDexFiles.add(name);
        }
    }
    return added;
}
Copy the code

After the patch is completed, generatePatchInfoFile is called to generate the patch file. First traversal oldAndNewDexFilePairList DexFiffDecoder. GeneratePatchInfoFile, to take out the old and new files.

If the MD5 values of the old and new files are equal or not, it indicates that there is a change. The DexPatchGenerator will be created based on the old and new files. The DexPatchGenerator constructor contains the comparison algorithm of 15 Dex regions:

private DexSectionDiffAlgorithm<StringData> stringDataSectionDiffAlg;
private DexSectionDiffAlgorithm<Integer> typeIdSectionDiffAlg;
private DexSectionDiffAlgorithm<ProtoId> protoIdSectionDiffAlg;
private DexSectionDiffAlgorithm<FieldId> fieldIdSectionDiffAlg;
private DexSectionDiffAlgorithm<MethodId> methodIdSectionDiffAlg;
private DexSectionDiffAlgorithm<ClassDef> classDefSectionDiffAlg;
private DexSectionDiffAlgorithm<TypeList> typeListSectionDiffAlg;
private DexSectionDiffAlgorithm<AnnotationSetRefList> annotationSetRefListSectionDiffAlg;
private DexSectionDiffAlgorithm<AnnotationSet> annotationSetSectionDiffAlg;
private DexSectionDiffAlgorithm<ClassData> classDataSectionDiffAlg;
private DexSectionDiffAlgorithm<Code> codeSectionDiffAlg;
private DexSectionDiffAlgorithm<DebugInfoItem> debugInfoSectionDiffAlg;
private DexSectionDiffAlgorithm<Annotation> annotationSectionDiffAlg;
private DexSectionDiffAlgorithm<EncodedValue> encodedArraySectionDiffAlg;
private DexSectionDiffAlgorithm<AnnotationsDirectory> annotationsDirectorySectionDiffAlg;
Copy the code

DexDiffDecoder. ExecuteAndSaveTo (OutputStream out) this function will be 15 algorithm based on the above comparison of different areas of dex each algorithm for each area, The purpose of the algorithm is just as we described DexDiff step 3 earlier, to know “what is removed, what is added”, and finally to generate differences in dex files.

This is the heart of the whole Dex Diff algorithm. StringDataSectionDiffAlgorithm, for example, the algorithm process is as follows:

Each algorithm executes the execute and simulatePatchOperation methods:

/ * * * * * * * * * * * * to omit N lines of code * * * * * * * * * * * * * / enclosing stringDataSectionDiffAlg. The execute (); this.patchedStringDataItemsOffset = patchedheaderSize + patchedIdSectionSize; if (this.oldDex.getTableOfContents().stringDatas.isElementFourByteAligned) { this.patchedStringDataItemsOffset = SizeOf.roundToTimesOfFour(this.patchedStringDataItemsOffset); } this.stringDataSectionDiffAlg.simulatePatchOperation(this.patchedStringDataItemsOffset); /************ omit N lines of code *************/Copy the code

So let’s start with execute. You can go to see com source. Tencent. Tinker. Build. Dexpatcher. Algorithms. The diff. DexSectionDiffAlgorithm)

public void execute() { this.patchOperationList.clear(); this.adjustedOldIndexedItemsWithOrigOrder = collectSectionItems(this.oldDex, true); this.oldItemCount = this.adjustedOldIndexedItemsWithOrigOrder.length; AbstractMap.SimpleEntry<Integer, T>[] adjustedOldIndexedItems = new AbstractMap.SimpleEntry[this.oldItemCount]; System.arraycopy(this.adjustedOldIndexedItemsWithOrigOrder, 0, adjustedOldIndexedItems, 0, this.oldItemCount); Arrays.sort(adjustedOldIndexedItems, this.comparatorForItemDiff); AbstractMap.SimpleEntry<Integer, T>[] adjustedNewIndexedItems = collectSectionItems(this.newDex, false); this.newItemCount = adjustedNewIndexedItems.length; Arrays.sort(adjustedNewIndexedItems, this.comparatorForItemDiff); int oldCursor = 0; int newCursor = 0; while (oldCursor < this.oldItemCount || newCursor < this.newItemCount) { if (oldCursor >= this.oldItemCount) { // rest item are all newItem. while (newCursor < this.newItemCount) { AbstractMap.SimpleEntry<Integer, T> newIndexedItem = adjustedNewIndexedItems[newCursor++]; this.patchOperationList.add(new PatchOperation<>(PatchOperation.OP_ADD, newIndexedItem.getKey(), newIndexedItem.getValue())); } } else if (newCursor >= newItemCount) { // rest item are all oldItem. while (oldCursor < oldItemCount) { AbstractMap.SimpleEntry<Integer, T> oldIndexedItem = adjustedOldIndexedItems[oldCursor++]; int deletedIndex = oldIndexedItem.getKey(); int deletedOffset = getItemOffsetOrIndex(deletedIndex, oldIndexedItem.getValue()); this.patchOperationList.add(new PatchOperation<T>(PatchOperation.OP_DEL, deletedIndex)); markDeletedIndexOrOffset(this.oldToPatchedIndexMap, deletedIndex, deletedOffset); } } else { AbstractMap.SimpleEntry<Integer, T> oldIndexedItem = adjustedOldIndexedItems[oldCursor]; AbstractMap.SimpleEntry<Integer, T> newIndexedItem = adjustedNewIndexedItems[newCursor]; int cmpRes = oldIndexedItem.getValue().compareTo(newIndexedItem.getValue()); if (cmpRes < 0) { int deletedIndex = oldIndexedItem.getKey(); int deletedOffset = getItemOffsetOrIndex(deletedIndex, oldIndexedItem.getValue()); this.patchOperationList.add(new PatchOperation<T>(PatchOperation.OP_DEL, deletedIndex)); markDeletedIndexOrOffset(this.oldToPatchedIndexMap, deletedIndex, deletedOffset); ++oldCursor; } else if (cmpRes > 0) { this.patchOperationList.add(new PatchOperation<>(PatchOperation.OP_ADD, newIndexedItem.getKey(), newIndexedItem.getValue())); ++newCursor; } else { int oldIndex = oldIndexedItem.getKey(); int newIndex = newIndexedItem.getKey(); int oldOffset = getItemOffsetOrIndex(oldIndexedItem.getKey(), oldIndexedItem.getValue()); int newOffset = getItemOffsetOrIndex(newIndexedItem.getKey(), newIndexedItem.getValue()); if (oldIndex ! = newIndex) { this.oldIndexToNewIndexMap.put(oldIndex, newIndex); } if (oldOffset ! = newOffset) { this.oldOffsetToNewOffsetMap.put(oldOffset, newOffset); } ++oldCursor; ++newCursor; }} /********** first half **********************/}Copy the code

Analysis:

1. First, read the data corresponding to oldDex and newDex and sort them, namely, adjustedOldIndexedItems and adjustedNewIndexedItems.

2. Get oldItem and newItem respectively, and compare their value pairs: If <0, the oldItem is considered to be deleted and recorded as patchoperation. OP_DEL. The oldItem index is recorded in the PatchOperation object and added to the patchOperationList.

1. After the above traversal, we have a patchOperationList object. Continue with the next half of the code:

/ * * * * * * * * * * * * * after half of the * * * * * * * * * * * * * * * * * * * * * * / / / So far all the diff works are done. Then we perform some optimize works. / / detail: {OP_DEL idx} followed by {OP_ADD the_same_idx newItem} // will be replaced by {OP_REPLACE idx newItem} Collections.sort(this.patchOperationList, comparatorForPatchOperationOpt); Iterator<PatchOperation<T>> patchOperationIt = this.patchOperationList.iterator(); PatchOperation<T> prevPatchOperation = null; while (patchOperationIt.hasNext()) { PatchOperation<T> patchOperation = patchOperationIt.next(); if (prevPatchOperation ! = null && prevPatchOperation.op == PatchOperation.OP_DEL && patchOperation.op == PatchOperation.OP_ADD ) { if (prevPatchOperation.index == patchOperation.index) { prevPatchOperation.op = PatchOperation.OP_REPLACE; prevPatchOperation.newItem = patchOperation.newItem; patchOperationIt.remove(); prevPatchOperation = null; } else { prevPatchOperation = patchOperation; } } else { prevPatchOperation = patchOperation; } } // Finally we record some information for the final calculations. patchOperationIt = this.patchOperationList.iterator(); while (patchOperationIt.hasNext()) { PatchOperation<T> patchOperation = patchOperationIt.next(); switch (patchOperation.op) { case PatchOperation.OP_DEL: { indexToDelOperationMap.put(patchOperation.index, patchOperation); break; } case PatchOperation.OP_ADD: { indexToAddOperationMap.put(patchOperation.index, patchOperation); break; } case PatchOperation.OP_REPLACE: { indexToReplaceOperationMap.put(patchOperation.index, patchOperation); break; }}}}Copy the code

Analysis:

1. Firstly, sort the patchOperationList according to index. If the index is consistent, DEL(delete) and then ADD(ADD) will be added.

2. The next iteration of all operations is to convert the index-consistent, sequential DEL and ADD operations into REPLACE operations.

3. Finally turn patchOperationList into three Map, respectively: indexToDelOperationMap, indexToAddOperationMap, indexToReplaceOperationMap.

4. After execute is completed, our main products are three maps, which record respectively: which indexes in oldDex need to be deleted; What items are added in newDex? Which items need to be replaced with new items?

5. It’s basically DexDif algorithm is the core idea of the (StringDataSectionDiffAlgorithm, for example, as well as other analysis);

Execute () and simulatePatchOperation() :

public void simulatePatchOperation(int baseOffset) { boolean isNeedToMakeAlign = getTocSection(this.oldDex).isElementFourByteAligned; int oldIndex = 0; int patchedIndex = 0; int patchedOffset = baseOffset; while (oldIndex < this.oldItemCount || patchedIndex < this.newItemCount) { if (this.indexToAddOperationMap.containsKey(patchedIndex)) { PatchOperation<T> patchOperation = this.indexToAddOperationMap.get(patchedIndex); if (isNeedToMakeAlign) { patchedOffset = SizeOf.roundToTimesOfFour(patchedOffset); } T newItem = patchOperation.newItem; int itemSize = getItemSize(newItem); updateIndexOrOffset(this.newToPatchedIndexMap,0,getItemOffsetOrIndex(patchOperation.index, newItem),0,patchedOffset); ++patchedIndex; patchedOffset += itemSize; } else if (this.indexToReplaceOperationMap.containsKey(patchedIndex)) { PatchOperation<T> patchOperation = this.indexToReplaceOperationMap.get(patchedIndex); /******* omit N code ***********/ ++patchedIndex; patchedOffset += itemSize; } else if (this.indexToDelOperationMap.containsKey(oldIndex)) { ++oldIndex; } else if (this.indexToReplaceOperationMap.containsKey(oldIndex)) { ++oldIndex; } else if (oldIndex < enclosing oldItemCount) {/ * * * * * * * to omit N code * * * * * * * * * * * / + + oldIndex; ++patchedIndex; patchedOffset += itemSize; } } this.patchedSectionSize = SizeOf.roundToTimesOfFour(patchedOffset - baseOffset); }Copy the code

There are several cases where patchedOffset += itemSize:

1. The indexToAddOperationMap contains the patchIndex

2. IndexToReplaceOperationMap contains patchIndex

3. Out of a indexToDelOperationMap and indexToReplaceOperationMap oldDex.

This patchedSectionSize actually corresponds to the size of this region of newDex. So, there are ITEMS that need to be added, ITEMS that will be replaced, and ITEMS from OLD ITEMS that have not been deleted or replaced.

After such an algorithm, we get PatchOperationList and the corresponding region sectionSize. After all the algorithms are executed, the PatchOperationList for each algorithm and sectionSize for each region should be obtained. The sectionSize for each region actually translates to the offset for each region.

private void writeResultToStream(OutputStream os) throws IOException { DexDataBuffer buffer = new DexDataBuffer(); buffer.write(DexPatchFile.MAGIC); buffer.writeShort(DexPatchFile.CURRENT_VERSION); buffer.writeInt(this.patchedDexSize); // we will return here to write firstChunkOffset later. int posOfFirstChunkOffsetField = buffer.position(); buffer.writeInt(0); buffer.writeInt(this.patchedStringIdsOffset); buffer.writeInt(this.patchedTypeIdsOffset); buffer.writeInt(this.patchedProtoIdsOffset); / * * * * * omit other algorithm * * * * * * * * * * * / buffer. Write (this.oldDex.com puteSignature (false)); int firstChunkOffset = buffer.position(); buffer.position(posOfFirstChunkOffsetField); buffer.writeInt(firstChunkOffset); buffer.position(firstChunkOffset); writePatchOperations(buffer, this.stringDataSectionDiffAlg.getPatchOperationList()); writePatchOperations(buffer, this.typeIdSectionDiffAlg.getPatchOperationList()); writePatchOperations(buffer, this.typeListSectionDiffAlg.getPatchOperationList()); / * * * * * omit other algorithm * * * * * * * * * * * / byte [] bufferData = buffer. The array (); os.write(bufferData); os.flush(); }Copy the code

Where the writePatchOperations method is the write method, we’ll just look at stringDataSectionDiffAlg:

private <T extends Comparable<T>> void writePatchOperations( DexDataBuffer buffer, List<PatchOperation<T>> patchOperationList ) { List<Integer> delOpIndexList = new ArrayList<>(patchOperationList.size()); List<Integer> addOpIndexList = new ArrayList<>(patchOperationList.size()); List<Integer> replaceOpIndexList = new ArrayList<>(patchOperationList.size()); List<T> newItemList = new ArrayList<>(patchOperationList.size()); for (PatchOperation<T> patchOperation : patchOperationList) { switch (patchOperation.op) { case PatchOperation.OP_DEL: { delOpIndexList.add(patchOperation.index); break; } case PatchOperation.OP_ADD: { addOpIndexList.add(patchOperation.index); newItemList.add(patchOperation.newItem); break; } case PatchOperation.OP_REPLACE: { replaceOpIndexList.add(patchOperation.index); newItemList.add(patchOperation.newItem); break; } } } buffer.writeUleb128(delOpIndexList.size()); int lastIndex = 0; for (Integer index : delOpIndexList) { buffer.writeSleb128(index - lastIndex); lastIndex = index; } buffer.writeUleb128(addOpIndexList.size()); lastIndex = 0; for (Integer index : addOpIndexList) { buffer.writeSleb128(index - lastIndex); lastIndex = index; } buffer.writeUleb128(replaceOpIndexList.size()); lastIndex = 0; for (Integer index : replaceOpIndexList) { buffer.writeSleb128(index - lastIndex); lastIndex = index; } for (T newItem : newItemList) { if (newItem instanceof StringData) { buffer.writeStringData((StringData) newItem); } else /*********** other *******************/}}Copy the code

1. The number of DEL operations and the index of each del

2. The number of add operations, and the index of each add

3. The number of replace operations, each requiring the index of replace

4. Write newItemList in turn.

Finally, let’s see what the patch we generated looks like:

1. Firstly, it contains several fields to prove that it is tinker Patch

2. Contains the offset of each region of newDex, that is, newDex can be divided into multiple regions and located to the starting point

3. OldDex contains deleted indexes (oldDex), new indexes and values, and replaced indexes and values of items in various sections of newDex

In this way, we guess the logic of Patch as follows:

1. Firstly, the starting point of each region is determined according to the offset of each region

2. Read the items in each area of oldDex, then remove the items to be deleted and replaced in oldDex according to the patch, and add the new items and replaced items to form the items in this area of newOld.

So, a region of newDex contains:

 oldItems - del - replace + addItems + replaceItems
Copy the code

In this way, the generation process of the patch package is completed. Then how does the server synthesize the new Dex after delivering the patch? Let’s analyze the second part:

2. Dex after patch package delivery;

How to synthesize the full amount of new Dex to run

When the app, after receipt of the patch server issued by trigger DefaultPatchListener. OnPatchReceived events, Call TinkerPatchService. RunPatchService start the patch process patch patch.

Upgradepatch.trypatch () will first check the validity and signature of the patch and whether the patch has been installed. After passing the check, the patch of dex, SO and RES files will be tried.

We mainly analyze DexDiffPatchInternal tryRecoverDexFiles, discuss the patch process of dex.

TryRecoverDexFiles call DexDiffPatchInternal. PatchDexFile:

private static void patchDexFile( ZipFile baseApk, ZipFile patchPkg, ZipEntry oldDexEntry, ZipEntry patchFileEntry, ShareDexDiffPatchInfo patchInfo, File patchedDexFile) throws IOException {/********** omits N lines of code and eventually calls this method ************/ new DexPatchApplier(oldDexStream, patchFileStream).executeAndSaveTo(patchedDexFile); }Copy the code

Finally passed DexPatchApplier. ExecuteAndSaveTo performing dex and production quantity.

public void executeAndSaveTo(File file) throws IOException { OutputStream os = null; try { os = new BufferedOutputStream(new FileOutputStream(file)); executeAndSaveTo(os); } finally { if (os ! = null) { try { os.close(); } catch (Exception e) { // ignored. } } } }Copy the code

ExecuteAndSaveTo (OS) The first of three parts

public void executeAndSaveTo(OutputStream out) throws IOException { // Before executing, we should check if this patch can be applied to // old dex we passed in. byte[] oldDexSign = this.oldDex.computeSignature(false); if (oldDexSign == null) { throw new IOException("failed to compute old dex's signature."); } if (this.patchFile == null) { throw new IllegalArgumentException("patch file is null."); } byte[] oldDexSignInPatchFile = this.patchFile.getOldDexSignature(); if (CompareUtils.uArrCompare(oldDexSign, oldDexSignInPatchFile) ! = 0) { throw new IOException( String.format( "old dex signature mismatch! expected: %s, actual: %s", Arrays.toString(oldDexSign), Arrays.toString(oldDexSignInPatchFile) ) ); } // Firstly, set sections' offset after patched, sort according to their offset so that // the dex lib of aosp can calculate section size. TableOfContents patchedToc = this.patchedDex.getTableOfContents(); patchedToc.header.off = 0; patchedToc.header.size = 1; patchedToc.mapList.size = 1; patchedToc.stringIds.off = this.patchFile.getPatchedStringIdSectionOffset(); patchedToc.typeIds.off = this.patchFile.getPatchedTypeIdSectionOffset(); Patchedtoc.typelists. off /***** omits other algorithm processes ************/ arrays.sort (Patchedtoc.sections); patchedToc.computeSizesFromOffsets();Copy the code

Then sort and set byteCount and other fields. PatchedDex is the final synthetic dex.

ExecuteAndSaveTo (OS) Part two of three

// Secondly, run patch algorithms according to sections' dependencies. this.stringDataSectionPatchAlg = new StringDataSectionPatchAlgorithm( patchFile, oldDex, patchedDex, oldToPatchedIndexMap ); this.typeIdSectionPatchAlg = new TypeIdSectionPatchAlgorithm( patchFile, oldDex, patchedDex, oldToPatchedIndexMap ); / * * * to omit other algorithms code * * * * * / this. StringDataSectionPatchAlg. The execute (); this.typeIdSectionPatchAlg.execute(); /*** omit other algorithm code *****/Copy the code

The second part initializes all 15 algorithms and executes execute(). We are still take stringDataSectionPatchAlg to analyze, in fact or call the execute method of abstract superclass DexSectionPatchAlgorithm:

public void execute() {
    final int deletedItemCount = patchFile.getBuffer().readUleb128();
    final int[] deletedIndices = readDeltaIndiciesOrOffsets(deletedItemCount);

    final int addedItemCount = patchFile.getBuffer().readUleb128();
    final int[] addedIndices = readDeltaIndiciesOrOffsets(addedItemCount);

    final int replacedItemCount = patchFile.getBuffer().readUleb128();
    final int[] replacedIndices = readDeltaIndiciesOrOffsets(replacedItemCount);

    final TableOfContents.Section tocSec = getTocSection(this.oldDex);
    Dex.Section oldSection = null;

    int oldItemCount = 0;
    if (tocSec.exists()) {
        oldSection = this.oldDex.openSection(tocSec);
        oldItemCount = tocSec.size;
    }

    // Now rest data are added and replaced items arranged in the order of
    // added indices and replaced indices.
    doFullPatch(
            oldSection, oldItemCount, deletedIndices, addedIndices, replacedIndices
    );
}
Copy the code

The algorithm here is a reverse process from the DexDiff that generated the patch. The merging algorithm of each region adopts two-way merging to delete, add and replace elements on the basis of the old dex. :

1. The number of del operations, the index of each del, is stored in an int[] deletedIndices;

2. The number of add operations, the index of each add, is stored in an int[] addedIndices;

3. The number of replace operations, each index to replace, stored in an int[] replacedIndices;

Next we get oldItems and oldItemCount from oldDex. Then execute doFullPatch(oldSection, oldItemCount, deletedIndices, addedIndices, replacedIndices) with these parameters:

private void doFullPatch( Dex.Section oldSection, int oldItemCount, int[] deletedIndices, int[] addedIndices, int[] replacedIndices ) { int deletedItemCount = deletedIndices.length; int addedItemCount = addedIndices.length; int replacedItemCount = replacedIndices.length; int newItemCount = oldItemCount + addedItemCount - deletedItemCount; int deletedItemCounter = 0; int addActionCursor = 0; int replaceActionCursor = 0; int oldIndex = 0; int patchedIndex = 0; while (oldIndex < oldItemCount || patchedIndex < newItemCount) { if (addActionCursor < addedItemCount && AddedIndices [addActionCursor] = = patchedIndex) {/ * * * * * * * * * * * * * * * * part 1 * * * * * * * * * * * * * * * * * * / T addedItem = nextItem(patchFile.getBuffer()); int patchedOffset = writePatchedItem(addedItem); ++addActionCursor; ++patchedIndex; } else if (replaceActionCursor < replacedItemCount && replacedIndices[replaceActionCursor] == patchedIndex) { / * * * * * * * * * * * * * * * * part 2 omit N lines of code, and the upper part of the similar, behind to do concrete analysis * * * * * * * * * * * * * * * * * * / int patchedOffset = writePatchedItem (addedItem); } else if (Arrays. BinarySearch (deletedIndices, oldIndex) >= 0) {/**************** ******************/ int patchedOffset = writePatchedItem(addedItem); } else if (Arrays. BinarySearch (replacedIndices, oldIndex) >= 0) {/**************** ******************/ int patchedOffset = writePatchedItem(addedItem); } else if (oldIndex < oldItemCount) {/**************** ******************/ int patchedOffset = writePatchedItem(addedItem); } } if (addActionCursor ! = addedItemCount || deletedItemCounter ! = deletedItemCount || replaceActionCursor ! = replacedItemCount ) { throw new IllegalStateException( /*************.. String...... /)); }}Copy the code

Item writing can be seen through the code (part 1, 2, 3(1), 3(2), and 4), the specific code is as follows:

1. First, check whether the patchIndex is included in addIndices. If so, write:

   if (addActionCursor < addedItemCount && addedIndices[addActionCursor] == patchedIndex) {

          T addedItem = nextItem(patchFile.getBuffer());
          int patchedOffset = writePatchedItem(addedItem);
          ++addActionCursor;
          ++patchedIndex;Copy the code

   }

2. RepalceIndices; repalceIndices;

   if (replaceActionCursor < replacedItemCount && replacedIndices[replaceActionCursor] == patchedIndex) {

   T replacedItem = nextItem(patchFile.getBuffer());
   int patchedOffset = writePatchedItem(replacedItem);
   ++replaceActionCursor;
   ++patchedIndex;Copy the code

   }

3. If oldIndex is deleted or replaced, skip:

   if (Arrays.binarySearch(deletedIndices, oldIndex) >= 0) {

   T skippedOldItem = nextItem(oldSection); // skip old item.
   markDeletedIndexOrOffset(
           oldToPatchedIndexMap,
           oldIndex,
           getItemOffsetOrIndex(oldIndex, skippedOldItem)
   );
   ++oldIndex;
   ++deletedItemCounter;Copy the code

   } else
   
   if (Arrays.binarySearch(replacedIndices, oldIndex) >= 0) {

   T skippedOldItem = nextItem(oldSection); // skip old item.
   markDeletedIndexOrOffset(
           oldToPatchedIndexMap,
           oldIndex,
           getItemOffsetOrIndex(oldIndex, skippedOldItem)
   );
   ++oldIndex;Copy the code

   }

4. The last index refers to the part of oldIndex that is not delete and replace, which is the same items as newDex.

   if (oldIndex < oldItemCount) {

   T oldItem = adjustItem(this.oldToPatchedIndexMap, nextItem(oldSection));
   
   int patchedOffset = writePatchedItem(oldItem);
   
   updateIndexOrOffset(
           this.oldToPatchedIndexMap,
           oldIndex,
           getItemOffsetOrIndex(oldIndex, oldItem),
           patchedIndex,
           patchedOffset
   );
   
   ++oldIndex;
   ++patchedIndex;
   }
   Copy the code

ExecuteAndSaveTo (OS) Part 3 of three parts

public void executeAndSaveTo(OutputStream out) throws IOException { / * * * * * * * * * * * * to omit this. StringDataSectionPatchAlg. The execute () code before * * * * * * * * * / enclosing stringDataSectionPatchAlg. The execute (); / * * * * * * to omit other algorithms perform the execute () * * * * * * * * * * * * * * * * * * / / / Thirdly, write the header, mapList. Calculate and write patched dex's sign and checksum. Dex.Section headerOut = this.patchedDex.openSection(patchedToc.header.off); patchedToc.writeHeader(headerOut); Dex.Section mapListOut = this.patchedDex.openSection(patchedToc.mapList.off); patchedToc.writeMap(mapListOut); this.patchedDex.writeHashes(); // Finally, write patched dex to file. this.patchedDex.writeTo(out); }Copy the code

Iii. Loading process after generating full Dex

The above is the generation process of the complete Dex, which is also the core of the algorithm, so it took a long time. The following is the loading process after we generated the complete Dex, which is mainly under this package:

TinkerApplication isolates the actual app business by reflection so that the actual app content can be modified during hot updates.

OnBaseContextAttached in TinkerApplication calls the tryLoad of TinkerLoader by reflection to load the synthesized dex.

private static final String TINKER_LOADER_METHOD = "tryLoad"; private void loadTinker() { //disable tinker, not need to install if (tinkerFlags == TINKER_DISABLE) { return; } tinkerResultIntent = new Intent(); try { //reflect tinker loader, because loaderClass may be define by user! Class<? > tinkerLoadClass = Class.forName(loaderClassName, false, getClassLoader()); Method loadMethod = tinkerLoadClass.getMethod(TINKER_LOADER_METHOD, TinkerApplication.class); Constructor<? > constructor = tinkerLoadClass.getConstructor(); tinkerResultIntent = (Intent) loadMethod.invoke(constructor.newInstance(), this); } catch (Throwable e) { //has exception, put exception error code ShareIntentUtil.setIntentReturnCode(tinkerResultIntent, ShareConstants.ERROR_LOAD_PATCH_UNKNOWN_EXCEPTION); tinkerResultIntent.putExtra(INTENT_PATCH_EXCEPTION, e); }}Copy the code

Here is the tryLoad method in TinkerLoader called by reflection:

@Override
public Intent tryLoad(TinkerApplication app) {
    Intent resultIntent = new Intent();

    long begin = SystemClock.elapsedRealtime();
    tryLoadPatchFilesInternal(app, resultIntent);
    long cost = SystemClock.elapsedRealtime() - begin;
    ShareIntentUtil.setIntentPatchCostTime(resultIntent, cost);
    return resultIntent;
}
Copy the code

(including tryLoadPatchFilesInternal is the core of the Patch file loading function code is more, everybody should be able to understand each annotation is what to do) :

private void tryLoadPatchFilesInternal(TinkerApplication app, Intent resultIntent) {
    final int tinkerFlag = app.getTinkerFlags();

    if (!ShareTinkerInternals.isTinkerEnabled(tinkerFlag)) {
        //tinkerFlag是否开启，否则不加载
        Log.w(TAG, "tryLoadPatchFiles: tinker is disable, just return");
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_DISABLE);
        return;
    }
    //tinker
    File patchDirectoryFile = SharePatchFileUtil.getPatchDirectory(app);
    if (patchDirectoryFile == null) {
        //tinker目录是否生成，没有则表示没有生成全量的dex，不需要重新加载
        Log.w(TAG, "tryLoadPatchFiles:getPatchDirectory == null");
        //treat as not exist
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_DIRECTORY_NOT_EXIST);
        return;
    }
    //tinker/patch.info
    File patchInfoFile = SharePatchFileUtil.getPatchInfoFile(patchDirectoryPath);

    //check patch info file whether exist
    if (!patchInfoFile.exists()) {
        //tinker/patch.info是否存在，否则不加载
        Log.w(TAG, "tryLoadPatchFiles:patch info not exist:" + patchInfoFile.getAbsolutePath());
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_INFO_NOT_EXIST);
        return;
    }
    //old = 641e634c5b8f1649c75caf73794acbdf
    //new = 2c150d8560334966952678930ba67fa8
    File patchInfoLockFile = SharePatchFileUtil.getPatchInfoLockFile(patchDirectoryPath);

    patchInfo = SharePatchInfo.readAndCheckPropertyWithLock(patchInfoFile, patchInfoLockFile);
    if (patchInfo == null) {
        //读取patch.info，读取失败则不加载
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_INFO_CORRUPTED);
        return;
    }

    String oldVersion = patchInfo.oldVersion;
    String newVersion = patchInfo.newVersion;
    String oatDex = patchInfo.oatDir;

    if (oldVersion == null || newVersion == null || oatDex == null) {
        //判断版本号是否为空，为空则不加载
        //it is nice to clean patch
        Log.w(TAG, "tryLoadPatchFiles:onPatchInfoCorrupted");
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_INFO_CORRUPTED);
        return;
    }

    resultIntent.putExtra(ShareIntentUtil.INTENT_PATCH_OLD_VERSION, oldVersion);
    resultIntent.putExtra(ShareIntentUtil.INTENT_PATCH_NEW_VERSION, newVersion);

    //tinker/patch.info/patch-641e634c
    String patchVersionDirectory = patchDirectoryPath + "/" + patchName;

    File patchVersionDirectoryFile = new File(patchVersionDirectory);

    if (!patchVersionDirectoryFile.exists()) {
        //判断patch version directory（//tinker/patch.info/patch-641e634c）是否存在
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_VERSION_DIRECTORY_NOT_EXIST);
        return;
    }

    //tinker/patch.info/patch-641e634c/patch-641e634c.apk
    File patchVersionFile = new File(patchVersionDirectoryFile.getAbsolutePath(), SharePatchFileUtil.getPatchVersionFile(version));

    if (!SharePatchFileUtil.isLegalFile(patchVersionFile)) {
        //判断patchVersionDirectoryFile（//tinker/patch.info/patch-641e634c/patch-641e634c.apk）是否存在
        Log.w(TAG, "tryLoadPatchFiles:onPatchVersionFileNotFound");
        //we may delete patch info file
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_VERSION_FILE_NOT_EXIST);
        return;
    }

    ShareSecurityCheck securityCheck = new ShareSecurityCheck(app);

    int returnCode = ShareTinkerInternals.checkTinkerPackage(app, tinkerFlag, patchVersionFile, securityCheck);
    if (returnCode != ShareConstants.ERROR_PACKAGE_CHECK_OK) {
        //checkTinkerPackage,（如tinkerId和oldTinkerId不能相等，否则不加载)
        Log.w(TAG, "tryLoadPatchFiles:checkTinkerPackage");
        resultIntent.putExtra(ShareIntentUtil.INTENT_PATCH_PACKAGE_PATCH_CHECK, returnCode);
        ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_PACKAGE_CHECK_FAIL);
        return;
    }

    if (isEnabledForDex) {
        //tinker/patch.info/patch-641e634c/dex
        boolean dexCheck = TinkerDexLoader.checkComplete(patchVersionDirectory, securityCheck, oatDex, resultIntent);
        if (!dexCheck) {
            //检测dex的完整性，包括dex是否全部生产，是否对dex做了优化，优化后的文件是否存在（//tinker/patch.info/patch-641e634c/dex）
            //file not found, do not load patch
            Log.w(TAG, "tryLoadPatchFiles:dex check fail");
            return;
        }
    }
    /****省略对so res文件进行完整性检测***************/
    final boolean isEnabledForNativeLib = ShareTinkerInternals.isTinkerEnabledForNativeLib(tinkerFlag);
    /***************************************/
    //now we can load patch jar
    if (isEnabledForDex) {
        /********************划重点---TinkerDexLoader.loadTinkerJars********************/
        boolean loadTinkerJars = TinkerDexLoader.loadTinkerJars(app, patchVersionDirectory, oatDex, resultIntent, isSystemOTA);

        if (isSystemOTA) {
            // update fingerprint after load success
            patchInfo.fingerPrint = Build.FINGERPRINT;
            patchInfo.oatDir = loadTinkerJars ? ShareConstants.INTERPRET_DEX_OPTIMIZE_PATH : ShareConstants.DEFAULT_DEX_OPTIMIZE_PATH;
            // reset to false
            oatModeChanged = false;

            if (!SharePatchInfo.rewritePatchInfoFileWithLock(patchInfoFile, patchInfo, patchInfoLockFile)) {
                ShareIntentUtil.setIntentReturnCode(resultIntent, ShareConstants.ERROR_LOAD_PATCH_REWRITE_PATCH_INFO_FAIL);
                Log.w(TAG, "tryLoadPatchFiles:onReWritePatchInfoCorrupted");
                return;
            }
            // update oat dir
            resultIntent.putExtra(ShareIntentUtil.INTENT_PATCH_OAT_DIR, patchInfo.oatDir);
        }
        if (!loadTinkerJars) {
            Log.w(TAG, "tryLoadPatchFiles:onPatchLoadDexesFail");
            return;
        }
    }

    return;
}
Copy the code

The TinkerDexLoader loadTinkerJars is used to handle load dex file.

public static boolean loadTinkerJars(final TinkerApplication application, String directory, String oatDir, Intent intentResult, Boolean isSystemOTA) {/***** omit some code ****************/ PathClassLoader classLoader = (PathClassLoader) TinkerDexLoader.class.getClassLoader(); /*********** omits N lines of code to generate a list of valid files, Optimize the dex file * * * * * * * * * * * * * * / / / loaded dex SystemClassLoaderAdder. InstallDexes (application, this optimizeDir, legalFiles); }Copy the code

Then SystemClassLoaderAdder. InstallDexes based upon the version of the android dex for installation:

@SuppressLint("NewApi") public static void installDexes(Application application, PathClassLoader loader, File dexOptDir, List<File> files) throws Throwable { Log.i(TAG, "installDexes dexOptDir: " + dexOptDir.getAbsolutePath() + ", dex size:" + files.size()); if (! files.isEmpty()) { files = createSortedAdditionalPathEntries(files); ClassLoader classLoader = loader; if (Build.VERSION.SDK_INT >= 24 && ! checkIsProtectedApp(files)) { classLoader = AndroidNClassLoader.inject(loader, application); } //because in dalvik, if inner class is not the same classloader with it wrapper class. //it won't fail at dex2opt if (Build.VERSION.SDK_INT >= 23) { V23.install(classLoader, files, dexOptDir); } else if (Build.VERSION.SDK_INT >= 19) { V19.install(classLoader, files, dexOptDir); } else if (Build.VERSION.SDK_INT >= 14) { V14.install(classLoader, files, dexOptDir); } else { V4.install(classLoader, files, dexOptDir); } //install done sPatchDexCount = files.size(); Log.i(TAG, "after loaded classloader: " + classLoader + ", dex size:" + sPatchDexCount); if (! checkDexInstall(classLoader)) { //reset patch dex SystemClassLoaderAdder.uninstallPatchDex(classLoader); throw new TinkerRuntimeException(ShareConstants.CHECK_DEX_INSTALL_FAIL); }}}Copy the code

When we talk about dex loading, we usually use PathClassLoader and DexClassLoader to load classes, and PathClassLoader acts as the loader of system classes and application classes. The DexClassLoader is used to load classes.dex files from inside.jar and.apk files.

How does install do this?

/** *Installer for platform versions 23. */ private static final class V23 { private static void install(ClassLoader loader, List<File> additionalClassPathEntries, File optimizedDirectory) throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, InvocationTargetException, NoSuchMethodException, IOException { /* The patched class loader is expected to be a descendant of *dalvik.system.BaseDexClassLoader. We modify  its *dalvik.system.DexPathList pathList field to append additional DEX *file entries. */ Field pathListField = ShareReflectUtil.findField(loader, "pathList"); Object dexPathList = pathListField.get(loader); ArrayList<IOException> suppressedExceptions = new ArrayList<IOException>(); ShareReflectUtil.expandFieldArray(dexPathList, "dexElements", makePathElements(dexPathList, new ArrayList<File>(additionalClassPathEntries), optimizedDirectory, suppressedExceptions)); if (suppressedExceptions.size() > 0) { for (IOException e : suppressedExceptions) { Log.w(TAG, "Exception in makePathElement", e); throw e; }}} / * * * * * * * * * * * * * omitted makePathElements method * * * * * * * * * * * * * * * /}Copy the code

First get the dexPathList object of The BaseDexClassLoader, and then use the makeDexElements method of dexPathList to convert the dex we want to install into an Element[] object. The new Element[] object is merged with the dexElements of the dexPathList. Since the added dex is placed at the top of the dexElements array, when we find the class using findClass, That’s the class that we use in our latest dex. There are some changes in the functions and fields of DexPathList and other classes in different versions, and others are similar.

At this point, the entire dex loading process is over!

Other updates using Tinker, such as so library updates, library updates we can see in the source code according to the dex loading process above.

Comparison of hot update schemes

Ok, we also mentioned several hot update schemes above, and you can search for other hot update schemes.

Ali compared AndFix to HotFix and Sophix above, so let’s compare the current hot update solutions to see which one is better:

From the comparison, we can also find that Sophix and Tinker are the two giants’ latest hot update schemes, which are quite impressive. You can try them if you need to.

Because of the time, I have not finished my hot update plan, so I will not put it up for the time being. When I finish writing, I will put the link of the next article. Thank you for your support!

This article is part of the third Android Bus Blog Competition: Don’t be a Dying frog android Bus blog Contest for you!

digression

Soon the company’s internship is about to end, I feel time flies, I have to prepare for the job. Coincidentally, @Quan Xiaoyang held three blog contests, which happened to be three months during my internship. I also learned a lot from a blog post every month. I would like to give Sunny a lot of praise here. The voice is also very good, the person is also very beautiful! Well, that’s it.

Of course, I also got to know a lot of great god bloggers, @Nanchen, @Joge, @chicken, @Meditation, @Kaidi, etc. The other is not a aite (seems so @ is useless), read everyone’s article also let me learn a lot of things!

Although we do not know each other, but so many people around the world, we can gather in the bus this place, learn to grow together (of course, and that together haendless cattle! I have to make fun of it, why markdown can’t add expressions (tears here,,)), which is also a great fate, I hope we can go wider and farther in the future! Come on, everybody!