One, a brief introduction

Hot fix is undoubtedly a hot new technology in the past two years, which is one of the skills that android engineers must learn. Before there were hot fix in an online app if there is a bug, even a very small bug, not timely update possible risks, if to update the app to repackage published to the application market, allow users to download again, thus greatly reduce the user experience, when hot fix, Such questions are no longer a problem.

At present, the hot repair scheme can be divided into two groups, which are:

  1. Ali: DeXposed, andfix: start from the bottom binary (C language).
  2. Tencent: Tinker: From the Java loading mechanism.

The topic of this article is not to discuss the use of either of these solutions, but rather to explore the principles and implementation of hot fixes based on the Java loading mechanism. (Similar to Tinker, but not so simple)

How to dynamically fix bugs in Android

I think there are two kinds of bugs in general (may not be quite accurate) :

  • The code does not function as expected by the project, i.e. the code logic is faulty.
  • The program code is not robust enough, causing the App to crash at runtime.

In both cases, there is usually a problem with one or more classes. In an ideal state, we can simply update the fixed classes to the app on the user’s phone to fix the bug. But for simplicity, how do you dynamically update these classes? In fact, whatever the hot fix, there are several steps:

  1. Deliver the patch (including the fixed class) to the user’s phone, that is, let the app download from the server (network transmission)
  2. The class in the patch is called by the app by **” somehow “**.

By “somehow”, for the purposes of this article, I mean using the Android classloader, which loads the fixed classes, overwrites the classes that are supposed to be problematic, and theoretically fixes the bug. So let’s start by understanding and analyzing the class loaders in Android.

Class loaders in Android

JVM based Java applications use a ClassLoader to load their classes. However, we know that Android optimized the JVM by using Dalvik, and the class files are packaged into a dex file. For example, if you want to load the class files in the dex file, you need to use a PathClassLoader or a DexClassLoader.

1, source view

The general source code can be found in Android Studio, but the source code of PathClassLoader and DexClassLoader are system-level sources, so they cannot be directly viewed in Android Studio. However, there are two ways to the external view: the first is a view by means of image download the Android source code, but general image source volume is larger, download, and just to see the source of 3, 4, a file on every download source, 3, 4 g does not too smart, so we usually adopts the second way: Go to androidXref.com this website to view directly, the following will be listed after the analysis of a few classes of source address, for visitors to browse.

Here is some of the source code for Android 5.0:

  • PathClassLoader.java
  • DexClassLoader.java
  • BaseDexClassLoader.java
  • DexPathList.java

2. Differences between PathClassLoader and DexClassLoader

1) Usage scenarios

  • PathClassLoader: It can only load apK files (/data/app directory) that have been installed in the Android system. It is the default class loader used by Android.
  • DexClassLoader: The DexClassLoader can load the dex/ JAR /apk/zip files in any directory. It is more flexible than the PathClassLoader and is the focus of hotfix.

2) Code differences

Because the source code of PathClassLoader and DexClassLoader is very simple, I will copy all the source code of them directly:

// PathClassLoader public class PathClassLoader extends BaseDexClassLoader { public PathClassLoader(String dexPath, ClassLoader parent) { super(dexPath, null, null, parent); } public PathClassLoader(String dexPath, String librarySearchPath, ClassLoader parent) { super(dexPath, null, librarySearchPath, parent); } } // DexClassLoader public class DexClassLoader extends BaseDexClassLoader { public DexClassLoader(String dexPath, String optimizedDirectory, String librarySearchPath, ClassLoader parent) { super(dexPath, new File(optimizedDirectory), librarySearchPath, parent); }}Copy the code

By comparison, two conclusions can be drawn:

  • PathClassLoader and DexClassLoader inherit from BaseDexClassLoader.
  • PathClassLoader and DexClassLoader both call the constructor of the parent class in their constructors, but DexClassLoader passes an extra optimizedDirectory.

3, BaseDexClassLoader

By looking at the source code of PathClassLoader and DexClassLoader, we can determine that the real meaningful processing logic must be in the BaseDexClassLoader, so the following focus on the source code of BaseDexClassLoader.

1) The constructor

Let’s take a look at what the BaseDexClassLoader constructor does:

public class BaseDexClassLoader extends ClassLoader { ... public BaseDexClassLoader(String dexPath, File optimizedDirectory, String libraryPath, ClassLoader parent){ super(parent); this.pathList = new DexPathList(this, dexPath, libraryPath, optimizedDirectory); }... }Copy the code
  • DexPath: indicates the directory where the program file to be loaded (usually dex file, or JAR /apk/zip file) resides.
  • OptimizedDirectory: The output directory of the dex file (this directory is used to store the dex files that are decompressed when you load the program files in compressed formats such as JAR /apk/zip).
  • LibraryPath: libraryPath needed to load program files.
  • Parent: Parent loader

***tip: ** The concept of “program file” is my own definition, because from the perspective of a complete App, the program file specifies the classes.dex file in the apK package; But from a hotfix perspective, program files refer to patches.

Because The PathClassLoader will only load the dex file in the installed package, and the DexClassLoader can not only load the dex file, but also load the dex in jar, APK, and zip files. We know that JAR, APK, and zip are actually some compression formats. To get the dex file in the package, you need to decompress it. Therefore, the DexClassLoader will specify a directory to decompress when calling the parent class constructor.

The optimizedDirectory is obsolete and no longer takes effect. For details, see the baseDexClassLoader.java source code for Android 8.0

2) Obtain class

The class loader will certainly provide a method for the outside world to find the class it loaded into. That method is findClass(), but there is no overwritten findClass() method in either the PathClassLoader or the DexClassLoader source code. But their parent class BaseDexClassLoader does override findClass(), so let’s take a look at what the findClass() method of BaseDexClassLoader does:

private final DexPathList pathList; @Override protected Class<? > findClass(String name) throws ClassNotFoundException { List<Throwable> suppressedExceptions = new ArrayList<Throwable>(); Class c = pathList.findClass(name, suppressedExceptions); if (c == null) { ClassNotFoundException cnfe = new ClassNotFoundException("Didn't find class \"" + name + "\" on path: " + pathList); for (Throwable t : suppressedExceptions) { cnfe.addSuppressed(t); } throw cnfe; } return c; }Copy the code

As you can see, the findClass() method of the BaseDexClassLoader actually gets the class from the findClass() method of the DexPathList object (pathList), This DexPathList object happens to have been created earlier in the BaseDexClassLoader constructor. So let’s take a look at what’s going on in the DexPathList class.

4, DexPathList

Before analyzing a large source code, what do we need to know from the source code? In this way, I will not get lost in the “sea of yards”. I have set two small goals, which are:

  • What does the constructor of DexPathList do?
  • How does the findClass() method of DexPathList get a class?

Why these two goals? Because in the source code of BaseDexClassLoader the main use of the DexPathList constructor and findClass() method.

1) The constructor

private final Element[] dexElements; public DexPathList(ClassLoader definingContext, String dexPath, String libraryPath, File optimizedDirectory) { ... this.definingContext = definingContext; this.dexElements = makeDexElements(splitDexPath(dexPath), optimizedDirectory,suppressedExceptions); . }Copy the code

This constructor holds the current classloader definingContext and calls makeDexElements() to get the Element collection.

Through tracing the source of splitDexPath(dexPath), it is found that the function of this method is to convert all program files in the dexPath directory into a collection of files. And also found that dexPath is a colon (” : “) as a delimiter joined the multiple program files directory string (such as: / data/dexdir1: / data/dexdir2:…). .

The next step is to analyze the makeDexElements() method. Since this section of code is quite long, I post the key code and analyze it as a comment:

private static Element[] makeDexElements(ArrayList<File> files, File optimizedDirectory, ArrayList<IOException> suppressedExceptions) { // 1. ArrayList<Element> elements = new ArrayList<Element>(); For (File File: files) {ZipFile zip = null; // 2. DexFile dex = null; String name = file.getName(); . If (name.endswith (DEX_SUFFIX)) {dex = loadDexFile(file, optimizedDirectory); } else {zip = file; // If it is an apk, jar, or zip file (this part is a little different in Android versions)} else {zip = file; // If it is an apk, jar, or zip file (this part is a little different in Android versions)} dex = loadDexFile(file, optimizedDirectory); }... // 3. Wrap the dex file or zip file as an Element object and add it to the Element collection if ((zip! = null) || (dex ! = null)) { elements.add(new Element(file, false, zip, dex)); Return Element. ToArray (new Element[Element. Size ()]); // 4. }Copy the code

In general, the constructor of DexPathList wraps a program file (maybe dex, apk, JAR, zip) into an Element object and adds it to the Element collection.

In fact, Android class loaders (whether PathClassLoader or DexClassLoader), they only recognize the dex file, loadDexFile() is the core method to load the dex file, you can extract the dex from jar, APK, zip, and so on. But I’m not going to analyze it here, because the first goal is done, so I’ll do it later.

2) the findClass ()

Let’s look at the findClass() method of DexPathList:

public Class findClass(String name, List<Throwable> suppressed) { for (Element element : DexFile dex = element. DexFile; if (dex ! = null) {Class clazz = dex. LoadClassBinaryName (name, definingContext, suppressed); if (clazz ! = null) { return clazz; } } } if (dexElementsSuppressedExceptions ! = null) { suppressed.addAll(Arrays.asList(dexElementsSuppressedExceptions)); } return null; }Copy the code

The findClass() method of DexPathList is simple. It simply iterates through the Element array and returns null if it finds a class with the same name.

Why do you call the loadClassBinaryName() method of DexFile to load the class? This is because an Element object corresponds to a dex file, and a dex file contains multiple classes. The Element array contains a dex file instead of a class file. This can be seen in the source code of the Element class and the internal structure of the dex file.

Four, the realization principle of hot repair

After analyzing PathClassLoader, DexClassLoader, BaseDexClassLoader, And DexPathList, we know that The Android classloader loads a class by fetching (Element[] dexElements) from the Element array in its Own DexPathList object to the corresponding class and then loading it later. We’re going to iterate through an array, but notice that we’re going to iterate through a dex file.

In the for loop, we first iterate over the dex file, and then we get the class from the dex file, so we just pack the fixed class into a dex file and put it in the first Element of the Element array. This ensures that the class you get is the latest fixed class (of course, buggy classes do exist, but they are placed in the last Element of the Element array, so there is no chance of getting them).

5. Simple implementation of hotfix

After all that theory, it’s time to put it to the test.

1. Obtain the dex patch

1) Fix the faulty Java file

This step is to modify the code according to the actual situation of the bug.

2) Compile Java files into class files

After fixing the bug, you can use Android Studio’s Rebuild Project feature to compile the code and find the corresponding class file from the Build directory.

Copy the fixed class file to somewhere else, such as the dex folder on your desktop. Note that when copying this class file, you need to copy the entire package directory in which it resides. Hypothesis above repair good class files is SimpleHotFixBugTest. Class, then copy the directory structure is:

3) Package the class file into the dex file

A. dx instruction program

To package a class file into a dex file, you need the DX directive, which is similar to a Java directive. We know that Java directives include Javac, JAR, and so on. We can use these directives because we have installed the JDK, which provides us with Java directives. Similarly, dx directives need to be provided by a program. It’s in the Android SDK’s build-tools directory in the Android version directory.

B. Use of the dx instruction

Dx directives are used in the same way as Java directives. There are two options:

  • The environment variables are configured (added to the classpath), and the command line window (terminal) can be used anywhere.
  • Use the command line window (terminal) in the build-tools/ Android version directory without environment variables.

The first method refers to the Java environment variable configuration, and I choose the second method here. The following commands we need are:

Dx –dex –output=dex File full path (space) Directory where the complete class file to be packaged is stored, for example:

dx –dex –output=C:\Users\Administrator\Desktop\dex\classes2.dex C:\Users\Administrator\Desktop\dex

See the following figure for specific operations:

In the blank area of the folder directory, hold down shift+ right click to display “open command line window here”.

2. Load the dex patch

Based on the principle, you can make a simple utility class:

/** * @creator CSDN_LQR * @ Description of hotfix tools (only fixes with the suffix dex, apk, JAR, and zip are recognized) */ Public class FixDexUtils {private static final String DEX_SUFFIX = ".dex"; private static final String APK_SUFFIX = ".apk"; private static final String JAR_SUFFIX = ".jar"; private static final String ZIP_SUFFIX = ".zip"; public static final String DEX_DIR = "odex"; private static final String OPTIMIZE_DEX_DIR = "optimize_dex"; private static HashSet<File> loadedDex = new HashSet<>(); static { loadedDex.clear(); } /** * load the patch, use the default directory: Data /data/ package name /files/odex * * @param context */ public static void loadFixedDex(context context) {loadFixedDex(context, null); } /** * Load patch ** @param Context context * @param patchFilesDir Patch directory */ public static void loadFixedDex(Context context, File patchFilesDir) { if (context == null) { return; } dex File fileDir = patchFilesDir! = null ? patchFilesDir : new File(context.getFilesDir(), DEX_DIR); // data/data/ package name /files/odex (this can be anywhere) File[] listFiles = filedir.listFiles (); for (File file : listFiles) { if (file.getName().startsWith("classes") && (file.getName().endsWith(DEX_SUFFIX) || file.getName().endsWith(APK_SUFFIX) || file.getName().endsWith(JAR_SUFFIX) || file.getName().endsWith(ZIP_SUFFIX))) { loadedDex.add(file); }} // dex doDexInject(context, loadedDex); } private static void doDexInject(Context appContext, HashSet<File> loadedDex) { String optimizeDir = appContext.getFilesDir().getAbsolutePath() + File.separator + OPTIMIZE_DEX_DIR; // data/data/ package name /files/optimize_dex (this must be a directory in your program) File fopt = new File(optimizeDir); if (! fopt.exists()) { fopt.mkdirs(); } try {/ / 1. Load the application of dex PathClassLoader pathLoader = (PathClassLoader) appContext. GetClassLoader (); for (File dex : loadedDex) { // 2. Load the specified fixed dex file. DexClassLoader dexLoader = New DexClassLoader(dex.getabSolutePath (),// The directory where the fixed dex (patch) is located Fopt.getabsolutepath (),// Store the decompression directory of dex (used for JAR, ZIP, and APK format patches) null,// Library pathLoader required for loading dex // parent class loader); // 3. Merge Object dexPathList = getPathList(dexLoader); Object pathPathList = getPathList(pathLoader); Object leftDexElements = getDexElements(dexPathList); Object rightDexElements = getDexElements(pathPathList); Object dexElements = combineArray(leftDexElements, rightDexElements); // Override Element in PathList [] dexElements; Assign Object pathList = getPathList(pathLoader); SetField (pathList, pathList.getClass(), "dexElements", dexElements); } } catch (Exception e) { e.printStackTrace(); */ private static void setField(Object obj, Class<? > cl, String field, Object value) throws NoSuchFieldException, IllegalAccessException { Field declaredField = cl.getDeclaredField(field); declaredField.setAccessible(true); declaredField.set(obj, value); */ private static Object getField(Object obj, Class<? > cl, String field) throws NoSuchFieldException, IllegalAccessException { Field localField = cl.getDeclaredField(field); localField.setAccessible(true); return localField.get(obj); } /** * Reflect the pathList Object in the classloader */ Private static Object getPathList(Object baseDexClassLoader) throws ClassNotFoundException, NoSuchFieldException, IllegalAccessException { return getField(baseDexClassLoader, Class.forName("dalvik.system.BaseDexClassLoader"), "pathList"); */ Private static Object getDexElements(Object pathList) throws NoSuchFieldException, IllegalAccessException { return getField(pathList, pathList.getClass(), "dexElements"); } private static Object combineArray(Object arrayLhs, Object arrayRhs) {Class<? > componentType = arrayLhs.getClass().getComponentType(); int i = Array.getLength(arrayLhs); Int j = array.getLength (arrayRhs); Int k = I + j; Object result = Array. NewInstance (componentType, k); System. arrayCopy (arrayLhs, 0, result, 0, I); // Create a new array with type componentType and length k. System.arraycopy(arrayRhs, 0, result, i, j); return result; }}Copy the code

The code is long, but it’s clearly commented out, so look carefully, and there are two points here:

1) Class ref in pre-verified Class resolved to unexpected implementation

After feedback, this is a problem that everyone encountered the most, here I put the matters needing attention and my solution to write clearly:

a.FixDexUtils

Object dexElements = combineArray(leftDexElements, rightDexElements); // Override Element in PathList [] dexElements; Assign Object pathList = getPathList(pathLoader); SetField (pathList, pathList.getClass(), "dexElements", dexElements);Copy the code

After merging the Element array, be sure to retrieve the original pathList in the App again. Do not reuse the previous pathList. Class ref in pre-verified Class resolved to unexpected implementation

b.Instant Run

The Instant Run function of Android Studio also uses the principle of hot fix. When the app is reinstalled, it will not be completely installed, but only dynamically modify the updated class part, which will affect the test results. If you are following the experiment in this article, please make sure that the Instant Run is closed.

C. the simulator

I used Genymotion during the test, and found that the Android 4.4 emulator was not able to patch, but the Android 5.0 emulator was ok, and the real test was ok, so it is not recommended to use the Android 5.0 emulator to test, but strongly recommended to use the real test!!

2) dexPath and optimizedDirectory directory problem

DexClassLoader dexLoader = New DexClassLoader(dex.getabSolutePath (),// The directory where the dex (patch) is located is fopt.getabSolutePath (),// The decompression directory of the DEX (used for jar, ZIP, and APK patches) is null. // Library pathLoader required for loading the DEX // Parent class loaderCopy the code

The above code creates a DexClassLoader object with one detail in the first and second arguments:

  • Parameter 1 is dexPath, which refers to all directories of the patch. It can be multiple directories (collated with colons), and it can be any directory, such as an SD card.
  • Parameter 2 is optimizedDirectory, which is the directory for storing the dex file decompressed from the compressed package. However, it cannot be any directory. It must be the directory where the program belongs, such as data/data/ package name/XXX.

If you specify optimizedDirectory as the SD card directory, the following error will be reported:

java.lang.IllegalArgumentException: Optimized data directory /storage/emulated/0/opt_dex is not owned by the current user. Shared storage cannot protect your application from code injection attacks.

The SD card directory does not belong to the current user. OptimizedDirectory not only stores the dex file from the compressed package, but also copies the patch file to the optimizedDirectory directory if the patch file is a dex file.

3. Load the jar, APK, and ZIP patches

The DexClassLoader can load patch files in jar, APK, and ZIP formats. What are the requirements for such patch files? The answer is: this type of compression package must put a dex file, and the name of the requirements, must be classes.dex. According to? This requires analyzing the loadDexFile() method in the DexPathList class.

private static DexFile loadDexFile(File file, Throws IOException {// If optimizedDirectory is null, If (optimizedDirectory == null) {return new DexFile(file); } // If the optimizedDirectory is not null, this is how DexClassLoader loads the dex file. Else {String optimizedPath = optimizedPathFor(file, optimizedDirectory); return DexFile.loadDex(file.getPath(), optimizedPath, 0); }}Copy the code

Parameter 1: file: dex file, JAR file, apk file, zip file.

The else branch is used by the DexClassLoader to load the dex file. It calls the optimizedPathFor() method to retrieve the dex file from the optimizedDirectory directory.

private static String optimizedPathFor(File path, File optimizedDirectory) { String fileName = path.getName(); if (! fileName.endsWith(DEX_SUFFIX)) { int lastDot = fileName.lastIndexOf("."); // If the patch does not have a suffix, add a ".dex" suffix if (lastDot < 0) {fileName += DEX_SUFFIX; } // If the patch suffix is dex, JAR, apk or zip, Else {StringBuilder sb = new StringBuilder(lastDot + 4); else {StringBuilder sb = new StringBuilder(lastDot + 4); sb.append(fileName, 0, lastDot); sb.append(DEX_SUFFIX); fileName = sb.toString(); } } File result = new File(optimizedDirectory, fileName); return result.getPath(); }Copy the code

As mentioned above, the Android classloader will only recognize dex files eventually, even if the patch is a jar, apK, zip, etc., it will extract the dex file, so this method must get a file name ending in dex. Ok, this optimizedPathFor() method is not the point, but if you look back at the else branch of loadDexFile() there is also a dexfile.loaddex () method, which is quite important.

static public DexFile loadDex(String sourcePathName, String outputPathName, int flags) throws IOException {
    return new DexFile(sourcePathName, outputPathName, flags);
}
Copy the code

This method calls its own constructor, passing in the arguments, and then looks at the DexFile constructor:

/** * Open a DEX file, specifying the file in which the optimized DEX * data should be written. If the optimized form exists and appears * to be current, it will be used; if not, the VM will attempt to * regenerate it. * * This is intended for use by applications that wish to download * and execute  DEX files outside the usual application installation * mechanism. This function should not be called directly by an * application; instead, use a class loader such as * dalvik.system.DexClassLoader. * * @param sourcePathName * Jar or APK file with "classes.dex". (May expand this to include * "raw DEX" in the future.) * @param outputPathName * File that will hold the  optimized form of the DEX data. * @param flags * Enable optional features. (Currently none defined.) * @return * A new or previously-opened DexFile. * @throws IOException * If unable to open the source or output file. */ private DexFile(String sourceName, String outputName, int flags) throws IOException { if (outputName ! = null) { try { String parent = new File(outputName).getParent(); if (Libcore.os.getuid() ! = Libcore.os.stat(parent).st_uid) { throw new IllegalArgumentException("Optimized data directory " + parent + " is not owned by the current user. Shared storage cannot protect" + " your application from code injection attacks."); } } catch (ErrnoException ignored) { // assume we'll fail with a more contextual error later } } mCookie = openDexFile(sourceName, outputName, flags); mFileName = sourceName; guard.open("close"); //System.out.println("DEX FILE cookie is " + mCookie + " sourceName=" + sourceName + " outputName=" + outputName); }Copy the code

Oddly enough, I didn’t leave the constructor’s comment out this time because it already has the desired answer in the comment:

@param sourcePathName Jar or APK file with "classes.dex".  (May expand this to include "raw DEX" in the future.)
Copy the code

This comment means that a patch file in jar or APK format needs to have a classes.dex. At this point, the requirements for patch files in compressed format are clear. All you need to do is generate patches in these formats and give them a try. Making this kind of compressed file is also very simple, directly with the compression software compressed into a ZIP file, and then change the suffix can be.

Six, test,

I didn’t want to write this part because it was easy, but I didn’t think it was complete, so let’s test it.

1, code,

1) the Activity

Layout file has two buttons, very simple do not paste the layout file code, look at the two buttons click on the event.

public class SimpleHotFixActivity extends AppCompatActivity { @Override protected void onCreate(@Nullable Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_simple_hot_fix); } public void fix(View View) {fixdexutils.loadFixeddex (this, Environment.getExternalStorageDirectory()); Public void clac(View View) {SimpleHotFixBugTest test = new SimpleHotFixBugTest(); test.getBug(this); }}Copy the code

As you can see, the “Fix” button click event is to load the patch file in the SD card directory.

2) SimpleHotFixBugTest

public class SimpleHotFixBugTest { public void getBug(Context context) { int i = 10; int a = 0; Toast.makeText(context, "Hello,I am CSDN_LQR:" + i / a, Toast.LENGTH_SHORT).show(); }}Copy the code

What’s going to happen? The divisor is a zero exception, a simple runtime exception, and fixing it is as simple as changing the value of a to non-zero.

2, presentations,

1, bug

I don’t have to talk about it.

ArithmeticException, no problem.

Caused by: java.lang.ArithmeticException: divide by zero

2. Dynamically fix bugs

First, I put the patch file classes2.dex in the SD directory of the phone.

Then click the Fix button first and then the Calculate button.

The patch in compressed format is the same as the patch in dex format. You can directly discard the patch in the SD card directory, but be sure to note that the file in compressed format must be classes.dex!!

Finally paste the Demo address

Github.com/GitLqr/HotF…

Permission application: The Demo provided in this article is to read the patch file under SD card, but it does not apply for dynamic permission for Android6.0 or above. If you use this Demo for testing, you should pay attention to the Android version of your test machine. If 6.0 or above, please be sure to assign SD card read and write operation permission to Demo first. Otherwise, you won’t know if the App crashes because of a bug. Remember that.