1 The origin of MultiDex

In Android, a DEX file stores 65536 methods at most, that is, a range of short type, so with the increasing number of application classes, when a DEX file breaks the number of methods, an exception will be reported. Although useless methods can be reduced through obfuscation and other methods, with the increase of APP functions, it is inevitable to break through the limit of method number. So with Android5.0, Android introduced an official solution: MultiDex. When packaging, divide an application into multiple dex, such as class. dex, classes2.dex, classes3.dex… , these dex are appended to the corresponding DexPathList array when loading, so as to solve the limitation of method number.

After 5.0, the system has the built-in function of loading multiple dex files, while before 5.0, the system can only load one main dex, and the other dex needs to be loaded by certain means. That’s what we’re going to talk about today, MultiDex.

MultiDex is stored in the Android.support. MultiDex package.

2 Use of MultiDex

In Gradle build, add the following configuration to the build. Gradle folder of the main application:

defaultConfig { ... multiDexEnabled true ... } dependencies {the compile 'com. Android. Support: multidex: 1.0.1'... }Copy the code

The latest version of Multidex is now 1.0.2.

Under the app node in Androidmanifest.xml, use MultiDexApplication as the application entry.

package android.support.multidex; . public class MultiDexApplication extends Application { @Override protected void attachBaseContext(Context base) { super.attachBaseContext(base); MultiDex.install(this); }}Copy the code

Of course, most of the time, we’ll customize our own Application to do some initialization. In this case, the multidex.install () method can be called in the attachBaseContext() method of our custom Application.

# Override protected void attachBaseContext(Context base) {super.attachBasecontext (base); MultiDex.install(this); }Copy the code

Note that the multidex.install () method should be called as early as possible to prevent ClassNotFoundException in the dex file that loads later.

3 MultiDex source analysis

The entry point for analyzing MultiDex is its static method install(). This method adds the dex from the application’s APK file to the Emlement array of the DexPathList in the application’s classloader, PathClassLoader.

public static void install(Context context) { Log.i(TAG, "install"); // Check whether the Android system already supports MultiDex. If it does, there is no need to install it. Return if (IS_VM_MULTIDEX_CAPABLE) {log. I (TAG, "VM has multidex support, multidex support library is disabled. return; If (build.version.sdk_int < MIN_SDK_VERSION) {throw new RuntimeException("Multi dex installation failed. SDK " + Build.VERSION.SDK_INT + " is unsupported. Min SDK version is " + MIN_SDK_VERSION + ". "); } try {// Get the application information ApplicationInfo ApplicationInfo = getApplicationInfo(context); Return if the application information is empty, such as running in a test Context. if (applicationInfo == null) { // Looks like running on a test Context, so just return without patching. return; } / / synchronization method synchronized (installedApk) {/ / access has been installed the full path of the APK String apkPath = applicationInfo. SourceDir; if (installedApk.contains(apkPath)) { return; } // add the path to the installedApk path installedapk. add(apkPath); If the compiled version is greater than the maximum supported version, If (build.version.sdk_int > MAX_SUPPORTED_SDK_VERSION) {log.w (TAG, "MultiDex is not guaranteed to work in SDK version " + Build.VERSION.SDK_INT + ": SDK version higher than " + MAX_SUPPORTED_SDK_VERSION + " should be backed by " + "runtime with built-in multidex capabilty but it's not the " + "case here: java.vm.version=\"" + System.getProperty("java.vm.version") + "\""); } /* The patched class loader is expected to be a descendant of * dalvik.system.BaseDexClassLoader. We modify its * dalvik.system.DexPathList pathList field to append additional DEX * file entries. */ ClassLoader loader; Try {// getClassLoader, which is actually PathClassLoader = context.getclassloader (); } catch (RuntimeException e) { /* Ignore those exceptions so that we don't break tests relying on Context like * a android.test.mock.MockContext or a android.content.ContextWrapper with a * null base Context. */ Log.w(TAG, "Failure while trying to obtain Context class loader. " + "Must be running in test mode. Skip patching.", e); return; If (loader == null) {// Note, the context class loader is null when running Robolectric tests. Log.e(TAG, "Context class loader is null. Must be running in test mode. " + "Skip patching."); return; } try {// Clear the old cached Dex directory, the source cache directory is "/data/user/0/${packageName}/files/secondary-dexes" clearOldDexDir(context); } catch (Throwable t) { Log.w(TAG, "Something went wrong when trying to clear old MultiDex extraction, " + "continuing without cleaning.", t); ${packageName}/code_cache/secondary-dexes"; /data/user/0/${packageName}/code_cache/secondary-dexes File dexDir = new File(applicationInfo.dataDir, SECONDARY_FOLDER_NAME); // Use the MultiDexExtractor utility class to extract the dex from APK into the dexDir directory. The returned files may be empty, indicating that there is no secondaryDex. List<File> files = multidexextractor. load(context, applicationInfo, dexDir, false); If (checkValidZipFiles(files)) {// Install secondaryDex installSecondaryDexes(loader, dexDir, files); } else { Log.w(TAG, "Files were not valid zip files. Forcing a reload."); // Try again, but this time force a reload of the zip file. There is some IO overhead files = multidexextractor. load(context, applicationInfo, dexDir, true); If (checkValidZipFiles(files)) {installSecondaryDexes(loader, dexDir, files); } else { // Second time didn't work, give up throw new RuntimeException("Zip files were not valid."); } } } } catch (Exception e) { Log.e(TAG, "Multidex installation failure", e); throw new RuntimeException("Multi dex installation failed (" + e.getMessage() + ")."); } Log.i(TAG, "install done"); }Copy the code

Setting aside the dex extraction and validation logic, let’s look at how MultiDex installs secondaryDex. The class loading mechanism is somewhat different for different versions of Android, so there are three types of installation: V19, V14 and V4. V19, V14, and V4 are all private static inner classes of MultiDex. V19 supports Android 19 (20 is wearable only), V14 supports versions 14, 15, 16, 17, and 18, and V4 supports versions 4 through 13.

# android.support.multidex.MultiDex private static void installSecondaryDexes(ClassLoader loader, File dexDir, List<File> files) throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, InvocationTargetException, NoSuchMethodException, IOException { if (! files.isEmpty()) { if (Build.VERSION.SDK_INT >= 19) { V19.install(loader, files, dexDir); } else if (Build.VERSION.SDK_INT >= 14) { V14.install(loader, files, dexDir); } else { V4.install(loader, files); }}}Copy the code

Let’s take a look at the source code of V19

/** * Installer for platform versions 19. */ private static final class V19 { private static void install(ClassLoader loader, List<File> additionalClassPathEntries, File optimizedDirectory) throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, InvocationTargetException, NoSuchMethodException { /* The patched class loader is expected to be a descendant of * dalvik.system.BaseDexClassLoader. We modify its * dalvik.system.DexPathList pathList field to append additional DEX * File entries. */ / the loader passed is PathClassLoader, FindFidld () : findFidld() : findFidld() : findFidld() : findFidld() : findFidld() : findFidld() "pathList"); Object DexPathList = pathListField.get(loader); ArrayList<IOException> suppressedExceptions = new ArrayList<IOException>(); // Expand the Element array of the DexPathList object, The name of the array is dexElements. The makeDexElements() method is used to create the dex element expandFieldArray by calling the makeDexElements() method of DexPathList. "dexElements", makeDexElements(dexPathList, new ArrayList<File>(additionalClassPathEntries), optimizedDirectory, suppressedExceptions)); // Add some IO exceptions, because makeDexElements calling DexPathList will have some IO operations, Corresponding might have some abnormal situation if (suppressedExceptions. The size () > 0) {for (IOException e: suppressedExceptions) { Log.w(TAG, "Exception in makeDexElement", e); } Field suppressedExceptionsField = findField(loader, "dexElementsSuppressedExceptions"); IOException[] dexElementsSuppressedExceptions = (IOException[]) suppressedExceptionsField.get(loader); if (dexElementsSuppressedExceptions == null) { dexElementsSuppressedExceptions = suppressedExceptions.toArray( new IOException[suppressedExceptions.size()]); } else { IOException[] combined = new IOException[suppressedExceptions.size() + dexElementsSuppressedExceptions.length];  suppressedExceptions.toArray(combined); System.arraycopy(dexElementsSuppressedExceptions, 0, combined, suppressedExceptions.size(), dexElementsSuppressedExceptions.length); dexElementsSuppressedExceptions = combined; } suppressedExceptionsField.set(loader, dexElementsSuppressedExceptions); } } /** * A wrapper around * {@code private static final dalvik.system.DexPathList#makeDexElements}. */ // Private static Object[] makeDexElements() makeDexElements(); Object dexPathList, ArrayList<File> files, File optimizedDirectory, ArrayList<IOException> suppressedExceptions) throws IllegalAccessException, InvocationTargetException, Method makeDexElements = findMethod(DexPathList, "makeDexElements", ArrayList.class, File.class, ArrayList.class); // Call makeDexElements(), Return (Object[]) makeDexElements. Invoke (dexPathList, files, optimizedDirectory, suppressedExceptions); }}Copy the code

The expandFieldArray() method of MultiDex extends elements in an array of objects. It’s actually a tool method. Take a quick look at the source code:

# android.support.multidex.MultiDex
private static void expandFieldArray(Object instance, String fieldName,
        Object[] extraElements) throws NoSuchFieldException, IllegalArgumentException,
        IllegalAccessException {
    Field jlrField = findField(instance, fieldName);
    Object[] original = (Object[]) jlrField.get(instance);
    Object[] combined = (Object[]) Array.newInstance(
            original.getClass().getComponentType(), original.length + extraElements.length);
    System.arraycopy(original, 0, combined, 0, original.length);
    System.arraycopy(extraElements, 0, combined, original.length, extraElements.length);
    jlrField.set(instance, combined);
}
Copy the code

After the install() method of V19 is called, the dex file in addition to the main dex file in the APK file is appended to the DexPathListde Element[] array in the PathClassLoader (BaseClassLoader). This allows all dex files to be traversed when a class is loaded, ensuring that the packaged classes are properly loaded.

As for the install() method in V14 and V4, the main idea is the same, there are some differences in the details, interested in the relevant source code can be viewed.

To summarize: The install() method of MultiDex actually extracts the.dex file from the APK file, then uses reflection to generate the corresponding array of the.dex file, and finally appends the dex path to the path where the PathClassLoader loaded the dex. This ensures that all classes in APK. Dex files can be loaded correctly.

Load the secondartDex file from the APK file. Take a look at the load() method for MultiDexExtractor:

# android.support.multidex.MultiDexExtractor static List<File> load(Context context, ApplicationInfo applicationInfo, File dexDir, boolean forceReload) throws IOException { Log.i(TAG, "MultiDexExtractor.load(" + applicationInfo.sourceDir + ", " + forceReload + ")"); / / sourceDir path "/ data/app /} ${packageName} - 1 / base. Apk" final File sourceApk = new File (applicationInfo. SourceDir); Long currentCrc = getZipCrc(sourceApk); List<File> files; // The isModified() method is based on the last modified timestamp of the APK file stored in SharedPreference and currentCrc to determine if the file has been modified. forceReload && ! IsModified (context, sourceApk, currentCrc)) {try {// Load the extracted files from the cache directory. Files = loadExistingExtractions(context, sourceApk, dexDir); } catch (IOException ioe) { Log.w(TAG, "Failed to reload existing extracted secondary dex files," + " falling back to fresh extraction", ioe); Files = performExtractions(sourceApk, dexDir); files = performExtractions(sourceApk, dexDir); PutStoredApkInfo (Context, getTimeStamp(sourceApk), currentCrc, files.size() + 1); }} else {// Reextract dex file log. I (TAG, "Detected that extraction must be performed.") if forced loading or APK file has been modified; files = performExtractions(sourceApk, dexDir); putStoredApkInfo(context, getTimeStamp(sourceApk), currentCrc, files.size() + 1); } Log.i(TAG, "load found " + files.size() + " secondary dex files"); return files; }Copy the code

The performExtractions() method of MultiDexExtractor is used to extract the dex file from APK for the first time.

# android.support.multidex.MultiDexExtractor private static List<File> performExtractions(File sourceApk, File dexDir) throws IOException {// extractedFilePrefix = "${apkName}. Classes "final String extractedFilePrefix = sourceApk.getName() + EXTRACTED_NAME_EXT; // Ensure that whatever deletions happen in prepareDexDir only happen if the zip that // contains a secondary dex file in there is not consistent with the latest apk. Otherwise, // multi-process race conditions can cause a crash loop where one process deletes the zip // while another had created // Since the dexDir cache directory may be used by more than one APK, if there are cached dex files related to APK before extracting an APK, it is necessary to delete them first. If the dexDir directory does not exist create prepareDexDir(dexDir, extractedFilePrefix); List<File> files = new ArrayList<File>(); final ZipFile apk = new ZipFile(sourceApk); try { int secondaryNumber = 2; // Get classes${secondaryNumber}. Dex ZipEntry dexFile = apk.getEntry(DEX_PREFIX + secondaryNumber + DEX_SUFFIX); // If dexFile is not null, keep iterating while (dexFile! ${apkName}. Classes ${secondaryNumber}. Zip "String fileName = extractedFilePrefix + secondaryNumber + EXTRACTED_SUFFIX; File extractedFile = new File(dexDir, fileName); Files.add (extractedFile); Log.i(TAG, "Extraction is needed for file " + extractedFile); Int numAttempts = 0; boolean isExtractionSuccessful = false; while (numAttempts < MAX_EXTRACT_ATTEMPTS && ! isExtractionSuccessful) { numAttempts++; // Create a zip file (extractedFile) containing only the secondary dex file // (dexFile) from the apk. // Extract the dex file with corresponding serial number from APK and store it in the extractedFile zip file. Extract (apk, dexFile, extractedFile, extractedFilePrefix); // Verify that the extracted file is indeed a zip file. // Verify that the extracted file is indeed a zip file verifyZipFile(extractedFile); // Log the sha1 of the extracted zip file Log.i(TAG, "Extraction " + (isExtractionSuccessful ? "success" : "failed") + " - length " + extractedFile.getAbsolutePath() + ": " + extractedFile.length()); if (! isExtractionSuccessful) { // Delete the extracted file extractedFile.delete(); if (extractedFile.exists()) { Log.w(TAG, "Failed to delete corrupted secondary dex '" + extractedFile.getPath() + "'"); } } } if (! isExtractionSuccessful) { throw new IOException("Could not create zip file " + extractedFile.getAbsolutePath() + " for secondary dex (" + secondaryNumber + ")"); } // continue the extraction of the next dex secondaryNumber++; dexFile = apk.getEntry(DEX_PREFIX + secondaryNumber + DEX_SUFFIX); } } finally { try { apk.close(); } catch (IOException e) { Log.w(TAG, "Failed to close resource", e); } } return files; }Copy the code

When performExtractions() method of MultiDexExtractor is finished when all dex files in APK are extracted, and a zip file with a certain file name format is saved in the cache directory. Then some key information is saved to SP by calling putStoredApkInfo(Context Context, Long timeStamp, Long CRC, int totalDexNumber).

When APK is restarted after, the dex file that has been extracted will be loaded from the cache directory. Let’s look at the LoadeXistingExtractor () method:

# android.support.multidex.MultiDexExtractor private static List<File> loadExistingExtractions(Context context, File sourceApk, File dexDir) throws IOException { Log.i(TAG, "loading existing secondary dex files"); ${apkName}.classes" Final String extractedFilePrefix = sourceapk.getName () + EXTRACTED_NAME_EXT; // Get the total number of. Dex files from SharedPreferences Int totalDexNumber = getMultiDexPreferences(context).getint (KEY_DEX_NUMBER, 1); final List<File> files = new ArrayList<File>(totalDexNumber); For (int secondaryNumber = 2; int secondaryNumber = 2; secondaryNumber <= totalDexNumber; SecondaryNumber++) {// file name, ${apkName}.classes${secondaryNumber}.zip" String fileName = extractedFilePrefix + secondaryNumber + EXTRACTED_SUFFIX; File extractedFile = new File(dexDir, fileName); If (extractedfile.isfile ()) {files.add(extractedFile); if (! verifyZipFile(extractedFile)) { Log.i(TAG, "Invalid zip file: " + extractedFile); throw new IOException("Invalid ZIP file."); } } else { throw new IOException("Missing extracted secondary dex file '" + extractedFile.getPath() + "'"); } } return files; }Copy the code

4 summarizes

It is necessary to extract the dex files and append the dex files to the Element[] array of the DexPathList as early as possible. This is typically in the attachBaseContext() method of the Application. Some hot-fix techniques allow pre-loading of the repaired class by inserting the repaired dex in front of the Element[] array of the DexPathList.

reference

  • Introduction to automatic unpacking and dynamic loading of Meituan Android DEX
  • Talk about MultiDex startup optimization
  • MultiDex working principle analysis and optimization scheme