background

Those of you who read my blog should know that I wrote a little gradle plugin in May 2018 github.com/yanbober/ap… , its function is to perform inline operations on the R constant generated by APP. That’s right, the R resource inline principle provided by Didi Booster and ByteX.

In these two days, as the project will be upgraded to adapt AGP4.1.0, we will investigate the changes of R of intermediates products in AGP4.1.0 to build pairs of submodules and synthesize the final APP. In this process, I accidentally discovered an interesting and profound thing, which was extremely scary when I thought about it, but the more I thought about it, the more interesting I became. Good accidents often make good stories.

I’ll keep you in suspense

As we all know, aapT2 generate the corresponding R.class file (AGP4.1.0 intermediates directly into JARS, Compile_and_runtime_not_namespaced_r_class_jar) and eventually merge into the dex. This is not clear enough to look into the little project I mentioned in my background.

Now I have some soul searching questions for you:

  1. What is the format of R files generated by resources? What are the differences between modules? (Check out my little project REDME if you can’t answer it.)
  2. Can methods or fields exceed 65535 when using the official Multidex scheme? What is the essential reason?
  3. Is there an upper limit on the number of App resources (including string number, etc.)? Why is that?
  4. Would it be a problem if I wrote a class with a field larger than 65535? Can it be used in official Multidex scenarios?

Can you elaborate on these four soul questions? If not, please continue to read, take you to play wave fun things.

Restore the scene

Where are you going to get the resources to blow up 65535 fields? Open my IDE and create a new createR. Sh file with the following contents (don’t worry about using the original echo, because it is lazy, it will work) :

#! /bin/bash
# encoding=utf-8
# 【 craftsman if water to add WeChat yanbo373131686 contact me, concern WeChat public number: code farmers a daily topic Without permission is strictly prohibited reproduced, https://blog.csdn.net/yanbober]

echo "start generate string_r.xml";

file_name="string_r.xml";

echo "
      " >> $file_name;
echo "<resources>" >> $file_name;

for((i=1; i<=65536; i++));do
  echo "<string name=\"public_r_$i\">TEST-$i</string>" >> $file_name;
done

echo "</resources>" >> $file_name;
echo "generate string_r.xml success!";
Copy the code

Save, hit enter, execute a wait, go to the toilet with pay shit for a while, come back file OK. 65,536 string resources in total:

To expose all the problems at once, just drop the resource file into the values of the main module, then click on the Android Studio run, and there it is:

You probably didn’t expect the error, did you? Why did this happen? An exception was raised when the resource merge task was executed. In fact, this resource merge task does many things, one of the important things is to generate the merged R file through ASM.

You might ask, where does it look like it’s generated by ASM? Do you believe what I see when I download from AGP? It’s really easy to verify, you just add -s to the build, so that when something goes wrong, you have a detailed call stack, and you can clearly see that the call relationship is ASM generating bytecode.

What is Class too large? Do you think you made a judgment in the AGP source code similar to the judgment in the AGP build of Multidex? . To tell you clearly, no, do not believe you to search AGP source code, what also can not search. So what’s going on?

The secret truth

Now let’s uncover what Class too large is step by step! We add -s to the build to see the following stack:

The cargo is in the task using the ASM bytecode generated times of wrong, so as shown above, go directly to the ASM search inside, indeed as expected to ha. Why is ASM limited to no more than 0xFFFF? If you’re not familiar with the basics of the JVM, read on and take a look at this ASM source code comment:

/ / 【 craftsman if water to add WeChat yanbo373131686 contact me, concern WeChat public number: code farmers a daily topic Without permission is strictly prohibited reproduced, https://blog.csdn.net/yanbober]

package org.objectweb.asm;

/**
 * A {@link ClassVisitor} that generates classes in bytecode form. More
 * precisely this visitor generates a byte array conforming to the Java class
 * file format. It can be used alone, to generate a Java class "from scratch",
 * or with one or more {@link ClassReader ClassReader} and adapter class visitor
 * to generate a modified class from one or more existing Java classes.
 * 
 * @author Eric Bruneton
 */
public class ClassWriter extends ClassVisitor {
    /** * Index of the next item to be added in the constant pool. */
    int index;
  
    /**
     * Returns the bytecode of the class that was build with this class writer.
     * 
     * @return the bytecode of the class that was build with this class writer.
     */
    public byte[] toByteArray() {
        if (index > 0xFFFF) {
            throw new RuntimeException("Class file too large!"); }... }}Copy the code

Index of the next item to be added in the constant pool. Index of the next item to be added in the constant pool. Constant pool, ha ha, that’s the fucking truth.

Still not understand? That goes to repair JVM foundation bar, zhou teacher’s god book a few chapters is enough!

Here are two direct links to the simple popular science constant pool:

Java Class File Structure: Constant Pool Java Class File Structure: Constant Pool

The answer to the soul torture

Now that we’ve got the context and the nature of the problem, let’s dive into the next mystery.

  1. What is the format of R files generated by resources? What are the differences between modules?

    Go to the github.com/yanbober/ap… REDME, that’s a lot of depth.

  2. Can methods or fields exceed 65535 when using the official Multidex scheme? What is the essential reason?

    Obviously, with the official Multidex scheme there will be no method or field exceeding 65535 because the JVM layer already limits the rules of play. The essence is the secret truth above.

  3. Is there an upper limit on the number of App resources (including string number, etc.)? Why is that?

    When a single class (string/anim/drawable, etc.) resource is merged, the total number of constant pools in a single class file cannot exceed 65536. Otherwise, the corresponding R cannot be generated because the constant pool explodes when ASM bytecode generates the merge R.

  4. Would it be a problem if I wrote a class with a field larger than 65535? Can it be used in official Multidex scenarios?

    There will be problems, will not compile, does not comply with the JVM Class specification. Since it does not compile, it does not exist in the official multidex scenario, because it will die if it cannot reach the dex step.

Will encounter the end of the world

You think this is the end of the story? The tiger body was shocked to reflect: “Will this pot like the multidex in the future carrier level app crash at some point?” The answer is yes, but not in the short term, because in order to reach this bottleneck, we need to cross the boundary of single type of resources. In fact, there are few apps to reach this size, even the carrier app is not easy to reach, unless you have a fine.

The essence of the problem actually goes back to Google’s official attitude towards R, which has been changing a little but has not managed to fix the root cause. In ancient APT times, field in R of submodule is static final constant. Later, In order to speed up construction, Google made submodule non-static final (resulting in some annotations framework to make their own R2). The main module is composed, and then the main module makes a static final, while keeping the non-static final of the child module, which has been called r.Java until now. The r.jar class jar is now in place in one step, and non-static final attributes of child modules are no longer assigned random values. This version is still not a cure.

What if one day multidex does go the same way? A good way to think about it would be for Google to step in and optimize it. Otherwise, as a tripartite APP, we may only be able to operate, and the two operations we currently think of are:

  • Solution 1: similar to plug-in, create multiple apK resources, hook resource loading cheat, but this is just my first YY, because I know the cheat Class constant pool limit, will be the resource loading block also buried, so it is not fun.
  • Scheme 2: Create a set of resources like Java resources, no longer participate in AAPT2 compilation, but directly participate in Java compilation and packaging, and then drag through the multi-language mapping scene, reserving a nameId for external access and docking with Android resource management class. This seems to work, but package size and performance must matter, or Google wouldn’t have made itresources.arscAnd R indexed.

As above is purely their own YY, look at it, don’t be serious.

conclusion

Right, was going to see other problems, was brought to think about this problem, ok. As you can see, in fact, the problem is not complicated, not difficult, a little with the code can know what is going on, just a little, do things also have to calm down, so that you can think deeply, and then can be thoughtful.

Day arch a pawn, work does not donate tang. Today arch, ha ha.