In the Android hotpatch framework Robust, several important processes include:

• Patch Loading Process
• Foundation pile insertion process
• Patch package generation process

In the last article analysis of the Robust principle of Android hot patches (I), we analyzed the first two, the patch loading process and the basic packing pile process, and the analysis version is 0.3.2. This article, the second in the series, examines the process of automated patch generation in version 0.4.82.

It’s a bit of a time span…

Series of articles:

• Robust Principle Analysis of Android Hot Patch
• Robust analysis of Android Hot Patch (2)
• Robust (3) pit settlement for Android Hot patch

The whole process

First, a Tranform named AutoPatchTranform is registered in the Gradle plug-in

class AutoPatchTransform extends Transform implements Plugin<Project> {
@Override
void apply(Project target) {
    initConfig();
    project.android.registerTransform(this)}def initConfig(a) { NameManger.init(); InlineClassFactory.init(); ReadMapping.init(); Config.init(); . ReadXML.readXMl(project.projectDir.path); Config.methodMap = JavaUtils.getMapFromZippedFile(project.projectDir.path + Constants.METHOD_MAP_PATH) }Copy the code

This class is the entrance to the plug-in, implements GradlePlugin and inherits from Transform, initializes the configuration and registers the Transform at the entrance. The configuration mainly includes reading Robust XML configuration, confusing optimized mapping file, methodsMap. Robust file generated in the process of interpolation, initializing inline factory class and so on. And then the main thing is the transform method

@Override
void transform(Context context, Collection<TransformInput> inputs, Collection<TransformInput> referencedInputs, TransformOutputProvider outputProvider, boolean isIncremental) throws IOException, TransformException, InterruptedException {
    project.android.bootClasspath.each {
        Config.classPool.appendClassPath((String) it.absolutePath)
    }
    def box = ReflectUtils.toCtClasses(inputs, Config.classPool)
    ...
    autoPatch(box)
    ...
}

def autoPatch(List<CtClass> box) {... ReadAnnotation.readAnnotation(box, logger);if(Config.supportProGuard) {
        ReadMapping.getInstance().initMappingInfo();
    }

    generatPatch(box,patchPath);

    zipPatchClassesFile()
    executeCommand(jar2DexCommand)
    executeCommand(dex2SmaliCommand)
    SmaliTool.getInstance().dealObscureInSmali();
    executeCommand(smali2DexCommand)
    //package patch.dex to patch.jar
    packagePatchDex2Jar()
    deleteTmpFiles()
}
Copy the code

In the Transform method, use the Javassist API to load all the classes that need to be processed into the queue to be scanned, and then call the autoPatch method to automatically generate a patch. In the autoPatch approach, there are several things that are done:

1. Read and record the methods or classes annotated by @add, @modify, robustmodify.modify ()
2. Parse the mapping file and record the information of each class and the method before and after the method is confused. The method stores the following information: return value, method name, parameter list, and the name after the confusion. The field stores the following information: field name and the obfuscated name
3. Based on the information I have,generatPatchMethod actually generates a patch
4. Package the generated patch class as a JAR package
5. jar -> dex
6. dex -> smali
7. Dealing with SMALI mainly involves dealing with super methods and dealing with obfuscation relationships
8. smali -> dex
9. dex -> jar

1, 2 better understand not step by step analysis, mainly see 3; The following 5, 6, 7, 8, and 9 are all for smALI processing in 7, so you just need to understand smALI processing. Let’s look at it step by step.

Generate the patch

The main logic is in the generatPatch method

def  generatPatch(List<CtClass> box,String patchPath){... InlineClassFactory.dealInLineClass(patchPath, Config.newlyAddedClassNameList) initSuperMethodInClass(Config.modifiedClassNameList);//auto generate all class
    for (String fullClassName : Config.modifiedClassNameList) {
        CtClass ctClass = Config.classPool.get(fullClassName)
        CtClass patchClass = PatchesFactory.createPatch(patchPath, ctClass, false, NameManger.getInstance().getPatchName(ctClass.name), Config.patchMethodSignatureSet) patchClass.writeFile(patchPath) patchClass.defrost(); createControlClass(patchPath, ctClass) } createPatchesInfoClass(patchPath); }... }Copy the code

Divided into two parts

Gradually the translation

First call InlineClassFactory. DealInLineClass (patchPath, Config. NewlyAddedClassNameList) identification method was optimized, the optimization is referring to, including optimization, inline, new classes and methods, The specific logic is to scan all modified classes and methods. If these classes and methods do not exist in the Mapping file, they can be defined as optimized, including the classes or methods added by @add. The initSuperMethodInClass method is then called to identify all the modified classes and methods in the class, and to see if they contain super methods, and if so, to cache them. Then call PatchesFactory. CreatePatch reflection translation to modify the classes and methods, specific implementation in

private CtClass createPatchClass(CtClass modifiedClass, boolean isInline, String patchName, Set patchMethodSignureSet, String patchPath) throws CannotCompileException, IOException, NotFoundException {
    // The cleaning method needs to be handled, omitted..

    CtClass temPatchClass = cloneClass(modifiedClass, patchName, methodNoNeedPatchList);
    
    JavaUtils.addPatchConstruct(temPatchClass, modifiedClass);
    CtMethod reaLParameterMethod = CtMethod.make(JavaUtils.getRealParamtersBody(), temPatchClass);
    temPatchClass.addMethod(reaLParameterMethod);

    dealWithSuperMethod(temPatchClass, modifiedClass, patchPath);

    for (CtMethod method : temPatchClass.getDeclaredMethods()) {
        // shit !! too many situations need take into consideration
        // methods has methodid and in patchMethodSignatureSet
        if(! Config.addedSuperMethodList.contains(method) && reaLParameterMethod ! = method && ! method.getName().startsWith(Constants.ROBUST_PUBLIC_SUFFIX)) { method.instrument(new ExprEditor() {
                        public void edit(FieldAccess f) throws CannotCompileException {...if (f.isReader()) { f.replace(ReflectUtils.getFieldString(f.getField(), memberMappingInfo, temPatchClass.getName(), modifiedClass.getName()));
                                } else if(f.isWriter()) { f.replace(ReflectUtils.setFieldString(f.getField(), memberMappingInfo, temPatchClass.getName(), modifiedClass.getName())); }... }@Override
                        void edit(NewExpr e) throws CannotCompileException {
                            //inner class in the patched class ,not all inner class.if(! ReflectUtils.isStatic(Config.classPool.get(e.getClassName()).getModifiers()) && JavaUtils.isInnerClassInModifiedClass(e.getClassName(), modifiedClass)) { e.replace(ReflectUtils.getNewInnerClassString(e.getSignature(), temPatchClass.getName(), ReflectUtils.isStatic(Config.classPool.get(e.getClassName()).getModifiers()), getClassValue(e.getClassName())));return; }...@Override
                        void edit(Cast c) throws CannotCompileException {
                            MethodInfo thisMethod = ReflectUtils.readField(c, "thisMethod");
                            CtClass thisClass = ReflectUtils.readField(c, "thisClass");

                            def isStatic = ReflectUtils.isStatic(thisMethod.getAccessFlags());
                            if(! isStatic) {//inner class in the patched class ,not all inner class
                                if (Config.newlyAddedClassNameList.contains(thisClass.getName()) || Config.noNeedReflectClassSet.contains(thisClass.getName())) {
                                    return;
                                }
                                // Static functions do not have this directive.
                                c.replace(ReflectUtils.getCastString(c, temPatchClass))
                            }
                        }

                        @Override
                        void edit(MethodCall m) throws CannotCompileException {...if(! repalceInlineMethod(m, method,false)) {
                                    Map memberMappingInfo = getClassMappingInfo(m.getMethod().getDeclaringClass().getName());
                                    if (invokeSuperMethodList.contains(m.getMethod())) {
                                        int index = invokeSuperMethodList.indexOf(m.getMethod());
                                        CtMethod superMethod = invokeSuperMethodList.get(index);
                                        if(superMethod.getLongName() ! =null && superMethod.getLongName() == m.getMethod().getLongName()) {
                                            String firstVariable = "";
                                            if (ReflectUtils.isStatic(method.getModifiers())) {
                                                // Fix static methods with super, such as Aspectj methods
                                                MethodInfo methodInfo = method.getMethodInfo();
                                                LocalVariableAttribute table = methodInfo.getCodeAttribute().getAttribute(LocalVariableAttribute.tag);
                                                int numberOfLocalVariables = table.tableLength();
                                                if (numberOfLocalVariables > 0) {
                                                    int frameWithNameAtConstantPool = table.nameIndex(0);
                                                    firstVariable = methodInfo.getConstPool().getUtf8Info(frameWithNameAtConstantPool)
                                                }
                                            }
                                            m.replace(ReflectUtils.invokeSuperString(m, firstVariable));
                                            return; } } m.replace(ReflectUtils.getMethodCallString(m, memberMappingInfo, temPatchClass, ReflectUtils.isStatic(method.getModifiers()), isInline)); }... }}); }}//remove static code block,pay attention to the class created by cloneClassWithoutFields which construct's
    CtClass patchClass = cloneClassWithoutFields(temPatchClass, patchName, null);
    patchClass = JavaUtils.addPatchConstruct(patchClass, modifiedClass);
    return patchClass;
}
Copy the code

This code is actually the core part of this plug-in, in general, is to translate all the modified code into reflection calls to generate xxxPatch class. We’ll just focus on method.instrument(), the Javassist API that iterates through the code logic in a method, including:

• FieldAccess, FieldAccess operations. Divided into two fields read and write, respectively calledReflectUtils.getFieldStringMethod to translate the code logic into reflection calls using Javassist, and then replace.
• NewExpr, new object operation. There are also two kinds
  • Non-static inner class, calledReflectUtils.getNewInnerClassStringTranslate to reflection, and then replace
  • External class, callReflectUtils.getCreateClassStringTranslate to reflection, and then replace
• Cast, strong transfer operation. callReflectUtils.getCastStringTranslate to reflection, and then replace
• MethodCall, MethodCall operations. The situation is more complex, the following several cases
  • The lamda expression is calledReflectUtils.getNewInnerClassStringGenerate the inner class method and translate it into reflection, then replace it
  • The modified method is an inline method calledReflectUtils.getInLineMemberStringMethod to generate a placeholder inline classxxInLinePatch, and in the modified class to translate the modified method into reflection, and then replace the call, this method has some other case judgment, interested readers can read for themselves
  • If it’s super, this is a separate case
  • Normal method, callReflectUtils.getMethodCallStringMethod translated as reflection, and then replaced
• Build a patch class and add constructors

Note that all of the above methods and methods that need to be handled require special attention to method signatures!

Controlling patch behavior

CreateControlClass (patchPath, ctClass), createPatchesInfoClass(patchPath); Generate PatchesInfoImpl and xxxPatchControl to write patch information and control patch behavior.

PatchesInfoImpl contains a one-to-one mapping for all the patch classes, such as MainActivity -> MainActivityPatch, as mentioned in the previous article in this series. The generated xxxPatchControl class is used to generate xxPatch class and judge whether the method in the patch matches the method ID in the methods.robust. If so, the method in the patch will be called.

At this point, the overall process is basically completed. Specific complications will be explained later.

How to handle this

First, in the patch class xxPatch, this refers to the object of the xxPatch class, and we want the object to be the object of the patched class.

In PatchesFactory. CreatePatchClass () method

CtMethod reaLParameterMethod = CtMethod.make(JavaUtils.getRealParamtersBody(), temPatchClass);
temPatchClass.addMethod(reaLParameterMethod);
    
public static String getRealParamtersBody(a) {
    StringBuilder realParameterBuilder = new StringBuilder();
    realParameterBuilder.append("public Object[] " + Constants.GET_REAL_PARAMETER + " (Object[] args){");
    realParameterBuilder.append("if (args == null || args.length < 1) {");
    realParameterBuilder.append(" return args;");
    realParameterBuilder.append("}");
    realParameterBuilder.append(" Object[] realParameter = new Object[args.length];");
    realParameterBuilder.append("for (int i = 0; i < args.length; i++) {");
    realParameterBuilder.append("if (args[i] instanceof Object[]) {");
    realParameterBuilder.append("realParameter[i] =" + Constants.GET_REAL_PARAMETER + "((Object[]) args[i]);");
    realParameterBuilder.append("} else {");
    realParameterBuilder.append("if (args[i] ==this) {");
    realParameterBuilder.append(" realParameter[i] =this." + ORIGINCLASS + ";");
    realParameterBuilder.append("} else {");
    realParameterBuilder.append(" realParameter[i] = args[i];");
    realParameterBuilder.append("}");
    realParameterBuilder.append("}");
    realParameterBuilder.append("}");
    realParameterBuilder.append(" return realParameter;");
    realParameterBuilder.append("}");
    return realParameterBuilder.toString();
}
Copy the code

Insert a getRealParameter() method into each xxPatch class and decompress the result:

public Object[] getRealParameter(Object[] objArr) {
    if (objArr == null || objArr.length < 1) {
        return objArr;
    }
    Object[] objArr2 = new Object[objArr.length];
    for (int i = 0; i < objArr.length; i++) {
        if (objArr[i] instanceof Object[]) {
            objArr2[i] = getRealParameter((Object[]) objArr[i]);
        } else if (objArr[i] == this) {
            objArr2[i] = this.originClass;
        } else{ objArr2[i] = objArr[i]; }}return objArr2;
}
Copy the code

This is used in each xxPatch to convert xx to the patched class object. OriginClass is the object of the patch class.

How to handle super

Similar to this, a super call in xxPatch also needs to be converted into a super call to the related method in the patched class.

Or in PatchesFactory. CreatePatchClass () method of dealWithSuperMethod (temPatchClass modifiedClass, patchPath); call

private void dealWithSuperMethod(CtClass patchClass, CtClass modifiedClass, String patchPath) throws NotFoundException, CannotCompileException, IOException {... methodBuilder.append("public static " + invokeSuperMethodList.get(index).getReturnType().getName() + "" + ReflectUtils.getStaticSuperMethodName(invokeSuperMethodList.get(index).getName()) + "(" + patchClass.getName() + " patchInstance," + modifiedClass.getName() + " modifiedInstance," + JavaUtils.getParameterSignure(invokeSuperMethodList.get(index)) + "{"); . CtClass assistClass = PatchesAssistFactory.createAssistClass(modifiedClass, patchClass.getName(), invokeSuperMethodList.get(index)); . methodBuilder.append(NameManger.getInstance().getAssistClassName(patchClass.getName()) +"." + ReflectUtils.getStaticSuperMethodName(invokeSuperMethodList.get(index).getName())  + "(patchInstance,modifiedInstance"); . }Copy the code

Leaving the main code, a new method is generated based on the method signature, named staticRobust+methodName, which calls a method of the same name in a class ending with RobustAssist, And call the PatchesAssistFactory createAssistClass method to generate the class, the class is the parent of the patch of the class parent class.

PatchesAssistFactory.createAssistClass:
static createAssistClass(CtClass modifiedClass, String patchClassName, CtMethod removeMethod) {... staticMethodBuidler.append("public static " + removeMethod.returnType.name + "" + ReflectUtils.getStaticSuperMethodName(removeMethod.getName())
            + "(" + patchClassName + " patchInstance," + modifiedClass.getName() + " modifiedInstance){");
                
    staticMethodBuidler.append(" return patchInstance." + removeMethod.getName() + "(" + JavaUtils.getParameterValue(removeMethod.getParameterTypes().length) + ");");
    staticMethodBuidler.append("}"); . }Copy the code

You then handle the MethodCall as you walk through the MethodCall

m.replace(ReflectUtils.invokeSuperString(m, firstVariable)); .def static String invokeSuperString(MethodCall m, String originClass) {... stringBuilder.append(getStaticSuperMethodName(m.methodName) +"(this," + Constants.ORIGINCLASS + ", \ \ $);"); . }Copy the code

The parameters passed, patch, originClass, the actual parameter list of the method.

The decompilation actually looks like this:

MainFragmentActivity:
public void onCreate(Bundle bundle) {
    super.onCreate(bundle); . } MainFragmentActivityPatch:public static void staticRobustonCreate(MainFragmentActivityPatch mainFragmentActivityPatch, MainFragmentActivity mainFragmentActivity, Bundle bundle) {
    MainFragmentActivityPatchRobustAssist.staticRobustonCreate(mainFragmentActivityPatch, mainFragmentActivity, bundle);
}

MainFragmentActivityPatchRobustAssist:
public class MainFragmentActivityPatchRobustAssist extends WrapperAppCompatFragmentActivity {
    public static void staticRobustonCreate(MainFragmentActivityPatch mainFragmentActivityPatch, MainFragmentActivity mainFragmentActivity, Bundle bundle) { mainFragmentActivityPatch.onCreate(bundler); }}Copy the code

The parameters are passed in according to the actual method parameters, and finally the xxpatch. superMethod method is called. But that doesn’t escape the super method, so let’s see.

In the process of dealing with Smali, there is this paragraph:

private String invokeSuperMethodInSmali(final String line, String fullClassName) {... result = line.replace(Constants.SMALI_INVOKE_VIRTUAL_COMMAND, Constants.SMALI_INVOKE_SUPER_COMMAND); . result = result.replace("p0"."p1"); . }Copy the code

Before processing, smali is long like this: invoke – invoke – virtual {p0, p2}, Lcom/at meituan/robust/patch/SecondActivityPatch; ->onCreate(Landroid/os/Bundle;) V, is this: after processing the invoke super {p1, p2}, Landroid/support/v7 / app/AppCompatActivity; ->onCreate(Landroid/os/Bundle;) V, which means that the normal method is called, but now the super method is called, and the parameter is changed, p0(the patched object) is replaced by P1 (the patched object), so that the super processing is completed. Final result after decompiling:

public class MainFragmentActivityPatchRobustAssist extends WrapperAppCompatFragmentActivity {
    public static void staticRobustonCreate(MainFragmentActivityPatch mainFragmentActivityPatch, MainFragmentActivity mainFragmentActivity, Bundle bundle) {
        super.onCreate(bundle); }}Copy the code

What about inline

Inlining is a broad concept that includes optimization (modification method signature, deletion method, etc.) and inlining in the confusion process. In the above analysis, the zi method is basically mentioned, and what is missing is made up: that is, the inline method is replaced again. For inline methods, the @modify annotation should not be used, but robustmodify.modify () annotation should only be used, as there are no methods in the base package and the L patch method is useless.

Main logic in the traversal MethodCall – > repalceInlineMethod () – > ReflectUtils. GetInLineMemberString ()

. stringBuilder.append(" instance=new " + NameManger.getInstance().getInlinePatchName(method.declaringClass.name) + "(\ $0);")
stringBuilder.append("\$_=(\$r)instance." + getInLineMethodName(method) + "(" + parameterBuilder.toString() + ");")...Copy the code

The effect is to replace the InLine method call with a new method in the InLine class.

The result is this:

public class Parent {
    private String first=null;
    //privateMethod is inlined
    // private void privateMethod(String fir){
    // System.out.println(fir);
    / /}
    public void setFirst(String fir){
        first=fir;
        Parent children=new Children();
        //children.privateMethod("Robust");
        // Inline substitution logic
        ParentInline inline= new ParentInline(children);
        inline.privateMethod("Robust"); }}public class ParentInline{
    private Parent children ;
    public ParentInline(Parent p){
       children=p;
    }
    // mix it up with c
    public void privateMethod(String fir){ System.out.println(fir); }}Copy the code

conclusion

The core of Robust is actually automatic generation of patches. Interpolation and patch loading are easy to achieve, because there are not many special cases to deal with. This article mainly analyzes the main workflow of the automatic patch plug-in, and the processing of some special cases, the article is limited, of course, there are many special cases without analysis, here just provide some analysis of the source code ideas, encountered special cases can be in accordance with this idea to solve the problem.

There is a comment in the code that I think encapsulates the journey of the team automating the patch generation, and I would like to thank the Meituan team again for their contribution to open source.

// shit !! too many situations need take into consideration
Copy the code

Generally speaking, there are Robust pits, but it is also a hotfix framework with high availability and compatibility. Especially, under the trend of increasingly tightening the openness of Android system, the Robust has more prominent advantages as a hotfix framework without hook system API. Although there may be some pitfalls, as long as we are familiar with the principles, we can be confident that we can solve them.

In the next article, I will mainly discuss some pits when using Robust with other frames.

reference

• Android hot update scheme Robust open source, new automatic patch tools
• Javassist User Guide (2)
• In-depth understanding of Dalvik bytecode instructions and Smali files