preface
AndFix, short for Android Hot-fix, is an Open-source Android hotfix framework that allows apps to fix online bugs without re-releasing the version. Currently, AndFix supports Android versions 2.3 to 6.0 and works on ARM and X86 devices. Perfect support for Dalvik and ART Runtime. The core technical points of this framework are twofold:
1. Use the apkpatch tool to generate apatch file in. Apatch format, load the patch file, and use @methodreplace to find a method to replace the patch file. 2. Find the pointer to the Method of Java layer corresponding to the structure Method of Native layer, and then replace each data of the structure member to complete the repair.
For the second point, in the native layer, Art and Dalvik VIRTUAL machines need to be processed separately, because the native structure corresponding to Method in the Java layer is completely different under the two virtual machines. This paper only analyzes the method replacement of Native layer under Art VIRTUAL machine, and writes a simple hot repair framework based on the repair principle of AndFix.
AndFix Hot repair principle
At the heart of the AndFix code is the replaceMethod function in andfix.java:
@AndFix/src/com/alipay/euler/andfix/AndFix.java
private static native void replaceMethod(Method src, Method dest);
Copy the codeThis is a native Method whose argument is the jobject corresponding to the Method object obtained by reflection in the Java layer. SRC corresponds to the old method that needs to be replaced, while DEST corresponds to the new method, which exists in the new class of the patch pack, namely the patch method. The specific implementation of replaceMethod method in Native layer is as follows:
@AndFix/jni/andfix.cpp
static void replaceMethod(JNIEnv* env, jclass clazz, jobject src,
jobject dest) {
if (isArt) {
art_replaceMethod(env, src, dest);
} else{ dalvik_replaceMethod(env, src, dest); }}Copy the codeIt can be clearly seen that the processing is different according to the virtual machine of Android. Dalvik virtual machine is used below 4.4, while Art virtual machine is used above 4.4. We mainly analyze the Art virtual machine situation here
@AndFix/jni/art/art_method_replace.cpp
extern void __attribute__ ((visibility ("hidden"))) art_replaceMethod(
JNIEnv* env, jobject src, jobject dest) {
if (apilevel > 23) {
replace_7_0(env, src, dest);
} else if (apilevel > 22) {
replace_6_0(env, src, dest);
} else if (apilevel > 21) {
replace_5_1(env, src, dest);
} else if (apilevel > 19) {
replace_5_0(env, src, dest);
}else{ replace_4_4(env, src, dest); }}Copy the codeAs we can see, the underlying data structure of Java objects in art virtual machines of different Android versions is different, so different functions need to be replaced according to different versions.
Art_44.h,art_5_0.h and other header files correspond to the corresponding structure of ArtMethod in different versions of Android, art_method_replace_44.cpp. Art_method_replace_5_0. CPP corresponds to a replacement method for the corresponding ArtMethod.
Each Java method in art corresponds to an ArtMethod, which records all the information about the Java method, including its class, access rights, code execution address, and so on.
By using env->FromReflectedMethod, we can get the real starting address of the ArtMethod corresponding to the Native layer from the Method object in the Java layer. You can then force it into an ArtMethod pointer, which you can then modify to all of its members.
This completes the hot repair logic once all the replacements are complete. Later calls to this method will go directly to the implementation of the new method.
Handwritten AndFix
Next, according to the idea of AndFix, we will create a new project and write a simple thermal repair function.
Here is the corresponding ArtMethod data structure in android7.0 source code, copy it to the local project code:
@AndFixProject/app/src/main/cpp/art_7_0.h
namespace art {
namespace mirror {
class ArtMethod {
public:
// Field order required by test "ValidateFieldOrderOfJavaCppUnionClasses".
// The class we are a part of.
uint32_t declaring_class_;
// Access flags; low 16 bits are defined by spec.
uint32_t access_flags_;
/* Dex file fields. The defining dex file is available via declaring_class_->dex_cache_ */
// Offset to the CodeItem.
uint32_t dex_code_item_offset_;
// Index into method_ids of the dex file associated with this method.
uint32_t dex_method_index_;
/* End of dex file fields. */
// Entry within a dispatch table for this method. For static/direct methods the index is into
// the declaringClass.directMethods, for virtual methods the vtable and for interface methods the
// ifTable.
uint16_t method_index_;
// The hotness we measure for this method. Incremented by the interpreter. Not atomic, as we allow
// missing increments: if the method is hot, we will see it eventually.
uint16_t hotness_count_;
// Fake padding field gets inserted here.
// Must be the last fields in the method.
// PACKED(4) is necessary for the correctness of
// RoundUp(OFFSETOF_MEMBER(ArtMethod, ptr_sized_fields_), pointer_size).
struct PtrSizedFields {
// Short cuts to declaring_class_->dex_cache_ member for fast compiled code access.
ArtMethod** dex_cache_resolved_methods_;
// Short cuts to declaring_class_->dex_cache_ member for fast compiled code access.
void* dex_cache_resolved_types_;
// Pointer to JNI function registered to this method, or a function to resolve the JNI function,
// or the profiling data for non-native methods, or an ImtConflictTable.
void* entry_point_from_jni_;
// Method dispatch from quick compiled code invokes this pointer whichmay cause bridging into // the interpreter. void* entry_point_from_quick_compiled_code_; } ptr_sized_fields_; }; }}Copy the codeJava layer create a native method:
@AndFixProject/app/src/main/java/com.wind.cache.andfixproject/AndFixManager.java
public static native void andFixMethod(Method srcMethod, Method dstMethod);
Copy the codeCPP file, and implement the Java layer native method, the implementation of the way is to replace every field in the ArtMethod structure, please see source code:
@AndFixProject/app/src/main/cpp/andFix.cpp
Java_com_wind_cache_andfixproject_AndFixManager_andFixMethod(JNIEnv *env, jobject instance,
jobject srcMethod, jobject dstMethod) {
LOGD("start fix art_method!!!!");
art::mirror::ArtMethod* meth = (art::mirror::ArtMethod*) env->FromReflectedMethod(srcMethod);
art::mirror::ArtMethod* target = (art::mirror::ArtMethod*) env->FromReflectedMethod(dstMethod);
target->declaring_class_ = meth->declaring_class_;
target->access_flags_ = meth->access_flags_;
target->dex_code_item_offset_ = meth->dex_code_item_offset_;
target->dex_method_index_ = meth->dex_method_index_;
target->method_index_ = meth->method_index_;
target->hotness_count_ = meth->hotness_count_;
target->ptr_sized_fields_.dex_cache_resolved_types_ = meth->ptr_sized_fields_.dex_cache_resolved_types_;
target->ptr_sized_fields_.dex_cache_resolved_methods_ = meth->ptr_sized_fields_.dex_cache_resolved_methods_;
target->ptr_sized_fields_.entry_point_from_jni_ = meth->ptr_sized_fields_.entry_point_from_jni_;
target->ptr_sized_fields_.entry_point_from_quick_compiled_code_ = meth->ptr_sized_fields_.entry_point_from_quick_compiled_code_;
}
Copy the codeLet’s write two classes to test whether the JNI interface can implement method substitution.
Of course, we can get two of the methods in the two classes in this project directly by reflection, and then replace them.
But to be more general, we will replace the class rightMethodClass. Java with the javac command to generate the rightMethodClass. class file, and then the class file with the dex tool in the Android_SDK to generate the dex file. Put the generated dex file into the assets directory of the project, read the classes in the DEX file for replacement, and simulate the process of downloading the differential file from the network for replacement.
The correct method class is:
public class RightMethodClass {
public int fixGet(int a, int b) {
returna+b+100000; }}Copy the codeThe class of the error method is:
public class WrongMethodClass {
public int get(int a, int b) {
Log.e("WrongMethodClass"."you have run the wrong method !!!!");
returna*b; }}Copy the codeThe specific substitution process is relatively simple, so this treatment will not be explained too much:
public static void startAndFix(Context context) { try { Class<? > clazz = loadRightMethodClass(context); Method srcMethod = clazz.getMethod("fixGet", int.class, int.class);
Method dstMethod = WrongMethodClass.class.getMethod("get", int.class, int.class); andFixMethod(srcMethod, dstMethod); } catch (NoSuchMethodException e) {e.printstackTrace ();} catch (NoSuchMethodException e) {e.printstackTrace (); } } private static String fixDexPath ="file:///android_asset/fix.dex"; // Load the correct methods in the dex file from assets. Private static Class<? > loadRightMethodClass(Context context) { DexClassLoader rightClassLoader = new DexClassLoader(fixDexPath, context.getDir("dex", Context.MODE_PRIVATE).getAbsolutePath(), null, context.getClassLoader()); Class<? > clazz = null; try { clazz = rightClassLoader.loadClass("com.wind.cache.andfixproject.RightMethodClass");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
return clazz;
}
Copy the codeRun the above code, sure enough, can achieve method replacement!
The disadvantages of AndFix
However, almost all native alternatives on the market, such as Andfix and Legend, another Hook framework, are written with the ArtMethod structure, which can cause huge compatibility issues.
Please see the following explanation for why.
As you can see from the analysis and hand-written code, Andfix has changed the underlying structure of Java Methods to art:: Mirror ::ArtMethod. But art::mirror::ArtMethod here is not necessarily the same as the art::mirror::ArtMethod at the bottom of the device VIRTUAL machine where app is running, but the art::mirror::ArtMethod constructed by Andfix itself. The size of each member in the ArtMethod structure is exactly the same as in the AOSP open source code. This is because Android source code is open, Andfix inside the ArtMethod naturally follow the Android VIRTUAL machine art source code inside the ArtMethod build.
However, since Android is open source, any phone manufacturer can modify the code, and the structure of ArtMethod in Andfix is written according to the structure of open Android source code. If a vendor makes changes to the ArtMethod structure that are inconsistent with the original open source structure, the replacement mechanism will fail on the modified device and method replacement will not be possible.
This is the biggest drawback of AndFix.
Optimization scheme
Is there an alternative that ignores the dependency on the underlying ArtMethod structure?
We found that the replacement idea at the native level like AndFix is to replace all the members of ArtMethod. So, instead of replacing the specific member fields of ArtMethod, we directly replace the ArtMethod as a whole. Isn’t that ok?
That is, replace the old ones one by one:
Change to whole replacement:
Hence Andfix’s cumbersome series of replacements:
target->declaring_class_ = meth->declaring_class_; target->access_flags_ = meth->access_flags_; target->dex_code_item_offset_ = meth->dex_code_item_offset_; target->dex_method_index_ = meth->dex_method_index_; target->method_index_ = meth->method_index_; target->hotness_count_ = meth->hotness_count_; .Copy the codeChanged to:
memcpy(target,meth, sizeof(ArtMethod));
Copy the codeThis is where we’ll look at the improved AndFix, the hot fix implemented by Sophix.
Improved AndFix, the Sophix thermal repair principle
According to the above analysis, only the word memcpy can replace the above pile of code, which is the alternative proposed by Alibaba Sophix.
As mentioned earlier, different phone vendors can change the underlying ArtMethod as much as they want, but even if they change the ArtMethod into something completely unrecognizable, as long as I replace the entire ArtMethod structure like this, all the members of the old method will automatically be replaced with members of the new method.
But the most important thing is sizeof ArtMethod. If there is a deviation in size calculation, some members are not replaced, or the replacement area exceeds the boundary, it will lead to serious problems.
For ROM developers, it is in the ART source code, so a simple sizeof(ArtMethod) will do as this is determined at compile time.
But we are the upper developers, the app will be sent to a variety of Android devices, so we need to dynamically get the size of the underlying ArtMethod on the device running the app at runtime, which is not so simple.
Through the analysis of the source code of Art virtual machine, we found that the ArtMethods of each class are closely arranged together in memory, so the size of an ArtMethod is not the difference of the starting address of the ArtMethod corresponding to two adjacent methods.
Because of this, we start from this arrangement, we construct a class, this class contains only two methods, by getting the difference between the two methods corresponding to the start address of the ArtMethod to get sizeOf(ArtMethod).
public class NativeArtMethodCalculator {
public static void method1(){}
public static void method2(){}
}
Copy the codeThen we can get the difference between their addresses at the JNI layer:
size_t art_method_length = 0;
Java_com_wind_cache_andfixproject_AndFixManager_getArtMethoLength(JNIEnv *env, jobject instance, jobject method1, jobject method2) {
if(art_method_length ! = 0) {return art_method_length;
}
size_t method1Ptr = (size_t)env->FromReflectedMethod(method1);
size_t method2Ptr = (size_t)env->FromReflectedMethod(method2);
art_method_length = method2Ptr - method1Ptr;
return art_method_length;
}
Copy the codeThen, we use this art_method_length as sizeof(ArtMethod) and substitute in the previous code to implement method substitution:
Java_com_wind_cache_andfixproject_AndFixManager_hotFixMethod(JNIEnv *env, jobject instance,
jobject srcMethod, jobject dstMethod) {
jmethodID meth = env->FromReflectedMethod(srcMethod);
jmethodID target = env->FromReflectedMethod(dstMethod);
memcpy(target, meth, art_method_length);
}
Copy the codeIt is worth mentioning that, because the difference of the underlying ArtMethod structure is ignored, it is no longer necessary to distinguish for all Android versions, but can be unified to memcpy implementation, greatly reducing the amount of code. Even if later Versions of Android constantly modify the ArtMethod members, as long as the ArtMethod array is still arranged in a linear structure, it can be directly applied to the future version of Android 8.0, 9.0 and other new versions, there is no need to adapt to the new system version.
Pure Java code implementation method hot update
The above is to use Native code to implement method replacement, the core is to obtain the Java method corresponding to the Native memory address (pointer), and then replace the memory address. Is it possible to implement this memory address substitution in pure Java code? It can! To implement a mechanism like the one in C++ that replaces references, we need to get the method’s memory address in the Java layer, which requires some hacking.
Sun. Misc. Unsafe and libcore. IO. The Memory
It is not impossible to manipulate memory at the Java layer; The JDK gives us a back door: the sun.misc.Unsafe class; This class is very powerful in OpenJDK, from memory manipulation to CAS to locking mechanism. .
The Unsafe class provides some lower-level functionality that bypasses the JVM, and its implementation can be more efficient. However, it’s a double-edged sword: as its name implies, it’s Unsafe because it bypasses the JVM, so its allocated memory needs to be manually free (not reclaimed by GC). The Unsafe class provides alternative implementations of some of THE functionality of JNI, ensuring efficiency while making things simpler.
With these two classes, we can do simple memory operations in the Java layer! Since these two classes are hidden and need to be called by reflection, a simple wrapper is written as follows:
public class MemoryWrapper {
private static final String UNSAFE_CLASS = "sun.misc.Unsafe";
private static Object THE_UNSAFE;
private static boolean is64Bit;
static {
THE_UNSAFE = Reflection.get(null, UNSAFE_CLASS, "THE_ONE", null);
Object runtime = Reflection.call(null, "dalvik.system.VMRuntime"."getRuntime", null, null, null);
is64Bit = (Boolean) Reflection.call(null, "dalvik.system.VMRuntime"."is64Bit", runtime, null, null);
}
// libcode.io.Memory#peekByte
private static byte peekByte(long address) {
return (Byte) Reflection.call(null, "libcore.io.Memory"."peekByte", null, new Class[]{long.class}, new Object[]{address});
}
static void pokeByte(long address, byte value) {
Reflection.call(null, "libcore.io.Memory"."pokeByte", null, new Class[]{long.class, byte.class}, new Object[]{address, value});
}
public static void memcpy(long dst, long src, long length) {
for (long i = 0; i < length; i++) {
pokeByte(dst, peekByte(src));
dst++;
src++;
}
}
public static long getMethodAddress(Method method) {
Object mirrorMethod = Reflection.get(Method.class.getSuperclass(), null, "artMethod", method);
return(Long) mirrorMethod; }}Copy the codeUse the Unsafe implementation method instead
From the above analysis, it can be seen that the core of method replacement (hot update) is to call this method implementation in native layer:
memcpy(target,meth, sizeof(ArtMethod));
Copy the codeNative layer memcpy methods can be implemented via memcpy methods in the MemoryWrapper class:
public static void memcpy(long dst, long src, long length) {
for(long i = 0; i < length; i++) { pokeByte(dst, peekByte(src)); dst++; src++; }}Copy the codeThe sizeof(ArtMethod) method of the native layer can be implemented by:
/ / principle: in the same class ArtMethod memory address is closely arranged in sequence Method method1. = NativeArtMethodCalculator class. GetMethod ("method1");
Method method2 = NativeArtMethodCalculator.class.getMethod("method2");
long method1Address = MemoryWrapper.getMethodAddress(method1);
long method2Address = MemoryWrapper.getMethodAddress(method2);
long sizeOfArtMethod = method2Address - method1Address;
Copy the codeThus, our core code for implementing method hot updates through pure Java code is:
//通过Java方法来操作内存，将ArtMethod的Native指针进行替换
public static void startFixByJava(Context context) {
try {
Method method1 = NativeArtMethodCalculator.class.getMethod("method1");
Method method2 = NativeArtMethodCalculator.class.getMethod("method2"); long method1Address = MemoryWrapper.getMethodAddress(method1); long method2Address = MemoryWrapper.getMethodAddress(method2); long sizeOfArtMethod = method2Address - method1Address; // Equivalent to calling JNI methods: sizeOfArtMethod = getArtMethoLength(method1, method2); Class<? > clazz = loadRightMethodClass(context); Method srcMethod = clazz.getMethod("fixGet", int.class, int.class);
Method dstMethod = WrongMethodClass.class.getMethod("get", int.class, int.class); long dstAddress = MemoryWrapper.getMethodAddress(dstMethod); long srcAddress = MemoryWrapper.getMethodAddress(srcMethod); MemoryWrapper.memcpy(dstAddress, srcAddress, sizeOfArtMethod); // equivalent to calling JNI methods: memcpy(dstAddress, srcAddress, art_method_length); } catch (NoSuchMethodException e) { e.printStackTrace(); }}Copy the codeAt this point, we have implemented an AndFix in pure Java code with less than 200 lines!! Isn’t it wonderful?