background

Plugins are not a new technology in the Android development industry, and have long been standard on larger platform apps. In 2017, Atlas, Replugin and VirtualAPK have been open source, indicating that plug-in technology has entered a mature stage. However, all the major plug-in frameworks are developed based on the business of their own App, and their goals are more or less different, so it is difficult to have a plug-in framework that can solve all the problems in a unified way. Finally is around the compatibility problem, Android each version of the upgrade will bring a lot of impact to each plug-in framework, have to work hard to adapt some, not to mention the domestic manufacturers on the ROM customization, as VirtualAPK author Ren Yugang said: It’s not that hard to Demo a plug-in framework, but it’s not that easy to develop a complete plug-in framework.

As early as 2014, Meituan Mobile technology team began to pay attention to plug-in technology, and realized that plug-in architecture is the best integration form of meituan platform App. However, due to the rapid growth, iteration and evolution of the business, limited by business coupling and architectural issues, plug-in has been unable to be implemented. By the end of 2016, after a series of code architecture adjustments and technical research, we were finally free to implement the plug-in technology.

Meituan Platform (together with dianping platform) currently carries the business of nearly 20 business lines of all business groups of Meituan Dianping. Among them, there are relatively mature businesses, such as takeout and catering, which require high stability of plug-ins. They should not cause business problems because of plug-ins. There are also businesses with rapid iteration changes, such as transportation, errands, finance, etc., which require rapid iteration to go online; In addition, since the binary AAR-dependent integration of Meituan App has been running for two years and the infrastructure is mature, we don’t want to have to change the development model after switching to plug-in access. Therefore, meituan platform’s appeal for plug-ins mainly focuses on compatibility and does not affect the development mode of the two points.

The principle and characteristics of meituan plug-in framework

The compatibility of the plugin framework can be seen in many ways. Due to the Android mechanism, some scripts that work well before the plugin are no longer effective after the plugin is added. If compatibility issues are not addressed, word of mouth and promotion of plug-ins will be greatly hindered. Compatibility refers not only to compatibility with the Android system and Android fragmentation, but also to compatibility with existing base libraries and build tools. Especially for the latter, we often see a large number of Crash issues in the open source plug-in framework on Github, which is caused by this aspect. Each App’s base library and existing build tools are different, so it’s important to choose the right solution for your App.

In order to ensure the compatibility of plug-ins and seamlessly compatible with the current AAR development mode, Meituan’s plug-in framework scheme mainly does the following:

  • Dex loading of plug-ins uses a similar scheme to MultiDex to ensure compatibility with reflection
  • Replace all assetManagers to ensure compatible access to resources
  • Four components embedded, proxy new Activity
  • Let the build system bridge the gap between AAR and plug-in development

MultiDex and component proxy are not discussed here, but there are many blogs on the web. The following focuses on the resource processing of Meituan plug-in framework and the construction system that supports AAR and plug-in one-click switch.

Resources to deal with

Readers of plugins know that if you want to access a plug-in’s resources, you need to use AssetManager to add the plug-in’s path. But this is not enough. This is because if you want the AssetManager to work, you have to put it in a specific Resources or ResourcesImpl. Most plug-in frameworks do this by wrapping a Resources that contains the AssetManager, the plug-in path, Then use only one resource in the plug-in.

This works most of the time, but there are at least three problems:

  1. If host Resources are used in the plug-in, for example:getApplicationContext().getResources(). This resource cannot access the Resources of the plug-in
  2. Resources outside the plug-in are not unique and need to be found and replaced globally
  3. There are many intermediates in Resoureces, such as Theme, TypedArray, and so on. These need to be cleaned up before they can be used properly

To completely solve these problems, we took a different approach and did a global resource approach:

  • Create or use an existing AssetManger to load the plug-in resources
  • Find all Resources/Theme and replace the AssetManger in them
  • Clean up the Resources cache and rebuild the Theme
  • Rebuild protection of AssetManager to prevent lost plug-in paths

This scheme is somewhat similar to InstantRun, but native InstantRun has too many problems:

  • The cleaning order is out of order. You should clean up appliciton first and then Activity
  • There is no mechanism for dealing with extreme situations
  • Theme light cleanup does not rebuild
  • Completely not suitable for Support packages buried in their own “thunder” and so on

For example, failing to find a Theme completely: InstantRun replaces the AssetManager in the Theme by fetching it from within each Activity.

for (Activity activity : activities) { ... Resources.theme = activity.gettheme (); try { try { Field ma = Resources.Theme.class.getDeclaredField("mAssets"); ma.setAccessible(true); ma.set(theme, newAssetManager); } catch (NoSuchFieldException ignore) { Field themeField = Resources.Theme.class.getDeclaredField("mThemeImpl"); themeField.setAccessible(true); Object impl = themeField.get(theme); Field ma = impl.getClass().getDeclaredField("mAssets"); ma.setAccessible(true); ma.set(impl, newAssetManager); }... } catch (Throwable e) { Log.e(LOG_TAG, "Failed to update existing theme for activity " + activity, e); } pruneResourceCaches(resources); }Copy the code

It’s a good idea, but it’s not enough. For example, Google’s own Support package inside of a class. Android Support. V7. The ContextThemeWrapper generates a new Theme to save:

public class ContextThemeWrapper extends ContextWrapper { private int mThemeResource; private Resources.Theme mTheme; private LayoutInflater mInflater; . private void initializeTheme() { final boolean first = mTheme == null; if (first) { mTheme = getResources().newTheme(); final Resources.Theme theme = getBaseContext().getTheme(); if (theme ! = null) { mTheme.setTo(theme); } } onApplyThemeResource(mTheme, mThemeResource, first); }... }Copy the code

If you don’t have to replace the ContextThemeWrapper Theme, if use Reources/AssetManager is new, it will lead to Crash: Java. Lang. RuntimeException: Failed to resolve attribute at index 0 this is an Issue that most open source frameworks have. To solve this problem, we not only clean up the Theme of all activities, but also the Context of all views.

try { List<View> list = getAllChildViews(activity.getWindow().getDecorView()); for (View v : list) { Context context = v.getContext(); if (context instanceof ContextThemeWrapper && context ! = activity && ! clearContextWrapperCaches.contains(context)) { clearContextWrapperCaches.add((ContextThemeWrapper) context); pruneSupportContextThemeWrapper((ContextThemeWrapper) context, newAssetManager); // Theme}}} catch (Throwable ignore) {log.e (LOG_TAG, ignore.getmessage ()); }Copy the code

However, these methods do not solve all problems. Sometimes, in order to fulfill a product requirement, Android engineers may adopt some unconventional writing methods, resulting in a failure to load resources after becoming plug-ins. For example, you save the Theme in your own class. It’s not possible to change the business code one by one, so can you make your plug-in compatible with this way of writing? We also make this behavior compatible: bytecode modification.

If you want to save a class variable, the instruction is PUTFIELD/PUTSTATIC. If you want to save a class variable, the instruction is PUTFIELD/PUTSTATIC.

static class MyMethodVisitor extends MethodVisitor { int stackSize = 0; MyMethodVisitor(MethodVisitor mv) { super(Opcodes.ASM5, mv); } @Override public void visitFieldInsn(int opcode, String owner, String name, String desc) { if (opcode == Opcodes.PUTFIELD || opcode == Opcodes.PUTSTATIC) { if ("Landroid/content/res/Resources$Theme;" .equals(desc)) { stackSize = 1; visitInsn(Opcodes.DUP); super.visitMethodInsn(Opcodes.INVOKESTATIC, "com/meituan/hydra/runtime/Transformer", "collectTheme", "(Landroid/content/res/Resources$Theme;) V", false); } } super.visitFieldInsn(opcode, owner, name, desc); } @Override public void visitMaxs(int maxStack, int maxLocals) { super.visitMaxs(maxStack + stackSize, maxLocals); stackSize = 0; }}Copy the code

This ensures that any Theme saved by the class is collected and cleaned up and rebuilt after the plugin is installed.

Build system for plug-ins

Way in order to achieve the integration of AAR and plug-in integration between a key switch, and solve the plug-in “API trap” of problems, and we spend a lot of time in the build system for the construction of the above, we build system in addition to build the plug-in support routine, also support the existing build tools and future possible build tools. We divided the normal build process into four stages:

  1. Collect rely on
  2. Processing resources
  3. Handling code
  4. Packaging signature

So how do you guarantee support for existing Gradle plug-ins? The best way to do this is not to interfere too much with the build process and to keep them running as normal and sequentially as possible. Therefore, without interfering with this order, our build system inserts the plug-in build process into the four stages of normal build, mainly doing the following work.

  • After the host resolves the dependencies, the plug-in’s dependencies are analyzed for dependency mediation and reference count analysis
  • Before the host processes resources, it processes plug-in resources to avoid the trap of resource access, generates a list of resources that need to be merged to the host, and develops Meituan AAPT to process plug-in resources
  • In the host processing code, avoid the trap used by the plug-in API, reuse the host Proguard and Gradle plug-ins, to achieve the maximum compatibility with the native build process. We also fixed Proguard Mapping, which will be covered in a blog post
  • Before the host packages the signature, the plug-in APK is built, the upgrade compatible Hash characteristics are calculated, and V2 signatures are used to speed up runtime validation

The process of building the system is shown below

API trap

Another very important reason for our plug-in build system is to avoid the “API trap.” Here are some of the issues to be aware of when accessing Atlas, which we call “API traps”

  1. The Activity by the switch used overridePendingTransition animation files should be put in the APK;
  2. If a custom style is available within the Bundle, then the parent of the style must be in the main APK if it is also custom. This is due to the inherent logic of style lookup in the system after 5.0, which is not fully compatible within the container
  3. If there is SO inside the Bundle, it cannot be decompressed to the APK lib directory during installation. Therefore, it is limited to use Dlopen directly through the native layer to use SO, which will affect the subsequent dynamic deployment of SO. Therefore, it is not recommended to use Dlopen in the Bundle at present

So how did we do that? We use build tools to automate processing of plug-in resources. Separate the plugin-specific dependencies from the dependencies that the host processes, and then prepare a separate resource directory for the host that contains only the resources that need to be merged. So how do you pull away? Let’s look at how tasks that process resources get those resources. Code in com. Android. Build. Gradle. Tasks. $ConfigAction MergeResources

ConventionMappingHelper.map(mergeResourcesTask, "inputResourceSets", new Callable<List<ResourceSet>>() { @Override public List<ResourceSet> call() throws Exception { List<File> generatedResFolders = Lists.newArrayList( scope.getRenderscriptResOutputDir(), scope.getGeneratedResOutputDir()); if (variantData.getExtraGeneratedResFolders() ! = null) { generatedResFolders.addAll( variantData.getExtraGeneratedResFolders()); } if (scope.getMicroApkTask() ! = null && variantData.getVariantConfiguration().getBuildType() .isEmbedMicroApp()) { generatedResFolders.add(scope.getMicroApkResDirectory()); } return variantData.getVariantConfiguration().getResourceSets( generatedResFolders, includeDependencies, validateEnabled); }});Copy the code

Those of you who know Groovy know that setting this inputResourceSets overrides the getInputResourceSets method of mergeResourcesTask. So we can also do this:

ConventionMapping conventionMapping = (ConventionMapping) ((GroovyObject) variantData.mergeResourcesTask).getProperty("conventionMapping"); def srcMethod = conventionMapping._mappings.get("inputResourceSets"); conventionMapping.map("inputResourceSets", new Callable<List<ResourceSet>>() { @Override public List<ResourceSet> call() throws Exception { List<ResourceSet> res =  srcMethod.getValue(null, null) ... Return res}})Copy the code

On the first issue: the previously mentioned resource folder that the plug-in provides to the host doesn’t mean anything if it’s empty. We merge all resources referenced by the plugin’s Androidmanifest.xml file as root, from files to a single value under the values folder. But just AndroidManifest. The XML file is not enough, all of the documents to the system, such as mentioned in the “Activity by changing the overridePendingTransition use animation files”, also in this folder. Here you need to use ASM to scan all API calls from the plugin, similar to the Theme search above, which is not expanded in detail.

Merge merge merge merge merge merge merge merge merge merge merge merge

3. API traps are not only resource traps, but also code level traps. The above plug-in so loading problem is a typical example. We found that this problem did not occur if the so loaded by the plugin Dlopen had been loaded before.

private static Pattern compile = Pattern.compile("dlopen failed: library \"lib(.+).so\" not found"); public static void system_loadLibrary(String libname) { LinkedList<String> list = new LinkedList<>(); list.add(libname); while (list.size() > 0) { try { System.loadLibrary(list.peekFirst()); list.pop(); } catch (UnsatisfiedLinkError error) { // dlopen failed: library "libglog_init.so" not found Matcher matcher = compile.matcher(error.getMessage()); if (matcher.matches()) { String group = matcher.group(1); list.addFirst(group); } else { throw error; }}}}Copy the code

Of course, there are many apis that need to be replaced, such as getIdentifier, Notification, Glide, etc.

conclusion

This paper mainly introduces the design ideas and some implementation of meituan plug-in. Through our efforts, the business integration mode of Meituan platform can smoothly switch between AAR integration mode and plug-in integration mode, and there is almost no compatibility problem when it goes online. At present, in the recent versions of Meituan App, important modules such as search, favorites and orders are loaded in the form of plug-ins.

Author’s brief introduction

Ting Li is a technical expert at Meituan Dianping. He joined Meituan in 2014. Responsible for several business projects and technical projects, committed to promoting the application of AOP and bytecode technology in Meituan. Independently responsible for the App preinstallation project of Meituan and promoted the automation of preinstallation. Led the design and development of Meituan plug-in framework, and the focus of my current work is preaching and promotion of Meituan plug-in framework.

Wei Xia is a senior engineer at Meituan Dianping. He joined Meituan in 2017. At present, I am engaged in the plug-in development of Meituan and the optimization of some underlying tools of Meituan platform, such as AAPT and ProGuard. I focus on Hook technology and reverse research and am used to finding solutions from source code.

Client technology team of Meituan platform, responsible for the development of basic business and mobile infrastructure of Meituan platform. Meituan-dianping’s rapid development has been supported by its platform based on a large number of users. At the same time, we have also made some active exploration in mobile development technology, and have accumulated certain accumulation in dynamic, quality assurance, development model and other aspects. While the client technology team is actively adopting open source technology, we also contribute some of our accumulation to the open source community, hoping to promote the efficiency and quality of mobile development together with the industry.








If you answer “thinking questions”, find mistakes in the article, or have questions about the content, you can leave a message to us at the background of wechat public account (Meituan-Dianping technical team). Each week, we will select one “excellent responder” and give a nice small gift. Scan the code to pay attention to us!