Introduction to the

In recent years, Tencent is also more and more support for open source. Today, what we want to talk about is Tencent’s widely used Shadow framework, a reflection full dynamic Android plug-in framework tested by hundreds of millions of users on the line. First, let’s take a look at the official description of Shadow:

Shadow is an Android plug-in framework independently developed by Tencent, which has been tested by hundreds of millions of users online. Shadow not only open-source shares the key code for plug-in technology, but also fully shares all the design required for live deployment.

Compared to other plug-in frameworks on the market, Shadow has the following features:

  • Reuse independently installed App source code: plug-in App source code is normally installed to run.
  • Zero reflection without Hack implementation plug-in technology: theoretically it has been determined that there is no need to do compatible development for any system, nor any hidden API calls, and Google’s policy of restricting access to non-public SDK interfaces is completely not in conflict.
  • Fully dynamic plug-in frameworks: It’s difficult to implement perfect plug-in frameworks all at once, but Shadow makes these implementations dynamic, making the plug-in framework code part of the plug-in. Iteration of a plug-in is no longer limited by the host packaging an older version of the plug-in framework.
  • Minimal host increment: Thanks to full dynamic implementation, the amount of code that actually fits into the host program is minimal (15KB, around 160 methods).
  • Kotlin support: Core. loader, core.transform core code is fully implemented by Kotlin, the code is simple and easy to maintain.

In addition, Shadow supports the following features:

  • Four major components
  • Fragment (Code add and Xml Add)
  • DataBinding (No special support required, but verified to work)
  • Use the plug-in Service across processes
  • Custom Theme
  • The plug-in accesses the host class
  • So the load
  • Segmented loading plug-in (multiple APKs loaded separately or multiple APKs loaded with this dependency)
  • Load multiple VIEWS from Apk in one Activity
  • …

Shadow solves the problem

Non-public SDK interface access

As we all know, with Android 9.0 restricting access to the non-public SDK interface, it’s fair to say that every plugin framework implementation we know of at the time was more or less a fit problem. Everyone’s response is basically a kind of confrontation, some to crack restrictions, some by communicating with Google to temporarily renew the validity of the grey list.

Faced with this problem, we chose not to fight this strategy, and we understand why Google restricts the use of the non-public SDK interface. Therefore, we reviewed the essence principle and design defects of the plug-in framework, and then designed a new plug-in framework Shadow. Shadow does not use any non-public SDK interfaces and implements the same functionality as the original implementation that uses a large number of non-public SDK interfaces.

In Shadow’s Sample, you can add code like the one shown below to enable strict check mode running, which is not possible with other plug-in frameworks.

StrictMode.VmPolicy.Builder builder = new StrictMode.VmPolicy.Builder();
builder.penaltyDeath();
builder.detectNonSdkApiUsage();
StrictMode.setVmPolicy(builder.build());

Copy the code

For example, we saw a plug-in framework that supports Android 9.0 and does not use the non-public SDK interface. After the strict mode is enabled in its Sample, the following Crash information appears:

W/.xxx.sampl: Accessing hidden method Landroid/view/View; ->computeFitSystemWindows(Landroid/graphics/Rect; Landroid/graphics/Rect;) Z (light greylist, reflection) W/System.err: StrictMode VmPolicy violation with POLICY_DEATH; shutting down.Copy the code

As you can see, even though it doesn’t have any references to the non-public SDK in its implementation code, it actually relies on third-party components that use the non-public SDK interface internally.

Plug-in frameworks are fully dynamic

Full dynamic means that in addition to the plug-in code, all the logic code of the plug-in framework itself is also dynamic. And the Shadow framework actually does the same thing: the code for the plug-in framework is packaged with the plug-in.

How important is a fully dynamic plug-in framework? It’s more important than a hack-free, zero-reflection implementation! Because with this feature, even if we use Hack solution, we need to be compatible with the system of various mobile phone manufacturers. We don’t have to wait for a host App update to fix the problem.

In fact, Shadow implemented this feature much earlier. We started using plug-in technology a lot back in 2015. This feature was implemented in 2016. With this feature, the code of the plug-in framework is constantly published dynamically to accommodate various compatibility issues. This year, I applied this feature even more, updating the old implementation with hundreds of reflection Hack calls to Shadow hack-free implementation without having to follow the host version at all. The new Shadow certainly does.

In practice, our hosts are extremely demanding on incremental business access. Shadow access uses only 15.1KB and 160 methods. Other plug-in frameworks that we know of typically add 110KB, 900 methods, to 370KB, 2300 methods to the host.

Realize the principle of

The previous plug-in framework always wanted to use some Hack means to modify the system behavior, find loopholes in the system to achieve the purpose. Shadow’s principle is not to fight the system. Since it only restricts access to non-public SDK interfaces, it does not restrict dynamically loaded code. There must be a way to do this without using a non-public SDK interface. Because the purpose of our plug-in technology is essentially to dynamically load code.

An important rule of thumb, then, is that if a component needs to be installed to use, don’t hand it over to the system without it being installed. The best plugin framework we know does not follow this principle, so although it has few Hook points, it is because it is not installed activities to the system, so later have to do some hacks.

So the shell scheme is very good. Other frameworks have had this idea for a long time, but they’ve always wanted to wrap a plug-in Activity around a host Activity and then figure out how to implement a mediation relationship. If the plug-in Activity is a real Activity, the plug-in can compile, install, and run normally, which is great for developing plug-ins or directly launching plug-in apps. But because it is a system Activity subclass, it has many methods that cannot be called directly, and it may even need to avoid its super method being called. If the plug-in Activity is not a real Activity, but just a normal class that has similar methods to an Activity, this is much easier; just let the shell Activity hold it and call it. However, the code of this plug-in will be troublesome to compile into an independent App installation and run normally, and there may be many if-else related to the plug-in in the code, which is also not good.

Shadow does a very simple thing by using AOP ideas, using bytecode editing tools, at compile time in the plug-in all activities of the parent class to a normal class, and then let the shell hold the normal type of the parent class to call it without hacking any system implementation. Although this is simple enough, there are some additional issues that need to be addressed, such as the getActivity() method returning no Activity. But Shadow’s implementation addresses these issues.

The principle diagram of Shadow framework is as follows:

Integrated Shadow

Environment to prepare

It is recommended that you compile the clone Shadow code on the command line for the first time.

  • Before compiling, you must set the ANDROID_HOME environment variable.
  • At compile time, gradlew scripts must be used to ensure that the Gradle version configured for the project is used.

You can perform the following compilation tasks during command line test compilation:

./gradlew build
Copy the code

If nothing goes wrong, try opening the project with Android Studio again.

  1. You must open the project using Android Studio version 3.4 or higher.
  2. Instant Run in Android Studio must be disabled.

Then you can select sample-host module in IDE to run directly, as follows:

  • The SDK contains all the code for the SDK
  • Test contains the automated test code for the SDK
  • Sample contains the demo code

Reference: github.com/Tencent/Sha…