This article introduces the cross-process design of Shadow and the principles behind the plug-in Service. I’m going to do these two parts together because they’re related. This article assumes that the reader is not familiar with Android’s Service and Binder communications, so it will cover some things that might seem simple to you.

Cross-process design is necessary for plug-in frameworks

On Android, applications can be multi-process. This should be a very advanced design in mobile operating systems, many mobile operating systems, embedded systems are not supported. Multi-process programs bring great benefits and more complexity to programming.

The four components (Activity, Service, Receiver, and Provider) in the Android system are all components that can communicate across processes (the components in the following paragraphs refer to these components). Each component can be configured to a specific process in AndroidManifest. Android doesn’t allow applications to manage their own process lifecycle. But since we only need to Start an Intent to Start the component that belongs to that process, we can Start that process. Using Java’s generic process manipulation API, such as the system.exit () method, can kill a process. This ease of doing has led many Android developers to assume that they can manage processes themselves. This is actually the wrong way to think about Android processes. Android processes are designed so that when the system receives a request for a component (such as a start Activity or bind Service), it checks to see if the process registered in the component’s AndroidManifest has already started. If the process is not already started, the system starts the process first, then constructs an application-registered Application object and calls the attachBaseContext() method of the Application object. Then construct all contentProviders that are registered and should be in the process, initialize them, and call their onCreate() method. Ensure that all subsequent components, including the onCreate() method of the Application object that has not yet been called, use all of the process’s contentProviders properly. The onCreate() method of the Application object is then called. Finally, initialize the components that would need to use the process. Therefore, the process is started according to the requirements of the component. Furthermore, this design should allow the process to terminate according to the requirements of the component. In fact, when all components of a process are no longer needed, such as Activity Finish, Service stop, or no bind, the system will consider the process unnecessary. At this point, the system decides to reclaim the process and kill it. Of course, the system can also recycle processes at “necessary” moments, such as running out of memory, or recycle activities and services that are not in the foreground when they are running out of memory. Therefore, when we shut down a process with system.exit () or Crash, the System will not assume that the process is no longer needed. Probably to avoid an infinite loop Crash, the components of a Crash will be considered no longer needed by the system. However, if the process has other active components, or if there are multiple activities in the Activity stack, and the top Activity crashes, the Activity under it should be exposed and become active. In this case, since the system decides that the process is still needed by other components, it resets the process creation process and starts the components that should be active. Therefore, it is important to understand that components to the system are not represented and recorded in the memory of their own running processes, but in the form of records in the system administration process. The Activity is the most special of these components. When you write an Activity, you cannot simply think of it as an object. To further understand, an Activity has persistent states, which are represented by savedInstanceState. So, the biggest difference between an Activity and a Service is not that an Activity has an interface while a Service has no interface. The biggest difference between an Activity and a Service is that an Activity is stateful while a Service is stateless.

There are many advantages to placing components in a separate process, essentially with separate memory resources around the process. Two things are necessary for a plug-in framework. First, plug-ins are generally hot updated, and their quality requirements may be lower. Once Crash occurs, components of other processes will not be affected. For example, a lobby interface is displayed in the host’s main process, where a button jumps to the plug-in. If the plugin crashes after a single process starts, the host’s lobby interface is not affected at all. If the plug-in is also in the host’s main process, this will cause the lobby interface to be recreated as the process restarts. Second, Android JVM does not support reverse loading of Native dynamic library, so different versions of the same SO library in the same process cannot be loaded at the same time, nor can they be loaded in a different way, which will cause so library conflict between the plug-in and the host.

Multiple processes also introduce more complexity, which is its disadvantage. For example, all parameters of a cross-process call must be serializable objects; When communicating across processes, the opposite process may not be started or may be dead. Cross-process communication is abnormal, the entire stack of cross-process calls is not contiguous, and exception objects are generally not serialized for cross-process transfers. How to control the plug-in process exit or restart for use by another business. In addition, the process starts slowly.

Cross-process design for Shadow

Based on the above two points, the basic model of Shadow plug-in framework is Manager, LoadParameters and Loader. Manager works in the process where the host enters the plug-in interface, is responsible for downloading and installing the plug-in, and then controls Loader to start the plug-in by encapsulating the plug-in information in LoadParameters. LoadParameters is a serializable structure that can be transmitted across processes. Loader works in the plug-in process and is responsible for running the plug-in without installation and solving the core problems of the plug-in framework.

A key part of cross-process design is a Service in Shadow called PluginProcessService, or PPS for short.

PPS has several functions:

  1. Represents the life cycle of the plug-in process. It triggers the creation of the plug-in process and is responsible for self-destruction.
  2. Receives a reverse-registered plugin file path Manager (UuidManager, more on plugin package management in future articles) for Loader to find the plugin file path installed by Manager.
  3. Load the Runtime and Loader for dynamic implementation.
  4. Obtain the Binder interface of the Loader.
  5. Enable services in plug-ins to work across processes.

As we reviewed earlier, the start of a process must be triggered by a component. A Service without an interface is a good choice because we usually “preload” the plug-in, perhaps silently starting the plug-in’s Application object, or starting the plug-in’s Service, etc. Also, in order for the system to know that the plug-in process is useful, there must be active components in the process. The components in our plug-in are all uninstalled components that the system does not know exist and cannot rely on. We can’t rely on the shell proxy components because we are a fully dynamic plug-in framework, and those shell proxy components are part of the plug-in and haven’t been loaded yet, so we can’t rely on them either. This requires a Service that is responsible for starting the plug-in process, so it is called PluginProcessService. The Bind semantics of the Service are normal here as well. The Manager starts the PPS as a Bind until the host decides it no longer needs the plug-in, and then unbinds the PPS through the Manager. The Binder Manager gets through Bind is PPSController, which manipulates PPS and enables the host to use the “plug-in service”. So PPS is a real Service.

How to implement plug-in Service

Another reason for choosing a Service to trigger the start of the plug-in process is that the plug-in process must have at least one Service registered with the host in order for the plug-in process’s plug-in Service to communicate across processes like a normal Service. This involves a basic knowledge of Binder, which is that Binder is a centralized cross-process communication framework. Each Binder has a local end for functionality and a remote end for other processes to invoke functionality. Binders are local, but remote. In fact, a local Binder is transferred across processes through another Binder that already exists on the remote end. This local Binder is automatically sent to the Binder’s central manager to register and generate the remote end. The newly generated remote end is exported through the remote end of that existing Binder. The problem of the first Binder may naturally come to mind here. To put it simply, the first Binder was specially dealt with in the design. You can Google the detailed design by yourself. Therefore, for the Service to work properly across processes, the Service’s Binder must be transferred once through an existing Binder. Therefore, the easiest way to do this is through PPS’s Binder.

Therefore, we design the Loader itself as a plug-in Service (dynamic-loader). Because in a fully dynamic design, the Loader is not operated directly by the code in the host, but by the dynamically implemented Manager. Therefore, Loader and Manager are implemented dynamically, so there is no need for interfaces on Loader to be fixed in PPS. Only the necessary methods for loading Runtime and Loader are kept on PPS. Binder of Loader itself communicates to Manager process through PPS cross-process first, thus making Binder of Loader become cross-process Binder. Then the bindPluginService method is exposed on the Loader and then the Binder of the plug-in Service is transferred to other processes through the Binder of the Loader, that is, the plug-in Service can really work for other processes. Our plug-in Service implementation is that simple. It can be seen that Shadow’s plug-in Service does not have a separate proxy shell Service. It only relies on a PPS to achieve unlimited number of plug-in Service support.

Why are Shadow services not implemented in AIDL?

This is because Binder cross-process calls are likely to fail and cannot Crash violently. Therefore, PPS and Dynamic-loader binders are semi-hand-written binders. Semi-manual is the ability to write code from Mr. Aidl and copy it to add a custom serializable Exception. Manager cross-process operation Loader can Catch exceptions.

You can have multiple PPS

Due to the full-dynamic design, there can be multiple Manager implementations in a single host. A Manager implementation can also operate on multiple PPS at the same time, simply by inheriting PPS registrations in different processes. Since Loader and Runtime are dynamic, different plug-in processes can use different versions of Loader.

To improve the

Shadow’s Sample, as well as our own business, specifies the process name for the shell proxy component. For our business, these shells are actually left over from the old framework in the host for Shadow reuse. In fact, after developing the PPS for Shadow, we realized that these shell components should be able to apply android: MultiProcess features.

According to the document: developer.android.com/guide/topic…

Android: When multiProcess is true, the Activity and Provider act in the same process in which the Intent was launched.

Since PPS has determined the process of the plug-in, start the plug-in Activity with PPS to make the shell work in the process of PPS. This allows fewer shell components to be registered when multiple plug-in processes are applied to the same host.

You are welcome to experiment and propose a PR to improve this problem.

Github.com/Tencent/Sha…

PS: The newly registered gold digging account, the exposure of the article is very low. Choosing to share is also a way to attract more developers to Shadow. So please support me and give me a thumbs up so that I can continue to share.