Iqiyi has just released its blockbuster open source project Andromeda

Preface: 2018 is destined to be an extraordinary year for IQiyi. From its listing on March 29, the stock price soared to 22.79 USD/share by May 29. Iqiyi continuously releases open source projects in the open source community. As a former iqi artist, I feel honored and blessed for my former employer. Today is the day long Hai of the Infrastructure group announced the open source of his Andromeda project. Long Hai joined iQiyi 17 years ago, used to work in Ele. me, in the annual meeting is one side of the edge. The Andromeda project, reviewed by the company, has been open sourced on Github. Welcome to star fork. Without more words, enter Longhai to introduce his project.

In fact, The componentization of Android has a long history, and there have been some good schemes, especially in the aspect of page jump, such as ARouter of Ali, The hopping protocol of Tmall and DeepLinkDispatch of Airbnb, which use annotations to complete the registration of pages, so as to skillfully realize the route jump.

However, although schemes like ARouter actually support interface routing, unfortunately only single-process interface routing is supported.

At present, due to complex business scenarios, iQiyi App has communication requirements of both single process and cross-process, and also supports Callback calls in cross-process communication as well as global event bus.

Can you design a program that meets these requirements?

This is the background of Andromeda. After identifying the above requirements, a number of solutions were analyzed, and the current solution was chosen to meet the requirements while blocking the entire communication between processes, thus avoiding ugly asynchronous connection code.

The function of Andromeda

Andromeda is now open source.

Andromeda no longer does page-routing because page-hopping is a complete and mature solution. Currently Andromeda includes the following features:

• The local service route is registerLocalService(Class, Object) and getLocalService(Class).

• The remote service route is registerRemoteService(Class, Object) and getRemoteService(Class).

• Global (including all processes) Event bus, subscribe(String, EventListener), publish(Event);

• Remote method callback. If a business interface requires remote callback, IPCCallback can be used when defining the AIDL interface.

Note: The services here are not the services of the four components of Android, but the interfaces and implementations provided. For the sake of distinction, the following services are used in this sense, while services refer to components in Android.

Why do we need to distinguish between local and remote services here?

One of the most important reasons is that local services are not limited in their parameter and return value types, whereas remote services are limited by binder communication.

The appearance of Andromeda, so to speak, completes the last piece of the puzzle for componentization.

A comparison of communication schemes between Andromeda and other components is as follows:

	Ease of use	The IPC performance	Support the IPC	Support for cross-process event buses	Support the IPC Callback
Andromeda	good	high	Yes	Yes	Yes
DDComponentForAndroid	poor	—	No	No	No
ModularizationArchitecture	poor	low	Yes	No	No

Interface dependency or protocol dependency

This discussion is interesting because some people feel that using Events or ModuleBeans as communication carriers between components eliminates the need for each business module to define its own interface and calls the same way.

However, the disadvantages of this approach are also obvious. First, although there is no need to define interfaces, many events need to be defined if events are used in order to adapt to their respective business requirements. If you use ModuleBeans, you need to define many fields for each ModuleBean, and even create a ModuleBean object for another party to call an empty method, which can be costly. In addition, as the number of services increases, more and more fields need to be defined in the ModuleBean corresponding to this module, resulting in higher consumption.

Second, the code is not readable. The way to define Event/ ModuleBeans is not as intuitive as interface calls and is not conducive to project maintenance;

Third, as mentioned in the wechat Android modular architecture reconstruction practice (part 1), “the protocol communication we understand refers to the cross-platform/serialized communication mode, similar to the communication between terminals and servers or restful. This form is now common in terminals. Protocol communication has a very powerful decoupling capability, but it also has a significant cost. Regardless of the form of communication, all protocol definitions need to be accessible to both sides of the communication. It is common to store all protocol definitions in a common area for convenience, similar to the problem raised by events. In addition, if the protocol changes, how to synchronize the two ends becomes a bit more complicated, at least with some framework to implement. Isn’t that a little complicated in an application? It’s not that convenient to use, is it? Let alone how many problems it solves.”

Obviously, protocol communication is too heavy for communication between components, making it inflexible in response to business changes.

Therefore, the method of “interface + data structure” is finally adopted for inter-component communication, and the business interfaces and data structures that need to be exposed are put into a common module.

Implementation of a cross-process routing scheme

Local service routing can be done with a Map.

More troublesome is the remote service, to solve the following problems:

• Make it easy for any two components to communicate, i.e. one component registers its own remote service, which can be easily called by any component

• Make registering and using remote services as simple as using local services by blocking calls

• The efficiency of communication cannot be reduced

Encapsulation bindService

Here is the most easy to think of Android IPC communication mode of traditional encapsulation, namely the bindService () on the basis of encapsulation, such as ModularizationArchitecture WideRouter is to do so, in the open source library architecture diagram is as follows:

This scheme has two obvious flaws:

• Each IPC needs to go through WideRouter, and then forward to the corresponding process. As a result, problems that can be solved by one IPC need to be solved by two IPC, and IPC itself is time-consuming

• Since bindService is asynchronous, you can’t really do a blocking call at all

• The WideConnectService must survive to the end of the process, so that the WideConnectService must live in the process with the longest lifetime. You cannot dynamically configure the process in which the WideConnectService resides, which makes it inconvenient to use

Taking these aspects into consideration, this plan was passed.

Hermes

This is an open source framework written by a colleague of Ele. me before, its biggest feature is that it does not need to write AIDL interface, and can directly call remote interface just like calling local interface.

Its principle is to use dynamic proxy + reflection to replace the static proxy generated by AIDL, but it still uses bindService() in the cross-process aspect, as follows:

Hermes.connect() is essentially bindService(), which suffers from the same problems. In addition, Hermes does not yet make it easy to configure processes and does not yet support IPC modifiers such as in, out, and inout.

However, despite these drawbacks, Hermes is an excellent open source framework that at least provides a way to make IPC communication as simple as local communication.

Final plan

In order to call the remote Service, we need to obtain the IBinder of the communication. The biggest problem of the two schemes is that the IBinder of the remote Service is bound to the Service. Is it necessary to bind the IBinder to the Service? Is it possible to get IBinder without a Service?

You can do it, all you need is a Binder manager.

The core processes

Finally, the registration-use approach was adopted, and the overall architecture is shown as follows:

The core of this architecture is Dispatcher and RemoteTransfer. Dispatcher is responsible for managing the business binder of all processes and the binder of RemoteTransfer in each process. RemoteTransfer manages the service binders for all modules in its process.

Detailed analysis follows.

Each process has a RemoteTransfer, which is responsible for managing remote services of all modules in the process, including registration, deregistration and acquisition of remote services. Remote service interfaces provided by RemoteTransfer are as follows:

interface IRemoteTransfer { oneway void registerDispatcher(IBinder dispatcherBinder); oneway void unregisterRemoteService(String serviceCanonicalName); oneway void notify(in Event event); }Copy the code

This interface is used by the binder manager Dispatcher, where registerDispatcher() reverse-registers its binder with RemoteTransfer, RemoteTransfer can then use the Dispatcher’s proxy to register and unregister services.

During process initialization, RemoteTransfer sends its own information (actually its own binder) to the DispatcherService of the same process as Dispatcher, which notifies the Dispatcher after receiving it. The Dispatcher registers itself reflectivity with RemoteTransfer’s binder, so RemoteTransfer gets the Dispatcher’s proxy.

The process is shown in a flowchart as follows:

The registration process generally occurs when the child process is initialized, but in fact, even if it doesn’t matter when the child process initialization not registered, actually can be postponed to need to own the remote service offers to go out, or you need to get the other processes of the Module of service can also do it all again in the next section will analyze the specific reasons.

The process for remote service registration is as follows:

Dispatcher holds agent binder of RemoteTransfer of all processes and business binder of all services. Remote service interface provided by Dispatcher is IDispatcher, which is defined as follows:

interface IDispatcher { BinderBean getTargetBinder(String serviceCanonicalName); IBinder fetchTargetBinder(String uri); void registerRemoteTransfer(int pid,IBinder remoteTransferBinder); void registerRemoteService(String serviceCanonicalName,String processName,IBinder binder); void unregisterRemoteService(String serviceCanonicalName); void publish(in Event event); }Copy the code

The services provided by Dispatcher are called by RemoteTransfer, and the names of each method are quite sure that everyone can understand them, so I won’t go into details.

Issues with binder acquisition synchronously

One of the issues we haven’t addressed in the previous scenario is the synchronous acquisition of service Binders.

Imagine a scenario where a Module wants to call a service in another process that has already been registered with Dispatcher before the Dispatcher is reverse-registered. How do you get it synchronously?

The core of this question is, how do I get IDispatcher’s binders synchronically?

There is a way to do this through a ContentProvider!

There are two ways to get IBinder directly from a ContentProvider. The easy one is to use the ContentProviderClient, which is called as follows:

public static Bundle call(Context context, Uri uri, String method, String arg, Bundle extras) { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.JELLY_BEAN_MR1) { return context.getContentResolver().call(uri, method, arg, extras); } ContentProviderClient client = tryGetContentProviderClient(context, uri); Bundle result = null; if (null == client) { Logger.i("Attention! ContentProviderClient is null"); } try { result = client.call(method, arg, extras); } catch (RemoteException ex) { ex.printStackTrace(); } finally { releaseQuietly(client); } return result; } private static ContentProviderClient tryGetContentProviderClient(Context context, Uri uri) { int retry = 0; ContentProviderClient client = null; while (retry <= RETRY_COUNT) { SystemClock.sleep(100); retry++; client = getContentProviderClient(context, uri); if (client ! = null) { return client; } //SystemClock.sleep(100); } return client; } private static ContentProviderClient getContentProviderClient(Context context, Uri uri) { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN) { return context.getContentResolver().acquireUnstableContentProviderClient(uri); } return context.getContentResolver().acquireContentProviderClient(uri); }Copy the code

It is possible to carry IBinder in the resulting Bundle, but the problem with this solution is that the ContentProviderClient is not compatible and will crash the first time it is run on some phones, which is obviously unacceptable.

The alternative is to use the ContentResolver query() method to place binder within Cursor as follows:

DispatcherCursor is defined as follows, where the generateCursor() method is used to put binder into the Cursor and the stripBinder() method is used to remove binder from the Cursor.

public class DispatcherCursor extends MatrixCursor { public static final String KEY_BINDER_WRAPPER = "KeyBinderWrapper";  private static Map<String, DispatcherCursor> cursorMap = new ConcurrentHashMap<>(); public static final String[] DEFAULT_COLUMNS = {"col"}; private Bundle binderExtras = new Bundle(); public DispatcherCursor(String[] columnNames, IBinder binder) { super(columnNames); binderExtras.putParcelable(KEY_BINDER_WRAPPER, new BinderWrapper(binder)); } @Override public Bundle getExtras() { return binderExtras; } public static DispatcherCursor generateCursor(IBinder binder) { try { DispatcherCursor cursor; cursor = cursorMap.get(binder.getInterfaceDescriptor()); if (cursor ! = null) { return cursor; } cursor = new DispatcherCursor(DEFAULT_COLUMNS, binder); cursorMap.put(binder.getInterfaceDescriptor(), cursor); return cursor; } catch (RemoteException ex) { return null; } } public static IBinder stripBinder(Cursor cursor) { if (null == cursor) { return null; } Bundle bundle = cursor.getExtras(); bundle.setClassLoader(BinderWrapper.class.getClassLoader()); BinderWrapper binderWrapper = bundle.getParcelable(KEY_BINDER_WRAPPER); return null ! = binderWrapper ? binderWrapper.getBinder() : null; }}Copy the code

BinderWrapper is a wrapper class for binder, which is defined as follows: BinderWrapper

public class BinderWrapper implements Parcelable {    private final IBinder binder;    public BinderWrapper(IBinder binder) {        this.binder = binder;    }    public BinderWrapper(Parcel in) {        this.binder = in.readStrongBinder();    }    public IBinder getBinder() {        return binder;    }    @Override    public int describeContents() {        return 0;    }    @Override    public void writeToParcel(Parcel dest, int flags) {        dest.writeStrongBinder(binder);    }    public static final Creator<BinderWrapper> CREATOR = new Creator<BinderWrapper>() {        @Override        public BinderWrapper createFromParcel(Parcel source) {            return new BinderWrapper(source);        }        @Override        public BinderWrapper[] newArray(int size) {            return new BinderWrapper[size];        }    };}Copy the code

To get back to our problem, we just need to set up a ContentProvider in the same process as the Dispatcher and the problem is solved.

Dispatcher process Settings

Because the Dispatcher is responsible for managing the Binder’s processes, it can’t be taken lightly.

For most apps, the main process is the one that lasts the longest, so just put the Dispatcher in the main process.

However, in some apps, the longest survival time is not necessarily the main process. For example, in some music apps, after the main process is killed, the playback process is still alive. In this case, it is obviously a better choice to place the Dispatcher in the playback process.

To enable developers using Andromeda to configure DispatcherExtension according to their own needs, the DispatcherExtension is provided by the developers in the apply plugin: After ‘org.qiyi.svg. Plugin ‘, you can configure it in Gradle:

dispatcher{    process ":downloader"}Copy the code

Of course, if the main process is the one that lives the longest, no configuration is required, just apply plugin: ‘org.qiyi.svg.plugin’.

Improve the process priorities of service providers

Andromeda is a framework for providing communication. I don’t want to do anything to provide process priorities, but based on past statistics, in order to avoid the binderDied problem as much as possible, At least during communication, the priority of the service provider process must be close to that of the client process to reduce the probability of the service provider process being killed.

In fact, bindService() does just that. BindService () essentially does the following:

• Get the service provider’s binder

• The client uses the bind operation to increase the priority of the Service process

The whole process is shown below

When a remote service is used for the first time in an Activity/Fragment, a bind operation is performed to increase the service provider’s process priority.

In the onDestroy() callback of the Activity/Fragment, unbind() to release the connection.

If you bind a Service, it is not visible to the user.

At compile time, StubService is staked for each process, and in the StubServiceMatcher class, the mapping between the process name and StubService is inserted (by JavAssist at compile time). Then you can get the StubService based on the process name.

The BinderBean retrieved from IDispatcher’s getRemoteService() method contains process name information.

Life cycle management

As mentioned in the previous section, you need to call unbind() in the Activity/Fragment onDestroy() to release the connection, which would be too cumbersome if the unbind() was called by the developer.

We need to be able to listen on the Activity/Fragment callback onDestroy() and unbind() automatically.

It is Glide’s approach to create a listening Fragment using the Fragment/Activity FragmentManager, so that when the Fragment/Activity callback onDestroy(), The Fragment that listens on will also receive a callback, which can be unbind.

The following figure shows how callback listening works:

At that time, I actually considered whether to use Arch ComponentSS launched by Google to deal with the life cycle problem. However, considering that there are still teams that have not access to this set, and the scheme of Arch Components has changed many times, I temporarily adopted this scheme. Arch Components’ solution for lifecycle management will be determined later.

IPCCallback

Why do YOU need IPCCallback?

For time-consuming operations, can we call directly from the work thread on the client side?

Although this is possible, the server may still need to perform time-consuming operations in its own work thread and then call back the results, so in this case the client work thread is a bit redundant.

So for ease of use, you need an IPCCallback that calls back after time-consuming operations are processed on the server side.

For AIDL interfaces that require callbacks, it is defined as follows:

interface IBuyApple {        int buyAppleInShop(int userId);        void buyAppleOnNet(int userId,IPCCallback callback);    }Copy the code

The client calls are as follows:

IBinder buyAppleBinder = Andromeda.getRemoteService(IBuyApple.class); if (null == buyAppleBinder) { return; } IBuyApple buyApple = IBuyApple.Stub.asInterface(buyAppleBinder); if (null ! = buyApple) { try { buyApple.buyAppleOnNet(10, new IPCCallback.Stub() { @Override public void onSuccess(Bundle result) throws RemoteException { ... } @Override public void onFail(String reason) throws RemoteException { ... }}); } catch (RemoteException ex) { ex.printStackTrace(); }}Copy the code

But considering that callbacks are in Binder threads, and most callers want them to be in the main thread, lib encapsulates a BaseCallback for the access party to use, as follows:

IBinder buyAppleBinder = Andromeda.getRemoteService(IBuyApple.class); if (null == buyAppleBinder) { return; } IBuyApple buyApple = IBuyApple.Stub.asInterface(buyAppleBinder); if (null ! = buyApple) { try { buyApple.buyAppleOnNet(10, new BaseCallback() { @Override public void onSucceed(Bundle result) { ... } @Override public void onFailed(String reason) { ... }}); } catch (RemoteException ex) { ex.printStackTrace(); }}Copy the code

Developers can choose according to their own needs.

Event bus

Since the Dispatcher has the RemoteTransfer binder for each process, implementing an event bus on this basis is a piece of cake.

Simply speaking, each RemoteTransfer records the subscribed event information in its own process during event subscription. When an event is published, the publisher notifies the Dispatcher, which notifies the processes, whose RemoteTransfer notifies the event subscribers.

The event

The definition of Event in Andromeda is as follows:

public class Event implements Parcelable { private String name; private Bundle data; . }Copy the code

That is, event = name + data. The data to be transferred is stored in the Bundle during communication. Where the name must be unique throughout the project, otherwise errors may occur. Because of the cross-process transfer, all data can only be wrapped in a Bundle.

Event subscription

Event subscription is simple, and you first need an object that implements the EventListener interface. Then you can subscribe to the events you are interested in, as follows:

    Andromeda.subscribe(EventConstants.APPLE_EVENT,MainActivity.this);Copy the code

MainActivity implements the EventListener interface, which indicates that an event named eventconstnts.apple_event is subscribed.

Event publishing

Event publishing is as simple as calling the publish method as follows:

Bundle bundle = new Bundle(); bundle.putString("Result", "gave u five apples!" ); Andromeda.publish(new Event(EventConstants.APPLE_EVENT, bundle));Copy the code

InterStellar

Two things stood out to me while writing the Andromeda framework. The first was SWT exceptions due to too many business binders (i.e., Android Watchdog Timeout).

The second thing is that in the process of communicating with colleagues, I thought about the possibility of not writing AIDL interface to make remote service as simple as local service.

So you have InterStellar, which can be easily interpreted as an enhanced version of Hermes, but not in the same way, and there is support for IPC modifiers like in, out, inout and oneway.

With the help of InterStellar, you can define a remote interface just like a local interface, like this:

public interface IAppleService {       int getApple(int money);       float getAppleCalories(int appleNum);       String getAppleDetails(int appleNum,  String manifacture,  String tailerName, String userName,  int userId);       @oneway       void oneWayTest(Apple apple);       String outTest1(@out Apple apple);       String outTest2(@out int[] appleNum);       String outTest3(@out int[] array1, @out String[] array2);       String outTest4(@out Apple[] apples);       String inoutTest1(@inout Apple apple);       String inoutTest2(@inout Apple[] apples);   }Copy the code

The implementation of the interface is exactly the same as that of the local service, as follows:

public class AppleService implements IAppleService { @Override public int getApple(int money) { return money / 2; } @Override public float getAppleCalories(int appleNum) { return appleNum * 5; } @Override public String getAppleDetails(int appleNum, String manifacture, String tailerName, String userName, int userId) { manifacture = "IKEA"; tailerName = "muji"; userId = 1024; if ("Tom".equals(userName)) { return manifacture + "-->" + tailerName; } else { return tailerName + "-->" + manifacture; } } @Override public synchronized void oneWayTest(Apple apple) { if(apple==null){ Logger.d("Man can not eat null apple!" ); }else{ Logger.d("Start to eat big apple that weighs "+apple.getWeight()); try{ wait(3000); //Thread.sleep(3000); }catch(InterruptedException ex){ ex.printStackTrace(); } Logger.d("End of eating apple!" ); }} @override public String outTest1(Apple Apple) {if (Apple == null) {Apple = new Apple(3.2f, "Shanghai"); } apple.setWeight(apple.getWeight() * 2); apple.setFrom("Beijing"); return "Have a nice day!" ; } @Override public String outTest2(int[] appleNum) { if (null == appleNum) { return ""; } for (int i = 0; i < appleNum.length; ++i) { appleNum[i] = i + 1; } return "Have a nice day 02!" ; } @Override public String outTest3(int[] array1, String[] array2) { for (int i = 0; i < array1.length; ++i) { array1[i] = i + 2; } for (int i = 0; i < array2.length; ++i) { array2[i] = "Hello world" + (i + 1); } return "outTest3"; } @Override public String outTest4(Apple[] apples) { for (int i = 0; i < apples.length; ++i) { apples[i] = new Apple(i + 2f, "Shanghai"); } return "outTest4"; } @Override public String inoutTest1(Apple apple) { Logger.d("AppleService-->inoutTest1,apple:" + apple.toString()); Apple. SetWeight (3.14159 f); apple.setFrom("Germany"); return "inoutTest1"; } @Override public String inoutTest2(Apple[] apples) { Logger.d("AppleService-->inoutTest2,apples[0]:" + apples[0].toString()); for (int i = 0; i < apples.length; ++ I) {apples[I].setweight (I * 1.5f); apples[i].setFrom("Germany" + i); } return "inoutTest2"; }}Copy the code

You can see that AIDL is not involved at all.

So how does it work?

The answer lies in Transfer. In essence, the Proxy generated after AIDL compilation is actually a static Proxy providing interfaces, so we can actually change it to a dynamic Proxy, passing the service method name and parameters to the service provider, then calling the corresponding method, and finally sending back the results.

The hierarchical architecture of InterStellar is as follows:

For details about InterStellar, you can go to InterStellar Github.

conclusion

Prior to Andromeda, most communication frameworks either didn’t address IPC issues or had inelegant solutions, probably due to a lack of complexity in the business scenario, but Andromeda’s point is to combine local and remote communication, and only then do I feel I have solved the component communication problem completely.

Cross-process communication is encapsulated with a binder, but Andromeda’s innovation is to separate Service from Binder, making it more flexible to use.

Finally, Andromeda is now open source. Welcome to Star and fork, and feel free to issue any questions you have.