“This is the fifth day of my participation in the November Gwen Challenge. See details of the event: The Last Gwen Challenge 2021”.

Recently, the company requested a preliminary study of MediaSession to provide multimedia services for third-party applications. For this purpose, the entire MediaSession architecture is summarized. The summary is as follows:

The Android multimedia services architecture mainly involves the following key classes

MediaBrowser: multimedia browsing client

MediaBrowserService: multimedia browsing service

MediaSession: connects the multimedia session to the player and responds to broadcast control operations

MediaController: multimedia controller

How MeidaBrowser works

MeidaBrowser is used for browsing multimedia information, and the connection and subscription process are analyzed in detail.

The MediaBrowser connects to the server

The structure of the MediaBrowser

public MediaBrowser(Context context, ComponentName serviceComponent, ConnectionCallback callback, Bundle rootHints) { //... Omit the code mContext = context; mServiceComponent = serviceComponent; mCallback = callback; mRootHints = rootHints == null ? null : new Bundle(rootHints); }Copy the code

The constructor takes four parameters: serviceComponent is the MediaBroswerService component to connect to, and callback is the connection state callback. RootHints are some of the other initialization parameters we pass.

MediaBrowser#connect initiates a connection

public void connect() { if (mState ! = CONNECT_STATE_DISCONNECTING && mState ! = CONNECT_STATE_DISCONNECTED) { throw new IllegalStateException("connect() called while neither disconnecting nor " + "disconnected (state=" + getStateLabel(mState) + ")"); } mState = CONNECT_STATE_CONNECTING; Mhandler.post (new Runnable() {@override public void run() {// Some other connection check //... Omit connection check code final Intent Intent = new Intent (MediaBrowserService. SERVICE_INTERFACE); intent.setComponent(mServiceComponent); mServiceConnection = new MediaServiceConnection(); boolean bound = false; Bound = McOntext. bindService(intent, mServiceConnection, context.bind_auto_create); } catch (Exception ex) { Log.e(TAG, "Failed binding to service " + mServiceComponent); } if (! bound) { // Tell them that it didn't work. forceCloseConnection(); mCallback.onConnectionFailed(); } / /... }}); }Copy the code

MediaBrowserService accepts bindings

When a server receives a request to bind the service, it calls its onBind and returns a Binder object

@Override
    public IBinder onBind(Intent intent) {
        if (SERVICE_INTERFACE.equals(intent.getAction())) {
            return mBinder;
        }
        return null;
    }
Copy the code

The implementation class for mBinder is MediaBroswerService#ServiceBinder, which has the following apis

The MediaBrowser binding service callback succeeded

MediaBrowser#MediaServiceConnection#onServiceConnected is called back when the binding service is successful

@Override public void onServiceConnected(final ComponentName name, Final IBinder binder) {postOrRun(new Runnable() {@override public void run() {//... / / Save their binder / / this is the server returned ServiceBinder mServiceBinder = IMediaBrowserService. The Stub. AsInterface (binder); // We make a new mServiceCallbacks each time we connect so that we can drop // responses from previous connections. / / mServiceCallbacks implementation class is inherited ServiceCallbacks IMediaBrowserServiceCallbacks Stub mServiceCallbacks = getNewServiceCallbacks(); // mark the current connection state as mState = CONNECT_STATE_CONNECTING; // Call connect, which is async. When we get a response from that we will // say that we're connected. try { if (DBG) { Log.d(TAG, "ServiceCallbacks.onConnect..." ); dump(); } // mRootHints is the initialization parameter mServiceBinder.connect(McOntext.getpackagename (), mRootHints, mServiceCallbacks) passed by the constructor;  } catch (RemoteException ex) { // Connect failed, which isn't good. But the auto-reconnect on the service // will take over and we will come back. We will also get the //  onServiceDisconnected, which has all the cleanup code. So let that do // it. Log.w(TAG, "RemoteException during connect for " + mServiceComponent); if (DBG) { Log.d(TAG, "ServiceCallbacks.onConnect..." ); dump(); }}}}); }Copy the code

MediaBrowserService#ServiceBinder receives the requested connection

@Override public void connect(final String pkg, final Bundle rootHints, final IMediaBrowserServiceCallbacks callbacks) { //... Mhandler.post (new Runnable() {@override public void run() {final IBinder b = callbacks.asbinder (); // Clear out the old subscriptions. We are getting new ones. mConnections.remove(b); final ConnectionRecord connection = new ConnectionRecord(); connection.pkg = pkg; connection.pid = pid; connection.uid = uid; connection.rootHints = rootHints; connection.callbacks = callbacks; mCurConnection = connection; connection.root = MediaBrowserService.this.onGetRoot(pkg, uid, rootHints); mCurConnection = null; // If they didn't return something, don't allow this client. // If they didn't return something, don't allow this client. OnConnectFailed if (Connection. root == null) {log. I (TAG, "No root for client " + pkg + " from service " + getClass().getName()); try { callbacks.onConnectFailed(); } catch (RemoteException ex) { Log.w(TAG, "Calling onConnectFailed() failed. Ignoring. " + "pkg=" + pkg); } } else { try { mConnections.put(b, connection); b.linkToDeath(connection, 0); if (mSession ! = null) {/ / connection success callback callbacks. The onConnect (connection. Root. GetRootId (), mSession and connection. The root. GetExtras ()); } } catch (RemoteException ex) { Log.w(TAG, "Calling onConnect() failed. Dropping client. " + "pkg=" + pkg); mConnections.remove(b); }}}}); }Copy the code

MediaBrowser receives a connection success callback

A successful callback does not go directly to the MediaBrowser, but rather to the ServiceCallbacks object passed in when it calls its connection. Call MediaBrowser#onServiceConnected indirectly through ServiceCallbacks

private void onServiceConnected(final IMediaBrowserServiceCallbacks callback, final String root, final MediaSession.Token session, Final Bundle extra) {handler.post (new Runnable() {@override public void run() {mRootId = root; mMediaSessionToken = session; mExtras = extra; mState = CONNECT_STATE_CONNECTED; McAllback.onconnected (); // Subscribe again for (Entry<String, Subscription> subscriptionEntry if the Subscription method was called before the connection was successful: mSubscriptions.entrySet()) { String id = subscriptionEntry.getKey(); Subscription sub = subscriptionEntry.getValue(); List<SubscriptionCallback> callbackList = sub.getCallbacks(); List<Bundle> optionsList = sub.getOptionsList(); for (int i = 0; i < callbackList.size(); ++i) { try { mServiceBinder.addSubscription(id, callbackList.get(i).mToken, optionsList.get(i), mServiceCallbacks); } catch (RemoteException ex) { // Process is crashing. We will disconnect, and upon reconnect we will // automatically reregister. So nothing to do here. Log.d(TAG, "addSubscription failed with RemoteException parentId=" + id); }}}}}); }Copy the code

Summary of connection process

MediaBrowser obtains the internal ServiceBinder class of MediaBrowserService by binding the service. The ServiceCallbacks are passed to ServiceBinder. When ServiceBinder determines that the connection is normal, ServiceBinder sends the corresponding data back through onConnect.

MediaBrowser#getRoot and MediaBrowser#getExtras return parameters in the BrowserRoot object returned by MediaBrowserService#onGetRoot

 public static final class BrowserRoot {
    private final String mRootId;
    private final Bundle mExtras;
 }
Copy the code

MediaBrowser subscription

MediaBrowser initiates the subscription

MediaBrowser has multiple SUBSCRIBE method overload methods, but they all end up calling subscribeInternal

private void subscribeInternal(String parentId, Bundle options, SubscriptionCallback callback) { // Check arguments. if (TextUtils.isEmpty(parentId)) { throw new IllegalArgumentException("parentId cannot be empty."); } if (callback == null) { throw new IllegalArgumentException("callback cannot be null"); } // Update or create the subscription. Subscription sub = mSubscriptions.get(parentId); if (sub == null) { sub = new Subscription(); mSubscriptions.put(parentId, sub); } sub.putCallback(mContext, options, callback); // If we are connected, tell the service that we are watching. If we aren't connected, // the service will be told when we connect. if (isConnected()) { try { if (options == null) { mServiceBinder.addSubscriptionDeprecated(parentId, mServiceCallbacks); } // The SubscriptionCallback object is not passed, But passed its token and mServiceCallbacks mServiceBinder. AddSubscription (parentId, callback mToken, options, mServiceCallbacks); } catch (RemoteException ex) { // Process is crashing. We will disconnect, and upon reconnect we will // automatically reregister. So nothing to do here. Log.d(TAG, "addSubscription failed with RemoteException parentId=" + parentId); }}}Copy the code

ServiceBinder receives subscription messages

ServiceBinder#addSubscription is called when the client initiates a subscription

@Override public void addSubscription(final String id, final IBinder token, final Bundle options, final IMediaBrowserServiceCallbacks callbacks) { mHandler.post(new Runnable() { @Override public void run() { final IBinder b = callbacks.asBinder(); // Get the record for the connection // Because binder objects are different when binding service return, Get the ConnectionRecord is different. Final ConnectionRecord Connection = McOnnection.get (b); if (connection == null) { Log.w(TAG, "addSubscription for callback that isn't registered id=" + id); return; } MediaBrowserService.this.addSubscription(id, connection, token, options); }}); }Copy the code

MediaBrowserService#addSubscription

private void addSubscription(String id, ConnectionRecord connection, IBinder token, Bundle options) { // Save the subscription List<Pair<IBinder, Bundle>> callbackList = connection.subscriptions.get(id); if (callbackList == null) { callbackList = new ArrayList<>(); } for (Pair<IBinder, Bundle> callback : callbackList) { if (token == callback.first && MediaBrowserUtils.areSameOptions(options, callback.second)) { return; } } callbackList.add(new Pair<>(token, options)); / / to the corresponding subscription saved, at the time of change when the data for a callback connection. Subscriptions. The put (id, callbackList); // send the results performLoadChildren(id, connection, options); }Copy the code

MediaBrowserService#performLoadChildren is used to notify MediaBrowserService to load data

MediaBrowserService#performLoadChildren

private void performLoadChildren(final String parentId, final ConnectionRecord connection, final Bundle options) { final Result<List<MediaBrowser.MediaItem>> result = new Result<List<MediaBrowser.MediaItem>>(parentId) { @Override void onResultSent(List<MediaBrowser.MediaItem> list, @ResultFlags int flag) { if (mConnections.get(connection.callbacks.asBinder()) ! = connection) { if (DBG) { Log.d(TAG, "Not sending onLoadChildren result for connection that has" + " been disconnected. pkg=" + connection.pkg + " id=" + parentId); } return; } //applyOptions is used to support data paging, List< mediabrowser. MediaItem> filteredList = (flag & RESULT_FLAG_OPTION_NOT_HANDLED) = 0? applyOptions(list, options) : list; final ParceledListSlice<MediaBrowser.MediaItem> pls = filteredList == null ? null : new ParceledListSlice<>(filteredList); Try {/ / a data object callback callbacks are MediaBrowser# ServiceCallbacks connection. The callbacks. OnLoadChildrenWithOptions (parentId, PLS. options); } catch (RemoteException ex) { // The other side is in the process of crashing. Log.w(TAG, "Calling onLoadChildren() failed for id=" + parentId + " package=" + connection.pkg); }}}; mCurConnection = connection; if (options == null) { onLoadChildren(parentId, result); } else { onLoadChildren(parentId, result, options); } mCurConnection = null; // This check is obvious: the Service onLoadChildren must call detach() or sendResult() to send the result if (! result.isDone()) { throw new IllegalStateException("onLoadChildren must call detach() or sendResult()" + " before returning for package=" + connection.pkg + " id=" + parentId); }}Copy the code

The performLoadChildren method is used to notify the client of data changes. This method is called not only when a subscription is initiated, but also indirectly through notifyChildrenChanged when data changes. Therefore, the client should call unsubscribe for each parentId subscription when it is not in use.

At the same time, if you need to conduct paging query when initiating subscription, you can pass:

MediaBrowser.EXTRA_PAGE
MediaBrowser.EXTRA_PAGE_SIZE
Copy the code

To perform data paging queries

MediaBrowser receives the data callback

The callback data is the object of MediaBrowser# ServiceCallbacks, its onLoadChildrenWithOptions implementation logic is very simple, in the presence of the MediaBrowser onLoadChildren down with it

private void onLoadChildren(final IMediaBrowserServiceCallbacks callback, final String parentId, final ParceledListSlice list, final Bundle options) { mHandler.post(new Runnable() { @Override public void run() { // Check that there hasn't been a disconnect or a different // ServiceConnection. if (! isCurrent(callback, "onLoadChildren")) { return; } if (DBG) { Log.d(TAG, "onLoadChildren for " + mServiceComponent + " id=" + parentId); } // Check that the subscription is still subscribed. final Subscription subscription = mSubscriptions.get(parentId); if (subscription ! = null) { // Tell the app. SubscriptionCallback subscriptionCallback = subscription.getCallback(mContext, options); if (subscriptionCallback ! = null) { List<MediaItem> data = list == null ? null : list.getList(); if (options == null) { if (data == null) { subscriptionCallback.onError(parentId); } else { subscriptionCallback.onChildrenLoaded(parentId, data); } } else { if (data == null) { subscriptionCallback.onError(parentId, options); } else { subscriptionCallback.onChildrenLoaded(parentId, data, options); } } return; } } if (DBG) { Log.d(TAG, "onLoadChildren for id that isn't subscribed id=" + parentId); }}}); }Copy the code

The logic is simple: detect the current connection and get the corresponding SubscriptionCallback to notify the app of the data callback.

Summary of subscription process

MediaBrowser naturally supports paging query. In the bundle of SUBSCRIBE, query parameters can be passed. When subscribes, parentId will be used as the storage key to store the corresponding callback, and the corresponding callback will be called when the corresponding data changes. Simultaneous subscribe and unsubscribe should come in pairs.

The creation of MediaSession

//MediaSession private final MediaSession.Token mSessionToken; private final MediaController mController; private final ISession mBinder; private final CallbackStub mCbStub; public MediaSession(@NonNull Context context, @NonNull String tag, int userId) { if (context == null) { throw new IllegalArgumentException("context cannot be null."); } if (TextUtils.isEmpty(tag)) { throw new IllegalArgumentException("tag cannot be null or empty"); } mMaxBitmapSize = context.getResources().getDimensionPixelSize( com.android.internal.R.dimen.config_mediaMetadataBitmapMaxSize); mCbStub = new CallbackStub(this); MediaSessionManager manager = (MediaSessionManager) context .getSystemService(Context.MEDIA_SESSION_SERVICE); Session mBinder = manager.createsession (mCbStub, tag, userId); // createSession mBinder = manager.createsession (mCbStub, tag, userId); mSessionToken = new Token(mBinder.getController()); mController = new MediaController(context, mSessionToken); } catch (RemoteException e) { throw new RuntimeException("Remote error creating session.", e); }}Copy the code

MediaSessionManager creates ISession objects

private final ISessionManager mService;
​
public @NonNull ISession createSession(@NonNull MediaSession.CallbackStub cbStub,
            @NonNull String tag, int userId) throws RemoteException {
        return mService.createSession(mContext.getPackageName(), cbStub, tag, userId);
    }
Copy the code

The mService type is ISessionManager and its initialization is in the constructor of MediaSessionManager where cbStub is initialized in the constructor of CallbackStub

public MediaSessionManager(Context context) {
        // Consider rewriting like DisplayManagerGlobal
        // Decide if we need context
        mContext = context;
        IBinder b = ServiceManager.getService(Context.MEDIA_SESSION_SERVICE);
        mService = ISessionManager.Stub.asInterface(b);
    }
Copy the code

MediaSessionService has a service corresponding to MEDIA_SESSION_SERVICE

publishBinderService(Context.MEDIA_SESSION_SERVICE, mSessionManagerImpl);
Copy the code

So the createSession of MediaSessionManager is ultimately the called MediaSessionService internal class SessionManagerImpl

SessionManagerImpl create ISession

@Override public ISession createSession(String packageName, ISessionCallback cb, String tag, int userId) throws RemoteException { final int pid = Binder.getCallingPid(); final int uid = Binder.getCallingUid(); final long token = Binder.clearCallingIdentity(); try { enforcePackageName(packageName, uid); int resolvedUserId = ActivityManager.handleIncomingUser(pid, uid, userId, false /* allowAll */, true /* requireFull */, "createSession", packageName); if (cb == null) { throw new IllegalArgumentException("Controller callback cannot be null"); } return createSessionInternal(pid, uid, resolvedUserId, packageName, cb, tag) .getSessionBinder(); } finally { Binder.restoreCallingIdentity(token); }}Copy the code

CreateSessionInternal will call createSessionLocked

MediaSessionService create MediaSessionRecord

private MediaSessionRecord createSessionLocked(int callerPid, int callerUid, int userId,
            String callerPackageName, ISessionCallback cb, String tag) {
        FullUserRecord user = getFullUserRecordLocked(userId);
        if (user == null) {
            Log.wtf(TAG, "Request from invalid user: " +  userId);
            throw new RuntimeException("Session request from invalid user.");
        }
​
        final MediaSessionRecord session = new MediaSessionRecord(callerPid, callerUid, userId,
                callerPackageName, cb, tag, this, mHandler.getLooper());
        try {
            cb.asBinder().linkToDeath(session, 0);
        } catch (RemoteException e) {
            throw new RuntimeException("Media Session owner died prematurely.", e);
        }
​
        user.mPriorityStack.addSession(session);
        mHandler.postSessionsChanged(userId);
​
        if (DEBUG) {
            Log.d(TAG, "Created session for " + callerPackageName + " with tag " + tag);
        }
        return session;
    }
Copy the code

You can see that MediaSessionRecord#getSessionBinder is called at the end

public MediaSessionRecord(int ownerPid, int ownerUid, int userId, String ownerPackageName,
            ISessionCallback cb, String tag, MediaSessionService service, Looper handlerLooper) {
        mOwnerPid = ownerPid;
        mOwnerUid = ownerUid;
        mUserId = userId;
        mPackageName = ownerPackageName;
        mTag = tag;
        mController = new ControllerStub();
        mSession = new SessionStub();
        mSessionCb = new SessionCb(cb);
        mService = service;
        mContext = mService.getContext();
        mHandler = new MessageHandler(handlerLooper);
        mAudioManager = (AudioManager) mContext.getSystemService(Context.AUDIO_SERVICE);
        mAudioManagerInternal = LocalServices.getService(AudioManagerInternal.class);
        mAudioAttrs = new AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build();
    }
​
    /**
     * Get the binder for the {@link MediaSession}.
     *
     * @return The session binder apps talk to.
     */
    public ISession getSessionBinder() {
        return mSession;
    }
Copy the code

So MediaSession holds the interprocess proxy object SessionStub while MediaSessionRecord holds the CallbackStub object that MeidaSession passes along. This allows MediaSession and MediaSessionRecord to call each other across processes with binder.

MediaSession# Token creation

 mSessionToken = new Token(mBinder.getController());
Copy the code

MBinder’s implementation class is MediaSessionRecord#SessionStub

@Override
public ISessionController getController() {
    return mController;
}
Copy the code

MController is an instance of ControllerStub, which means the Token holds the ControllerStub’s cross-process agent.

The creation of a MediaController

public static final class Token implements Parcelable { private ISessionController mBinder; /** * @hide */ public Token(ISessionController binder) { mBinder = binder; } ISessionController getBinder() { return mBinder; }}Copy the code

When the MediaController is created, the Token is passed directly

public MediaController(@NonNull Context context, @nonnull MediaSession.Token Token) {// Note that Binder is mediasessionRecordcontrollerStub this(context, token.getbinder ()); } public MediaController(Context context, ISessionController sessionBinder) { if (sessionBinder == null) { throw new IllegalArgumentException("Session token cannot be null"); } if (context == null) { throw new IllegalArgumentException("Context cannot be null"); } mSessionBinder = sessionBinder; mTransportControls = new TransportControls(); mToken = new MediaSession.Token(sessionBinder); mContext = context; }Copy the code

This gives the MediaController the ability to interact with the MediaSessionRecord.

MediaSession summary:

Mediassession #ISession Object initialization process

MediaSession UML class diagram relationships

MediaSession contains the internal class CallbackStub, where CallbakStub inherits from isessionCallback.stub MediaSession The CallbackStub object is passed to MediaSessionRecord during construction so that the MediaSessionRecord cross-process callback notifies MediaSession.

MediaSession initializes ISession mBinder through MediaSessionManager An instance of ISession is a SessionStub so that MediaSession can interact with the MediaSessionRecord across processes.

The MediaSession internal class Token initialization needs to rely on ISessionController and its implementation class is ControllerStub

mSessionToken = new Token(mBinder.getController());
Copy the code

The Token holds the ControllerStub, which means that the Token can play control through the Token’s member variable ISessionController mBinder. This is why the MediaController needs to use the Mediassession #Token when it is initialized.

The MediaController#CallbackStub implementation is consistent with MediaSession, which listens for the corresponding callback by registering it with MediaSessionRecord.