This time let’s talk about cross-process transfer of Surface. We know that a view is drawn on a Surface, so how does the Surface information transfer across processes?

Below, we analyze the following issues:

  • What about Surface? Is it a buffer?
  • If not, what is the relationship between surface and buffer?
  • How does Surface deliver across processes?

1. Surface source code analysis

public class Surface implements Parcelable {
    private static final String TAG = "Surface";
    // Guarded state.
    final Object mLock = new Object(); // protects the native state
    private String mName;
    long mNativeObject; // package scope only for SurfaceControl access
    private long mLockedObject;
    private int mGenerationId; // incremented each time mNativeObject changes
    private final Canvas mCanvas = new CompatibleCanvas();
    }
Copy the code

As you can see, surface is parcelable, which means it can be passed across processes. Since it is parcelable, two functions are required:

  • writeToParcel
  • readFromParcel

Let’s look at the implementation of these two functions in detail.

1.1 writeToParcel ()

 @Override
    public void writeToParcel(Parcel dest, int flags) {
        if (dest == null) {
            throw new IllegalArgumentException("dest must not be null");
        }
        synchronized (mLock) {
            // NOTE: This must be kept synchronized with the native parceling code
            // inframeworks/native/libs/Surface.cpp dest.writeString(mName); dest.writeInt(mIsSingleBuffered ? 1:0); nativeWriteToParcel(mNativeObject, dest); }if ((flags & Parcelable.PARCELABLE_WRITE_RETURN_VALUE) != 0) {
            release();
        }
    }
Copy the code

The code here is simple, just a few lines:

  • Write the name
  • Whether the write is singleBuffer
  • Writes a pointer to the native layer

The key is the nativeWriteToParcel function:

- > / frameworks/base/core/jni android_view_Surface. CPP static void nativeWriteToParcel (env JNIEnv *, jclass clazz, jlong nativeObject, jobject parcelObj) { Parcel* parcel = parcelForJavaObject(env, parcelObj);if (parcel == NULL) {
       doThrowNPE(env);
      return; } // Restore Native layer surface object sp< surface > self(reinterpret_cast< surface *>(nativeObject)) through Java layer pointer; parcel->writeStrongBinder( self ! = 0? IInterface::asBinder(self->getIGraphicBufferProducer()) : NULL); }Copy the code

Look at the source website: xref

Here we can also see that the Surface object is actually a pair, there is a Surface in the application layer and a Surface in the Native layer. MesageQueue and Looper are the same if you have seen the source code for the Handler mechanism. There is a Java layer and a Native layer. But I don’t want to expand much here, so let’s move on.

1.2 the self – > getIGraphicBufferProducer ()

----> /frameworks/av/media/libstagefright/filters/GraphicBufferListener.h


sp<IGraphicBufferProducer> getIGraphicBufferProducer() const {
        return mProducer;
}
Copy the code

So what we see here is that we have this IGraphicBufferProducer object, what is that object? So let’s stop here and look at this readFromParcel method and see what this class is doing

1.3 readFromParcel ()

   public void readFromParcel(Parcel source) {
        if (source == null) {
            throw new IllegalArgumentException("source must not be null");
        }

        synchronized(mLock) { mName = source.readString(); mIsSingleBuffered = source.readInt() ! =0;
            setNativeObjectLocked(nativeReadFromParcel(mNativeObject, source));
        }
Copy the code

It also does simple things:

  • Read the Name
  • Read if it is a singleBuffer and set it to mIsSingleBuffered
  • Read a pointer to the native layer and assign it to the variable mNativeObject

1.4 nativeReadFromParcel ()

static jlong nativeReadFromParcel(JNIEnv* env, jclass clazz, jlong nativeObject, jobject parcelObj) {... Parcel* parcel = parcelForJavaObject(env, parcelObj); sp<Surface> self(reinterpret_cast<Surface *>(nativeObject));
    sp<Surface> sur;
    if(surfaceShim.graphicBufferProducer ! =nullptr) {
        // we have a new IGraphicBufferProducer, create a new Surface for it
        sur = new Surface(surfaceShim.graphicBufferProducer, true); . }Copy the code

The logic of this paragraph is:

  • Fetch a Parcel object from the Native layer based on the Java layer’s Parcel object
  • Get the native layer surface object based on the pointer
  • Read a Binder object from the Parcel, and this Binder object is oneIGraphicBufferProducerobject
  • And then, according to thisIGraphicBufferProducerA Surface object is reconstructed
  • Finally, return the newly created Surface object

Here, DO not know whether to see dizzy, let’s briefly summarize:

  • For surface in The Java layer, its core is the Surface object in the Native layer
  • For surface objects in the Native layer, its core lies inIGraphicBufferProducer

Because I am not going to cover Surface drawing this time, the analysis of this Binder object – IGraphicBufferProducer – will be left for next time (if I remember to fill in the holes).

How to transmit the Surface of the Activity?

As we know, if an application wants to draw, it needs to apply for a piece of memory from SurfaceFlinger. The general process is as follows:

This application process will involve the transfer of surface, so how is surface transferred to the application terminal?

Let’s roll up our sleeves right now.

2.1 performTraversals ()

-- -- -- -- -- > categories: ViewRootImpl/ / 1
    private void performTraversals(a) {
        // cache mView since it is used so much below...
        finalView host = mView; . relayoutResult = relayoutWindow(params, viewVisibility, insetsPending); . }/ / 2
        private int relayoutWindow(WindowManager.LayoutParams params, int viewVisibility,
            boolean insetsPending) throws RemoteException {
            // Notice the last parameter, mSurface
            int relayoutResult = mWindowSession.relayout(mWindow, mSeq, params,
                (int) (mView.getMeasuredWidth() * appScale + 0.5 f),
                (int) (mView.getMeasuredHeight() * appScale + 0.5 f), viewVisibility,
                insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0, frameNumber,
                mWinFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets,
                mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingDisplayCutout,
                mPendingMergedConfiguration, mSurface);
                
            }
Copy the code

Here, a new object appears ————WindowSession, what does it do?

2.2 IWindowSession

/frameworks/base/core/java/android/view/IWindowSession.aidl

interface IWindowSession {
    int add(IWindow window, int seq, in WindowManager.LayoutParams attrs,
            in int viewVisibility, out Rect outContentInsets, out Rect outStableInsets,
            out InputChannel outInputChannel);
    int addToDisplay(IWindow window, int seq, in WindowManager.LayoutParams attrs,
            in int viewVisibility, in int layerStackId, out Rect outContentInsets,
            out Rect outStableInsets, out Rect outOutsets, out InputChannel outInputChannel);
    int addWithoutInputChannel(IWindow window, int seq, in WindowManager.LayoutParams attrs,
            in int viewVisibility, out Rect outContentInsets, out Rect outStableInsets);
    int addToDisplayWithoutInputChannel(IWindow window, int seq, in WindowManager.LayoutParams attrs,
            in int viewVisibility, in int layerStackId, out Rect outContentInsets,
            out Rect outStableInsets);
    void remove(IWindow window);
}
Copy the code

WindowSessioin is an AIDL file that communicates with WMS, opening a communication channel.

Let’s look at how IWindowSession is initialized:

You can see that its implementation class is Session, so you can keep tracking.

2.3 relayout (..)

class Session extends IWindowSession.Stub implements IBinder.DeathRecipient {
     @Override
    public int relayout(IWindow window, int seq, WindowManager.LayoutParams attrs,
            int requestedWidth, int requestedHeight, int viewFlags, int flags, long frameNumber,
            Rect outFrame, Rect outOverscanInsets, Rect outContentInsets, Rect outVisibleInsets,
            Rect outStableInsets, Rect outsets, Rect outBackdropFrame,
            DisplayCutout.ParcelableWrapper cutout, MergedConfiguration mergedConfiguration,
            Surface outSurface) {...int res = mService.relayoutWindow(this, window, seq, attrs, requestedWidth, requestedHeight, viewFlags, flags, frameNumber, outFrame, outOverscanInsets, outContentInsets, outVisibleInsets, outStableInsets, outsets, outBackdropFrame, cutout, mergedConfiguration, outSurface); . }}Copy the code

As you can see, it ends up calling the relayoutWindow method of WMS, focusing on the argument passed at the end: outSurface. The outSurface is actually empty at this point, it’s just passing a shell to WMS.

Next, we need to look at how WMS handles the empty shell passed to WMS. How do you fill it with data?

2.4 service. RelayoutWindow (..)

-- -- -- -- > categories: WindowManagerService.java

 public int relayoutWindow(Session session, IWindow client, int seq, LayoutParams attrs,
            int requestedWidth, int requestedHeight, int viewVisibility, int flags,
            long frameNumber, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
            Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame,
            DisplayCutout.ParcelableWrapper outCutout, MergedConfiguration mergedConfiguration,
            Surface outSurface) {... result = createSurfaceControl(outSurface, result, win, winAnimator);if(surfaceController ! =null) {
            surfaceController.getSurface(outSurface);
            if (SHOW_TRANSACTIONS) Slog.i(TAG_WM, " OUT SURFACE " + outSurface + ": copied"); }... WindowSurfaceController} -- -- -- -- -- ".java
    void getSurface(Surface outSurface) {
        outSurface.copyFrom(mSurfaceControl);
    }
            
Copy the code

The logic of this code is as follows:

  • To create aSurfaceControl
  • fromSurfaceControlTo copy data to the previous empty shellSurfaceIn the

So how does it do that?

Let’s move on to the copyFrom() function:

2.5 surface. CopyFrom ()

---->Surface.java

 public void copyFrom(SurfaceControl other) {...longsurfaceControlPtr = other.mNativeObject; .longnewNativeObject = nativeGetFromSurfaceControl(surfaceControlPtr); . setNativeObjectLocked(newNativeObject); }}Copy the code

Analyze what these lines of code do:

  • fromSurfaceControlObject to obtain a pointer to a native object
  • Create a native layer from this native layer objectsurfaceobject
  • And then put this native layersurfaceObject with the Java layersurfaceObjects are bound together

Put the Native layer code here to make it clearer:

-----> /frameworks/base/core/jni/android_view_Surface.cpp

static jlong nativeGetFromSurfaceControl(JNIEnv* env, jclass clazz, jlong surfaceControlNativeObj) {
    /* * This is used by the WindowManagerService just after constructing * a Surface and is necessary for returning the Surface reference to * the caller. At this point, we should only have a SurfaceControl. This method is called by WMS to return a pointer to the Native layer Surface to the Java layer */

    sp<SurfaceControl> ctrl(reinterpret_cast<SurfaceControl *>(surfaceControlNativeObj));
    sp<Surface> surface(ctrl->getSurface());
    if(surface ! =NULL) {
        surface->incStrong(&sRefBaseOwner);
    }
    return reinterpret_cast<jlong>(surface.get());
}
Copy the code
-----> /frameworks/native/libs/gui/SurfaceControl.cpp
sp<Surface> SurfaceControl::getSurface() const
{
    Mutex::Autolock _l(mLock);
    if (mSurfaceData == 0) {
        return generateSurfaceLocked();
    }
    return mSurfaceData;
}


sp<Surface> SurfaceControl::generateSurfaceLocked() const
{
    // This surface is always consumed by SurfaceFlinger, so the
    // producerControlledByApp value doesn't matter; using false.
    // Create a Surface using GraphicBufferProducer
    mSurfaceData = new Surface(mGraphicBufferProducer, false);

    return mSurfaceData;
}

Copy the code

Well, at this point, the entire flow of Surface cross-process communication is over.

So let’s finish up with a review.

Third, summary

  • Application byViewRootImplCreate an emptySurface
  • throughIWindowSessionLeave this emptySurfacePassed to theWMS
  • WMSthroughSurfaceControlCreate a Native layerSurfaceAnd through the pointer willSurfaceWith Java layerSurfaceBind to completeSurfaceTransfer across processes

Iv. Problem solving

  • surfaceIt’s just a shell, not a buffer object,surfaceIt contains a buffer that can be producedbinderThe object, that is, the objectGraphicBufferProducer
  • surfaceCross process delivery, essentiallyGraphicBufferProducerThe transfer
  • surfaceThe cross-process transfer process is summarized above