Windowmanager. addView(contentView) will be handled by WindowManagerService. WindowManagerService creates the ContentView-related UI carrier Surface. The purpose of this article is to clarify the Surface creation process and how it relates to SurfaceFlinger.

The article is longer, but hope you can insist to read, believe that there will be certain harvest

This article is based on the relatively new source code in the Google Android Repo. Android UI rendering this piece of change is still relatively large, the logic may be and most online articles have some differences.

Before we get started, here are some basics related to SurfaceFlinger:

  • Basics related to SurfaceFlinger

SurfaceFlinger can be said to be the core of The Android UI rendering system. The SurfaceFlinger service will be started when the Android system is started. Its main function is to be called by Android applications to draw (measurement, layout, Draw) after the window (Surface) is rendered to the phone screen. So the entire UI refresh process might look like this:

Of course, the actual UI rendering principle is not as simple as above. In order not to get lost in the process of source tracing, we start our analysis with the following questions:

  1. How does WindowManagerService manage Windows?
  2. How was Surface created?
  3. How does SurfaceFlinger manage UI rendering for multiple applications?
  4. What are the basic units of UI rendering in SurfaceFlinger?

Ok, then we will start to analyze the main process of creating the Surface based on the principle of Android UI display. Because the source code flow of the whole system is very complex, I only post some major nodes in the analysis of the whole process and add some notes when tracing the source code.

As we learned in the last article, the View Wrootimpl manages the entire View Tree. For viewrootimpl.setView (), we can simply think of it as an entry point for a UI rendering operation, so we’ll start with this method:

WindowManagerService manages Windows

ViewRootImpl.java

public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView) { ... //mWindowSession is an AIDL, ViewRootImpl uses it to interact with WindowManagerService //mWindow is an AIDL, WindowManagerService can use this object to interact with the server //mAttachInfo can be understood as a data bean, Can be passed across processes res = mWindowSession. AddToDisplay (mWindow mSeq, mWindowAttributes, getHostVisibility (), mDisplay.getDisplayId(), mWinFrame, mAttachInfo.mContentInsets, mAttachInfo.mStableInsets, mAttachInfo.mOutsets, mAttachInfo.mDisplayCutout, mInputChannel); . }Copy the code

Namely ViewRootImpl. SetView () method will request to add a Window, to WindowManagerService mWindowSession. AddToDisplay () across processes ultimately calls the WindowManagerService. A ddWindow():

WindowManagerService.java

public int addWindow(Session session, IWindow client...) {... Windows final WindowState win = new WindowState(this, Session, Client, Token, parentWindow, appOp[0], seq, attrs, viewVisibility, session.mUid, session.mCanAddInternalSystemWindow); . win.attach(); // Create a SurfaceSession mWindowmap. put(client.asbinder (), win); //mWindowMap is a new set of Windows that WindowManagerService uses to store all current Windows... win.mToken.addWindow(win); // A token can have multiple Win states. In fact, token corresponds to PhoneWindow one by one. . }Copy the code

WindowState is an object used by WindowManagerService to describe a Window of an application. The above comments I marked the win. The attach (), this method can be said to be the starting point of the Window and SurfaceFlinger link, it will eventually call to the Session. The windowAddedLocked () :

Session.java

void windowAddedLocked(String packageName) {
    ...
    if(mSurfaceSession == null) { ... mSurfaceSession = new SurfaceSession(); . Public Final Class SurfaceSession {private long mNativeClient; // SurfaceComposerClient* publicSurfaceSession() {
        mNativeClient = nativeCreate(); 
    }
Copy the code

This calls the native method nativeCreate(), which returns a SurfaceComposerClient pointer. So how is this object created?

The creation of SurfaceComposerClient

android_view_SurfaceSession.cpp

static jlong nativeCreate(JNIEnv* env, jclass clazz) { SurfaceComposerClient* client = new SurfaceComposerClient(); Client ->incStrong((void*)nativeCreate);return reinterpret_cast<jlong>(client);
}
Copy the code

A SurfaceComposerClient object is constructed. And returns its pointer. This object is one for each application and serves as a bridge between the application and SurfaceFlinger. The following method is called when the SurfaceComposerClient pointer is first used:

// This method calls void SurfaceComposerClient the first time it uses a pointer to SurfaceComposerClient::onFirstRef() {... sp<ISurfaceComposerClient> conn; //sf is SurfaceFlinger conn = (rootProducer! = nullptr) ? sf->createScopedConnection(rootProducer) : sf->createConnection(); . }Copy the code

Create an ISurfaceComposerClient object via SurfaceFlinger(which itself has the ability to communicate across processes) :

SurfaceFlinger.cpp

sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() {
    return initClient(new Client(this)); //initClient这个方法其实并没有做什么，
}
Copy the code

That is, a Client object is constructed, and the Client implements ISurfaceComposerClient interface. Is an AIDL object that can communicate across processes. The SurfaceComposerClient can communicate with the SurfaceFlinger. It can also create surfaces and maintain all layers of an application (which we’ll examine below). It is a very important object, so let’s take a look at its composition, and the other things it involves are covered in the following analysis:

Client.h

class Client : public BnSurfaceComposerClient { public: ... void attachLayer(const sp<IBinder>& handle, const sp<Layer>& layer); void detachLayer(const Layer* layer); . Private: // ISurfaceComposerClient Interface. GBP is important, it maintains the application's render Buffer queue virtual Status_t createSurface(... sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp); virtual status_t destroySurface(const sp<IBinder>& handle); // Uint32_t onTransact(Uint32_t code, const Parcel& data, Parcel* reply, Uint32_t flags); . // constant sp<SurfaceFlinger> mFlinger; // protected by mLock DefaultKeyedVector< wp<IBinder>, wp<Layer> > mLayers; // All layers of an application... };Copy the code

Viewrootimpl.setview () ¶

  1. WindowManagerServiceCreated aWindowState. One used to represent the clientWindow
  2. WindowManagerServiceCreated aSurfaceSession.SurfaceSessionWith theSurfaceFlingerBuild the link and create oneSurfaceComposerClientObject, an application has only one of these objects.
  3. SurfaceComposerClientCreated aClientThis object is very important because it maintains the application’s rendering core data and is responsible for communicating with itSurfaceFlingerCommunication.

The diagram below:

After the above steps, the application ofViewRootImplHas beenWindowManagerServiceIdentify, and the application has been associated withSurfaceFlingerEstablish a connection. That created theSurfaceComposerClientandClientobject

Surface is the UI carrier for Windows (View Windows PL). Where is Surface created?

The creation of a Surface

In fact, a view file is a Surface. This can be seen in the ViewRootImpl source code:

ViewRootImpl.java

public final Surface mSurface = new Surface();
Copy the code

That is, the view rule PL creates a new Surface when it is constructed. But the new Surface has no logic. Its constructor is empty. So where was the Surface actually built?

Trigger the Surface creation operation

Let’s go back to viewrootimpl.setView () :

public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView) { ... requestLayout(); // Susion requests layout. Add to render queue first... res = mWindowSession.addToDisplay(mWindow, ...) ; //WindowManagerService creates a WindowState for mWindow... }Copy the code

RequestLayout () is called before a WindowManagerService request is made to create a WindowState, which causes the entire View Tree managed by ViewRootImpl to be rerendered. It will eventually call scheduleTraversals():

void scheduleTraversals() {... mChoreographer.postCallback(Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null); . }Copy the code

Instead of going into detail about Choreographer this article, you can learn about this class by referring to the Android Choreographer source Analysis analysis.

ScheduleTraversals () will post a mTraversalRunnable through Choreographer, which receives the display system’s time pulses (VSync signal, 16ms once), Control to execute this mTraversalRunnable on the next frame render.

But mTraversalRunnable is executed at least after the application has connected to the SurfaceFlinger. This is because SurfaceFlinger schedules the rendering operation, and SurfaceFlinger will not render the application if it has not already created a connection with SurfaceFlinger. So the execution of the mWindowSession. AddToDisplay (mWindow,…). After that, mTraversalRunnable will be executed:

ViewRootImpl.java

final class TraversalRunnable implements Runnable {
    @Override
    public void run() {
        doTraversal(); }}Copy the code

DoTraversal () will be called to ViewRootImpl. PerformTraversals (), most of the students may know that this method is a view of tree measure/layout/control method of the draw:

private void performTraversals() { finalView host = mView; //mView is a Window root View, which is a DecorView for an Activity... relayoutWindow(params, viewVisibility, insetsPending); . performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); . performLayout(lp, mWidth, mHeight); . performDraw(); . }Copy the code

RelayoutWindow (Params, viewVisibility, insetsPending) is used to create the Surface. This method will be through the IPC calls to WindowManagerService. RelayoutWindow () :

ViewRootImpl.java

private int relayoutWindow(WindowManager.LayoutParams params, ...) throws RemoteException { ... int relayoutResult = mWindowSession.relayout(mWindow,... , mSurface); . }Copy the code

I left out a lot of arguments to the mWindowSession.relayout() method, but one very important argument I didn’t leave out is the mSurface. It is an empty Surface object, as analyzed earlier. In fact:

The real Surface was created bySurfaceControlFinished, applicationViewRootImpltheSurfaceIt’s just a pointer to thisSurface

SurfaceControl creates a Surface using SurfaceControl:

MWindowSession. Relayout () will call to WindowManagerService. RelayoutWindow () :

WindowManagerService.java

Public int relayoutWindow(Session Session, IWindow client.... Surface outSurface){ ... result = createSurfaceControl(outSurface, result, win, winAnimator); . } private int createSurfaceControl(Surface outSurface, int result, WindowState win,WindowStateAnimator winAnimator) { ... surfaceController = winAnimator.createSurfaceLocked(win.mAttrs.type, win.mOwnerUid); . surfaceController.getSurface(outSurface); }Copy the code

WinAnimator. CreateSurfaceLocked is actually created a SurfaceControl. Construct the SurfaceControl first, and then construct the Surface.

The creation of SurfaceControl

WinAnimator createSurfaceLocked actually is through SurfaceControl constructor creates a SurfaceControl object, the object function is responsible for maintaining the Surface, the Surface is also responsible for creating by the objects Let’s look at the constructor of this object:

SurfaceControl.java

long mNativeObject; // Member pointer variable, Private SurfaceControl(SurfaceSession Session, String name, int W, int H, int format, int flags, SurfaceControl parent, int windowType, int ownerUid){ ... mNativeObject = nativeCreate(session, name, w, h, format, flags, parent ! = null ? parent.mNativeObject : 0, windowType, ownerUid); . }Copy the code

NativeCreate () is called and a SurfaceControl pointer is returned:

android_view_SurfaceControl.cpp

static jlong nativeCreate(JNIEnv* env, ...) SurfaceComposerClinent sp<SurfaceComposerClient> client(android_view_SurfaceSession_getClient(env, sessionObj)); sp<SurfaceControl> surface; // After the creation is successful, Err = Client ->createSurfaceChecked(String8(name.c_str())), w, h, format, & Surface, flags, parent, windowType, ownerUid); .returnreinterpret_cast<jlong>(surface.get()); // Return the address of this SurfaceControl}Copy the code

The call to SurfaceComposerClient. CreateSurfaceChecked () :

SurfaceComposerClient.cpp

/ / outSurface will point to the newly created SurfaceControl status_t SurfaceComposerClient: : createSurfaceChecked (... sp<SurfaceControl>* outSurface..) { sp<IGraphicBufferProducer> gbp; // This object is very important... err = mClient->createSurface(name, w, h, format, flags, parentHandle, windowType, ownerUid, &handle, &gbp);if(err == NO_ERROR) {outSurface = new SurfaceControl(this, handle, GBP,true);
    }
    return err;
}
Copy the code

The above method actually calls Client.createsurface () to create a Surface. There is an important parameter sp

GBP that we will focus on in the following source analysis. This is because every frame rendered by the application is actually added to the IGraphicBufferProducer to wait for SurfaceFlinger to render it. This process will be examined in the next article, but we will continue to look at how the Surface is created by the Client:

Client.cpp

status_t Client::createSurface(...) {... // GBP passes directly to SurfaceFlingerreturn mFlinger->createLayer(name, this, w, h, format, flags, windowType, ownerUid, handle, gbp, &parent);
}
Copy the code

????? What about the Surface? MFlinger ->createLayer() Well, I don’t understand this sudden shift either, but let’s just know:

SurfaceinSurfaceFlingerThe corresponding entity inLayer

Let’s continue with mFlinger->createLayer()

SurfaceFlinger.cpp

status_t SurfaceFlinger::createLayer(const String8& name,const sp<Client>& client...) { status_t result = NO_ERROR; sp<Layer> layer; / / will create layer switch (flags & ISurfaceComposerClient: : eFXSurfaceMask) {caseISurfaceComposerClient::eFXSurfaceNormal: result = createBufferLayer(client, uniqueName, w, h, flags, format, handle, gbp, &layer); // Note that GBP is not yet constructed.break; . // There are several types of Layer}... result = addClientLayer(client, *handle, *gbp, layer, *parent); // This layer is associated with the client and added to the Client's mLayers collection. .return result;
}
Copy the code

Can be seen from SurfaceFlinger. CreateLayer () method Layer is divided into several kinds. CreateBufferLayer createBufferLayer();

status_t SurfaceFlinger::createBufferLayer(const sp<Client>& client... sp<Layer>* outLayer)
{
    ...
    sp<BufferLayer> layer = new BufferLayer(this, client, name, w, h, flags);
    status_t err = layer->setBuffers(w, h, format, flags); // Set the layer width and heightif(err == NO_ERROR) { *handle = layer->getHandle(); // Create handle * GBP = layer->getProducer(); // create GBP IGraphicBufferProducer *outLayer = layer; // Copy the pointer to the new layer to the outLayer so that the outLayer points to the new BufferLayer}return err;
}
Copy the code

IGraphicBufferProducer(GBP) is an important object that I mentioned earlier because it involves SurfaceFlinger’s rendering logic. Let’s look at the creation logic of this object:

The creation of a IGraphicBufferProducer (GBP)

BufferLayer.cpp

sp<IGraphicBufferProducer> BufferLayer::getProducer() const {
    return mProducer;
}
Copy the code

That is, mProducer is a Layer member variable that is created when the Layer is first used:

void BufferLayer::onFirstRef() {... BufferQueue::createBufferQueue(&producer, &consumer,true); mProducer = new MonitoredProducer(producer, mFlinger, this); . }Copy the code

So an instance of mProducer is MonitoredProducer, but it’s really just a decorator class that delegates its actual functions to the argument producer that constructed it:

BufferQueue.cpp

void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer, ... sp<IGraphicBufferProducer> producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger)); sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core)); // Notice this consumer... *outProducer = producer; *outConsumer = consumer; }Copy the code

So the queueProducer that actually does the work of mProducer is BufferQueueProducer.

So what you do when you construct a SurfaceControl is you create a SurfaceControl and have the SurfaceFlinger create a Layer, There is one IGraphicBufferProducer in Layer, and its instance is BufferQueueProducer.

You can use the following diagram to illustrate the SurfaceControl creation process:

Get the Surface from the SurfaceControl

We look back WindowManagerService. CreateSurfaceControl (), take a look at what the Java layer Surface of the object is a:

WindowManagerService.java

private int createSurfaceControl(Surface outSurface, int result, WindowState win,WindowStateAnimator winAnimator) { ... surfaceController = winAnimator.createSurfaceLocked(win.mAttrs.type, win.mOwnerUid); . surfaceController.getSurface(outSurface); }Copy the code

We have seen above winAnimator. The whole process of createSurfaceLocked, We see surfaceController. GetSurface (outSurface), surfaceController is WindowSurfaceController instances:

//WindowSurfaceController.java void getSurface(Surface outSurface) { outSurface.copyFrom(mSurfaceControl); } //Surface.java public void copyFrom(SurfaceControl other) { ... long surfaceControlPtr = other.mNativeObject; . long newNativeObject = nativeGetFromSurfaceControl(surfaceControlPtr); . mNativeObject = ptr; // mNativeObject points to native Surface}Copy the code

The Surface. The copyFrom () method call nativeGetFromSurfaceControl () to obtain a pointer, the pointer is created according to the previous SurfaceControl Pointers to look for, namely the incoming parameters surfaceControlPtr:

android_view_Surface.cpp

static jlong nativeGetFromSurfaceControl(JNIEnv* env, jclass clazz, jlong surfaceControlNativeObj) { sp<SurfaceControl> ctrl(reinterpret_cast<SurfaceControl *>(surfaceControlNativeObj)); Sp <Surface> Surface(CTRL ->getSurface()); // Create a Surface directly, point to CTRL ->getSurface()if(surface ! = NULL) { surface->incStrong(&sRefBaseOwner); // strong reference}return reinterpret_cast<jlong>(surface.get());
}
Copy the code

The CTRL here points to the SurfaceControl created earlier, tracing back to CTRL ->getSurface():

SurfaceControl.cpp

sp<Surface> SurfaceControl::getSurface() const
{
    Mutex::Autolock _l(mLock);
    if (mSurfaceData == 0) {
        return generateSurfaceLocked();
    }
    returnmSurfaceData; } sp < Surface > SurfaceControl: : generateSurfaceLocked const () {/ / the mGraphicBufferProducer BufferQueueProducer is actually the above analysis mSurfaceData = new Surface(mGraphicBufferProducer,false); 
    return mSurfaceData;
}
Copy the code

That is, a new Nativie Surface is directly returned to the Java layer, and the Surface of the Java layer points to the Surface of the Native layer.

So the actual Surface creation can be represented as follows:

SurfaceControl is also called SurfaceControl.

So far this article has been a long one, but the UI rendering process: how viewrotimPL renders to Surface will be analyzed in the next article.

Welcome to mineAndroid Advanced PlansSee more dry goods

Welcome to follow my wechat official account: Susion Heart

Reference article:

AndroidUI series – a brief introduction to the image rendering mechanism

An article to understand the relationship between The Android graphics system Surface and SurfaceFlinger