An overview of the

Note: This article is based on the Android 10 source code, for the sake of brevity, the source code may be omitted. If the content of the article is wrong, welcome to point out, common progress! Leave a “like” if you think it’s good

• Blog links.

Before starting to parse Choreographer’s source flow, let’s take a look at some basic concepts and background and go straight to the big guys’ summary – “At Last” series: Android Screen Refresh Mechanisms – VSync, Choreographer Understands everything! . The Display system generally consists of CPU, GPU and Display. CPU is responsible for calculating frame data and handing the calculated data to GPU, which will render the graphics data and store it in buffer(image buffer) after rendering. Then the Display (screen or Display) is responsible for rendering the data in the buffer onto the screen. Of course, the actual process is much more complicated. For example, the drawing process of APP involves software drawing and hardware acceleration, as well as writing rendering data to the graphics cache through Surface. In addition, the SurfaceFlinger process will synthesize these different Surface(Layer) data into the display cache and so on.

Recently, I have been sorting out the process related to Android graphics system. In order to have a systematic understanding of these knowledge points, I read others’ blog summaries while reading the source code of the system. I have summarized SurfaceFlinger startup and working process before. This article will take a look at Choreographer working with Vsync signals, then update the direction of Surface and Layer data during View drawing, and finally combine these together to connect the logic of the entire graphics system. See these knowledge points again apart next, should be able to have a better result. Source code analysis process is inevitable errors, if found welcome correction ha!

Choreographer: Choreographer controls when VSync signals are received before starting a painting task, ensuring that the painting has a full 16.6ms.

Entry: scheduleTraversals

After calling context.startActivity, AMS does some processing. Back again by Binder call target process ActivityThread. HandleResumeActivity method, in this way can the Activity’s onResume callback target and makeVisible method, The makeVisible method completes the WindowManager.addView process, which calls the Viewrootimpl. setView method and, internally, the scheduleTraversals method. Finally, performTraversals, and then measure, layout, and draw.

. In addition, look at the View invalidate method source code, can also find the last invoked to ViewRootImpl. ScheduleTraversals method.

// ViewRootImpl
final ViewRootHandler mHandler = new ViewRootHandler();

void scheduleTraversals(a) {
    if(! mTraversalScheduled) {// Ensure that multiple changes at the same time are refreshed only once, such as TextView setText() twice in a row
        mTraversalScheduled = true;
        // Add a synchronization barrier to ensure that VSync is executed immediately
        mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier();
        mChoreographer.postCallback(Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null);
        // ...}}// mTraversalRunnable is an instance of Runnable
final class TraversalRunnable implements Runnable {
    @Override
    public void run(a) { doTraversal(); }}void doTraversal(a) {
    if (mTraversalScheduled) {
        mTraversalScheduled = false;
        // Remove the synchronization barrier
        mHandler.getLooper().getQueue().removeSyncBarrier(mTraversalBarrier);
        // Execute View measure, layout, draw processperformTraversals(); }}Copy the code

The mTraversalScheduled field is used first to ensure that multiple changes at the same time will only be refreshed once, and then a synchronization barrier is added to the current thread’s MessageQueue to prevent synchronization messages from being drawn as soon as the VSync signal comes, rather than waiting for previous synchronization messages. Call mChoreographer. PostCallback () method sends a callback can be carried in the next frame, TraversalRunnable–>doTraversal()–>performTraversals()–> render when the next VSync signal comes. Synchronization barriers can be referenced to the Android messaging mechanism.

Choreographer instantiation

Let’s start by looking at Choreographer’s instantiation process, which is instantiated in the ViewRootImpl constructor. The instantiation timing of ViewRootImpl can follow the principles of the Android-Window mechanism.

// ViewRootImpl is created in WindowManager.addView
public ViewRootImpl(Context context, Display display) {
    // ...
    mChoreographer = Choreographer.getInstance();
    // ...
}

public final class Choreographer {
    private static volatile Choreographer mMainInstance;

    private static final ThreadLocal<Choreographer> sThreadInstance = new ThreadLocal<Choreographer>() {
        @Override
        protected Choreographer initialValue(a) {
            Looper looper = Looper.myLooper();
            // VSYNC_SOURCE_APP = 0; -- APP
            // VSYNC_SOURCE_SURFACE_FLINGER = 1; -- SurfaceFlinger
            Choreographer choreographer = new Choreographer(looper, VSYNC_SOURCE_APP);
            // ...
            returnchoreographer; }};public static Choreographer getInstance(a) {
        returnsThreadInstance.get(); }}Copy the code

Choreographer, like Looper, is a thread singleton implemented by ThreadLocal.

public final class Choreographer {
    // 4.1 The default above is true
    // Enable/disable vsync for animations and drawing.
    private static final boolean USE_VSYNC = SystemProperties.getBoolean("debug.choreographer.vsync".true);

    // VSync event receiver
    private final FrameDisplayEventReceiver mDisplayEventReceiver;

    private Choreographer(Looper looper, int vsyncSource) {
        mLooper = looper;
        mDisplayEventReceiver = USE_VSYNC ? new FrameDisplayEventReceiver(looper, vsyncSource) : null;
        // ...}}Copy the code

The Choreographer instantiation creates a FrameDisplayEventReceiver object, it is used to register Vsync signal.

Vsync signal registration

DisplayEventReceiver

MDisplayEventReceiver FrameDisplayEventReceiver type of instance, instantiated in the Choreographer construction method, its parent class for DisplayEventReceiver.

public abstract class DisplayEventReceiver {
    public static final int VSYNC_SOURCE_APP = 0;
    private long mReceiverPtr;

    public DisplayEventReceiver(Looper looper, int vsyncSource) {
        mMessageQueue = looper.getQueue();
        // Register the VSYNC listener
        mReceiverPtr = nativeInit(new WeakReference<DisplayEventReceiver>(this), mMessageQueue, vsyncSource);
    }

    private static native long nativeInit(WeakReference<DisplayEventReceiver> receiver, MessageQueue messageQueue, int vsyncSource);
}
Copy the code

nativeInit

NativeInit is a native method, in fact now frameworks/base/core/jni/android_view_DisplayEventReceiver CPP in:

static jlong nativeInit(JNIEnv* env, jclass clazz, jobject receiverWeak, jobject messageQueueObj, jint vsyncSource) {
    sp<MessageQueue> messageQueue = android_os_MessageQueue_getMessageQueue(env, messageQueueObj);
    sp<NativeDisplayEventReceiver> receiver = new NativeDisplayEventReceiver(env, receiverWeak, messageQueue, vsyncSource);
    status_t status = receiver->initialize(a); receiver->incStrong(gDisplayEventReceiverClassInfo.clazz); // retain a reference for the object
    return reinterpret_cast<jlong>(receiver.get());
}
Copy the code

NativeDisplayEventReceiver inherited from DisplayEventDispatcher:

DisplayEventReceiver::DisplayEventReceiver(ISurfaceComposer::VsyncSource vsyncSource) {
    sp<ISurfaceComposer> sf(ComposerService::getComposerService());
    if(sf ! =NULL) {
        mEventConnection = sf->createDisplayEventConnection(vsyncSource);
        if(mEventConnection ! =NULL) {
            mDataChannel = std::make_unique<gui::BitTube>();
            mEventConnection->stealReceiveChannel(mDataChannel.get()); }}}sp<IDisplayEventConnection> SurfaceFlinger::createDisplayEventConnection( ISurfaceComposer::VsyncSource vsyncSource) {
    if (vsyncSource == eVsyncSourceSurfaceFlinger) {
        return mSFEventThread->createEventConnection(a); }else {
        / / vsyncSource is APP
        return mEventThread->createEventConnection();
    }
}
Copy the code

From the SurfaceFlinger start and work flow Can know EventThread. CreateEventConnection created a interested in Vsync signal connection, specific logic can read this article. The initialize method looks like this:

// frameworks/base/libs/androidfw/DisplayEventDispatcher.cpp
status_t DisplayEventDispatcher::initialize(a) {
    // DisplayEventReceiver mReceiver;
    status_t result = mReceiver.initCheck(a);int rc = mLooper->addFd(mReceiver.getFd(), 0, Looper::EVENT_INPUT, this.NULL);
    if (rc < 0) {
        return UNKNOWN_ERROR;
    }
    return OK;
}
Copy the code

MReceiver is DisplayEventReceiver type instances, is located in the frameworks/native/libs/GUI/DisplayEventReceiver CPP. MLooper ->addFd(mreceiver.getfd (), 0, Looper::EVENT_INPUT, this, NULL) When there is a connection that is interested in the Vsync signal and the Vsync signal is received, the data is sent to the mReceiver and then called back to the handleEvent method in DisplayEventDispatcher, Specific source code refer to SurfaceFlinger startup and workflow addFd resolution.

Request Vsync signal

Above have registered an interest in Vsync signal connection, the Vsync signal arrives, will callback to DisplayEventDispatcher. The handleEvent method. So next we need to request the Vsync signal. Look at the calling code above: mChoreographer postCallback (Choreographer. CALLBACK_TRAVERSAL mTraversalRunnable, null), its invocation chain is: Choreographer.postCallback -> Choreographer.postCallbackDelayedInternal -> Choreographer.scheduleFrameLocked -> Choreographer scheduleVsyncLocked method, save space, specific code is not posted:

private void postCallbackDelayedInternal(int callbackType, Object action, Object token, long delayMillis) {
    synchronized (mLock) {
        final long now = SystemClock.uptimeMillis();
        final long dueTime = now + delayMillis;
        // Add a Callback to the corresponding type of CallbackQueue
        mCallbackQueues[callbackType].addCallbackLocked(dueTime, action, token);
        // ...}}private void scheduleVsyncLocked(a) {
    mDisplayEventReceiver.scheduleVsync();
}

// DisplayEventReceiver
public void scheduleVsync(a) {
    if (mReceiverPtr == 0) {
        // ...
    } else{ nativeScheduleVsync(mReceiverPtr); }}Copy the code

Then we get to the native layer code:

// frameworks/base/core/jni/android_view_DisplayEventReceiver.cpp
static void nativeScheduleVsync(JNIEnv* env, jclass clazz, jlong receiverPtr) {
    sp<NativeDisplayEventReceiver> receiver = reinterpret_cast<NativeDisplayEventReceiver*>(receiverPtr);
    status_t status = receiver->scheduleVsync(a);// ...
}

// frameworks/base/libs/androidfw/DisplayEventDispatcher.cpp
status_t DisplayEventDispatcher::scheduleVsync(a) {
    if(! mWaitingForVsync) {// ...
        // mReceiver is the DisplayEventReceiver instance
        status_t status = mReceiver.requestNextVsync(a); mWaitingForVsync =true;
    }
    return OK;
}

// frameworks/native/libs/gui/DisplayEventReceiver.cpp
status_t DisplayEventReceiver::requestNextVsync(a) {
    if(mEventConnection ! =NULL) {
        // Request to receive the next Vsync signal callback
        mEventConnection->requestNextVsync(a);return NO_ERROR;
    }
    return NO_INIT;
}
Copy the code

As you can see, the requestNextVsync function is finally called. The logic for requestNextVsync has been parsed in the SurfaceFlinger startup and workflow to request the next Vsync signal. You can wake up the EventThread and call back to the APP when the Vsync signal arrives.

Vsync callback process

After the Vsync signal came, it came here.

DisplayEventDispatcher::handleEvent

int DisplayEventDispatcher::handleEvent(int.int events, void*) {
    // ...
    // Distribute Vsync in a subclass
    dispatchVsync(vsyncTimestamp, vsyncDisplayId, vsyncCount);
    // Return 1 in order to keep the listener callback added in addFd
    // See SurfaceFlinger for more details
    return 1; // keep the callback
}

void NativeDisplayEventReceiver::dispatchVsync(nsecs_t timestamp, int32_t id, uint32_t count) {
    JNIEnv* env = AndroidRuntime::getJNIEnv();
    ScopedLocalRef<jobject> receiverObj(env, jniGetReferent(env, mReceiverWeakGlobal));
    if (receiverObj.get()) {
        env->CallVoidMethod(receiverObj.get(), gDisplayEventReceiverClassInfo.dispatchVsync, timestamp, id, count);
    }
    mMessageQueue->raiseAndClearException(env, "dispatchVsync");
}

int register_android_view_DisplayEventReceiver(JNIEnv* env) {
    // ...
    jclass clazz = FindClassOrDie(env, "android/view/DisplayEventReceiver");
    gDisplayEventReceiverClassInfo.clazz = MakeGlobalRefOrDie(env, clazz);
    gDisplayEventReceiverClassInfo.dispatchVsync = GetMethodIDOrDie(env,
            gDisplayEventReceiverClassInfo.clazz, "dispatchVsync"."(JII)V");
    return res;
}
Copy the code

By, it will call into Java android/view/DisplayEventReceiver dispatchVsync method:

public abstract class DisplayEventReceiver {
    private void dispatchVsync(long timestampNanos, int builtInDisplayId, int frame) {
        onVsync(timestampNanos, builtInDisplayId, frame);
    }

    public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {}}Copy the code

In subclasses FrameDisplayEventReceiver onVsync implementation.

FrameDisplayEventReceiver.onVsync

private final class FrameDisplayEventReceiver extends DisplayEventReceiver implements Runnable {
    private boolean mHavePendingVsync;
    private long mTimestampNanos;
    private int mFrame;

    @Override
    public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {
        if(builtInDisplayId ! = SurfaceControl.BUILT_IN_DISPLAY_ID_MAIN) {/ / the main display
            Log.d(TAG, "Received vsync from secondary display, but we don't support "
                    + "this case yet. Choreographer needs a way to explicitly request "
                    + "vsync for a specific display to ensure it doesn't lose track "
                    + "of its scheduled vsync.");
            scheduleVsync();
            return;
        }

        long now = System.nanoTime();
        if (timestampNanos > now) {
            timestampNanos = now;
        }

        if (mHavePendingVsync) {
            Log.w(TAG, "Already have a pending vsync event. There should only be one at a time.");
        } else {
            mHavePendingVsync = true;
        }

        mTimestampNanos = timestampNanos;
        mFrame = frame;
        // The run method is called
        Message msg = Message.obtain(mHandler, this);
        msg.setAsynchronous(true);
        mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
    }

    @Override
    public void run(a) {
        mHavePendingVsync = false; doFrame(mTimestampNanos, mFrame); }}Copy the code

Choreographer.doFrame

After receiving the Vsync signal, Choreographer executes back to Choreographer:

private static final int SKIPPED_FRAME_WARNING_LIMIT = SystemProperties.getInt("debug.choreographer.skipwarning".30);

void doFrame(long frameTimeNanos, int frame) {
    final long startNanos;
    synchronized (mLock) {
        if(! mFrameScheduled) {return; // no work to do
        }
        // Schedule execution time
        long intendedFrameTimeNanos = frameTimeNanos;
        startNanos = System.nanoTime();
        final long jitterNanos = startNanos - frameTimeNanos;
        if (jitterNanos >= mFrameIntervalNanos) {
            // Whether the time is longer than one frame, because the synchronization barrier is added, but if there are synchronization tasks in progress, it will cause doFrame delay
            // Count the number of frames lost
            final long skippedFrames = jitterNanos / mFrameIntervalNanos;
            if (skippedFrames >= SKIPPED_FRAME_WARNING_LIMIT) {
                // By default, the number of lost frames exceeds 30
                Log.i(TAG, "Skipped " + skippedFrames + " frames! "
                    + "The application may be doing too much work on its main thread.");
            }
            final long lastFrameOffset = jitterNanos % mFrameIntervalNanos;
            frameTimeNanos = startNanos - lastFrameOffset;
        }

        if (frameTimeNanos < mLastFrameTimeNanos) {
            scheduleVsyncLocked(); // Request the next Vsync signal
            return;
        }

        if (mFPSDivisor > 1) {
            long timeSinceVsync = frameTimeNanos - mLastFrameTimeNanos;
            if (timeSinceVsync < (mFrameIntervalNanos * mFPSDivisor) && timeSinceVsync > 0) {
                scheduleVsyncLocked(); // Request the next Vsync signal
                return;
            }
        }

        mFrameInfo.setVsync(intendedFrameTimeNanos, frameTimeNanos);
        mFrameScheduled = false;
        mLastFrameTimeNanos = frameTimeNanos;
    }

    try {
        // Execute by type, there are four types within Choreographer
        AnimationUtils.lockAnimationClock(frameTimeNanos / TimeUtils.NANOS_PER_MS);

        mFrameInfo.markInputHandlingStart();
        doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);

        mFrameInfo.markAnimationsStart();
        doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);

        mFrameInfo.markPerformTraversalsStart();
        doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);

        doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
    } finally{ AnimationUtils.unlockAnimationClock(); }}Copy the code

Choreographer.doCallbacks

void doCallbacks(int callbackType, long frameTimeNanos) {
    CallbackRecord callbacks;
    synchronized (mLock) {
        final long now = System.nanoTime();
        // Find callbackRecords that reach execution time based on the specified type CallbackkQueue
        callbacks = mCallbackQueues[callbackType].extractDueCallbacksLocked(now / TimeUtils.NANOS_PER_MS);
        // ...
    }
    // Iterates through all tasks in the queue
    for(CallbackRecord c = callbacks; c ! =null; c = c.next) { c.run(frameTimeNanos); }}private static final class CallbackRecord {
    public CallbackRecord next;
    public long dueTime;
    public Object action; // Runnable or FrameCallback
    public Object token;

    public void run(long frameTimeNanos) {
        if (token == FRAME_CALLBACK_TOKEN) {
            ((FrameCallback)action).doFrame(frameTimeNanos);
        } else { // Call Runnable's run method directly((Runnable)action).run(); }}}Copy the code

This is where the actual drawing begins.

conclusion

• Choreographer: Make CPU/GPU drawing start when VSYNC arrives. Choreographer will create a connection indicating interest in the Vsync signal when it is initialized, request the next Vsync signal through the postCallback method when the signal arrives, and only start the drawing task when the signal arrives.
• Only when the App registers to listen for the next Vsync signal will it receive the incoming Vsync callback. If the interface stays the same, the App will not receive Vsync events every 16.6ms, but the underlying layer will still switch frames at this frequency (also by listening for Vsync signals). That is, when the interface remains unchanged, the screen is refreshed every 16.6ms, but the CPU or GPU does not go through the drawing process.
• When the View requests a refresh, the task does not start immediately, but only when the next Vsync signal arrives. The interface will not refresh immediately after the measure/layout/draw process runs, but will be cached and displayed when the next VSync signal arrives.
• There are two main reasons for frame loss: one is to traverse the drawing View tree and calculate screen data over 16.6ms; The second is that the main thread has been processing other time-consuming messages, causing the drawing task to be slow to start (synchronization barriers do not completely solve this problem).
• By Choreographer. GetInstance (). PostFrameCallback () to listen on frame rate, reference postFrameCallback usage principle and its usage.

Reading this article it is recommended to start with SurfaceFlinger Startup and Workflows and then combine Choreographer’s workflows, You can get a clear idea of how Vsync signals coordinate drawing tasks on the App side and SurfaceFlinger compositing tasks.

A diagram summarises Choreographer’s workflow: