The Android interface is stuck because frames drop, and frames drop because the speed of producing frames can’t keep up with the speed of consuming frames.
The consumption rate of frames is linked to the screen refresh rate. The screen is like a comic strip. If 60 frames are played in a second, the consumption rate of one frame is 1000/60 = 16.6ms, that is, the screen will remove the display content of the next frame every 16.6ms. It sounds like losing something that can’t be found, but it actually describes the behavior of screen hardware by “missing an opportunity to display content.”
Why does the screen not fetch the display content? You have to look at the software. With that in mind, read the Framework source code.
This is the fifth article in the long knowledge series of read source code.
Read the source code long knowledge better RecyclerView | click listener
Android custom controls | source there is treasure in the automatic line feed control
Android custom controls | three implementation of little red dot (below)
Reading knowledge source long | dynamic extension class and bind the new way of the life cycle
Reading knowledge source long | Android caton true because “frame”?
Choreographer
The ViewRootImpl is the root View for all views in an Activity, which initiates the View tree traversal:
public final class ViewRootImpl implements ViewParent.View.AttachInfo.Callbacks.ThreadedRenderer.DrawCallbacks {
/ / choreographer
Choreographer mChoreographer;
// Trigger to traverse the View tree
void scheduleTraversals(a) {
if(! mTraversalScheduled) { mTraversalScheduled =true;
mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier();
// Throw View tree traversal tasks to Choreographer
mChoreographer.postCallback(Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null);
if (!mUnbufferedInputDispatch) {
scheduleConsumeBatchedInput();
}
notifyRendererOfFramePending();
pokeDrawLockIfNeeded();
}
}
// Walk through the View tree task
final class TraversalRunnable implements Runnable {
@Override
public void run(a) { doTraversal(); }}// Build the traversal View tree task instance
final TraversalRunnable mTraversalRunnable = new TraversalRunnable();
}
Copy the codeThe ViewRootImpl triggers View tree traversal by throwing tasks to Choreographer.
Choreographer is a class under the Android. view package that literally means “Choreographer”, which implicitly means “need to synchronize the two”, Choreographer is about synchronizing movement with rhythm. Choreographer is about synchronizing “draw content” with “vSYNC signals”.
Storing the drawing task
public final class Choreographer {
// Enter the task
public static final int CALLBACK_INPUT = 0;
// Animation task
public static final int CALLBACK_ANIMATION = 1;
// View tree traverses tasks
public static final int CALLBACK_TRAVERSAL = 2;
/ / COMMIT task
public static final int CALLBACK_COMMIT = 3;
// Hold a chained array of tasks
private final CallbackQueue[] mCallbackQueues;
// Main thread message handler
private final FrameHandler mHandler;
// Discard the drawing task
public void postCallback(int callbackType, Runnable action, Object token) {
postCallbackDelayed(callbackType, action, token, 0);
}
// Delay throwing the drawing task
public void postCallbackDelayed(int callbackType,
Runnable action, Object token, long delayMillis) {... postCallbackDelayedInternal(callbackType, action, token, delayMillis); }// Throw the concrete implementation of the drawing task
private void postCallbackDelayedInternal(int callbackType, Object action, Object token, long delayMillis) {
synchronized (mLock) {
final long now = SystemClock.uptimeMillis();
final long dueTime = now + delayMillis;
// 1. Temporarily store the drawing task in the chain structure according to the type
mCallbackQueues[callbackType].addCallbackLocked(dueTime, action, token);
// 2. Subscribe to the next vSYNC signal
if (dueTime <= now) {
// Subscribe to the next vSYNC signal immediately
scheduleFrameLocked(now);
} else {
Subscribe to vSYNC signals at some point in the future
Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_CALLBACK, action);
msg.arg1 = callbackType;
msg.setAsynchronous(true); mHandler.sendMessageAtTime(msg, dueTime); }}}// Main thread message handler
private final class FrameHandler extends Handler {
@Override
public void handleMessage(Message msg) {
switch (msg.what) {
...
case MSG_DO_SCHEDULE_CALLBACK:
Subscribe to vSYNC signal at future point in time
doScheduleCallback(msg.arg1);
break; }}}void doScheduleCallback(int callbackType) {
synchronized (mLock) {
if(! mFrameScheduled) {final long now = SystemClock.uptimeMillis();
if (mCallbackQueues[callbackType].hasDueCallbacksLocked(now)) {
// Subscribe to the next vSYNC signal
scheduleFrameLocked(now);
}
}
}
}
}
Copy the codeAfter Choreographer receives a new draw task, it performs two actions:
- Draw tasks into the chain:
public final class Choreographer {
// Draw the task chain
private final class CallbackQueue {
// Task header
private CallbackRecord mHead;
// Draw tasks into the chain (in ascending order of time)
public void addCallbackLocked(long dueTime, Object action, Object token) {
CallbackRecord callback = obtainCallbackLocked(dueTime, action, token);
CallbackRecord entry = mHead;
if (entry == null) {
mHead = callback;
return;
}
/ / insert
if (dueTime < entry.dueTime) {
callback.next = entry;
mHead = callback;
return;
}
// Middle insert or tail insert
while(entry.next ! =null) {
if (dueTime < entry.next.dueTime) {
callback.next = entry.next;
break; } entry = entry.next; } entry.next = callback; }}// Draw the task node
private static final class CallbackRecord {
// Next drawing task
public CallbackRecord next;
// The drawing task should be executed at this point
public long dueTime;
// Describe the code snippet for the drawing task
publicObject action; .// Perform the drawing task
public void run(long frameTimeNanos) {
if (token == FRAME_CALLBACK_TOKEN) {
((FrameCallback)action).doFrame(frameTimeNanos);
} else{ ((Runnable)action).run(); }}}}Copy the codeChoreographer accepts four types of tasks, namely input, animation, View tree traversal, and COMMIT. Each task is abstracted into a CallbackRecord, and similar tasks form a task chain CallbackQueue in chronological order. Four task chains are stored in the mCallbackQueues[] array structure.
- Subscribe to the next vSYNC signal
public final class Choreographer {
private void scheduleFrameLocked(long now) {
// Do nothing if you have subscribed to the next vSYNC signal
if(! mFrameScheduled) {// When the next vSYNC signal arrives, the drawing task needs to be performed
mFrameScheduled = true;
if (USE_VSYNC) {
ScheduleVsyncLocked () is eventually called to register for receiving vSYNC signals regardless of which branch you go
if (isRunningOnLooperThreadLocked()) {
scheduleVsyncLocked();
} else {
Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_VSYNC);
msg.setAsynchronous(true); mHandler.sendMessageAtFrontOfQueue(msg); }}else{... }}}// Delegate DisplayEventReceiver to register vSYNC signals
private void scheduleVsyncLocked(a) {
mDisplayEventReceiver.scheduleVsync();
}
// Draw a frame
void doFrame(long frameTimeNanos, int frame) {
synchronized (mLock) {
// If you do not need to respond to the vSYNC signal, it returns directly, without drawing anything
if(! mFrameScheduled) {return; }... }... }private final class FrameDisplayEventReceiver extends DisplayEventReceiver implements Runnable {
// VSYNC signal callback
@Override
public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {...// Throw a task to the main thread, draw a frame
Message msg = Message.obtain(mHandler, this);
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
}
@Override
public void run(a) {
mHavePendingVsync = false;
// Draw current framedoFrame(mTimestampNanos, mFrame); }}}// VSYNC receiver
public abstract class DisplayEventReceiver {
// Register to receive the next vSYNC signal
public void scheduleVsync(a) {
if (mReceiverPtr == 0) {
Log.w(TAG, "...");
} else {
Subscribe to SurfaceFlinger for the next vSYNC signalnativeScheduleVsync(mReceiverPtr); }}... }Copy the codeNot every VSYNC signal will be subscribed to and processed by the upper layer, and only when Choreographer subscribed to the next VSYNC signal will SurfaceFlinger call back to the upper layer via onVsync().
Before the first vertical signal in the figure arrives, postCallback() is called to throw a draw task to Choreographer, subscribs to the next signal and sets mFrameScheduled to true, indicating that the next frame needs to be drawn. When the first signal is generated, onVsync() will be called back, doFrame() will be thrown to the main thread, and mFrameScheduled uled to false. Since there are no further subscriptions, no onVsync() call-back will be received, and nothing new will be drawn.
Perform a drawing task
Once ViewRootImpl throws a draw task to Choreographer, the task does not immediately execute, but is temporarily stored in the draw task chain and registered to receive the next vSYNC signal. It will only be executed after the next signal passes the onVsync() callback:
public final class Choreographer {
// VSYNC receiver
private final class FrameDisplayEventReceiver extends DisplayEventReceiver implements Runnable {
private boolean mHavePendingVsync;
private long mTimestampNanos;
private int mFrame;
@Override
public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {...// Send an asynchronous message to the main thread and execute the current Runnable, i.e. DoFrame ()
Message msg = Message.obtain(mHandler, this);
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
}
@Override
public void run(a) {
mHavePendingVsync = false;
// Draw a frame of contentdoFrame(mTimestampNanos, mFrame); }}}Copy the codeEvery time the vSYNC signal is called back, doFrame() is pushed to the main thread.
public final class Choreographer {
void doFrame(long frameTimeNanos, int frame) {
final long startNanos;
synchronized (mLock) {
// If there is no vSYNC signal for this frame, exit without drawing
if(! mFrameScheduled) {return; // no work to do}...try {
// Handle the input event for this frame
doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
// Animate this frame
doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
// Handle the View tree traversal for this frame
doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
// Execute the COMMIT task after all drawing tasks are complete
doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
} finally{... }... }}Copy the codeAs each frame is drawn, tasks are processed in the order of “Input event”, “animation”, “View tree traversal”, and “COMMIT”.
The handler doCallback() is defined as follows:
public final class Choreographer {
// Hold a chained array of draw tasks
private final CallbackQueue[] mCallbackQueues;
void doCallbacks(int callbackType, long frameTimeNanos) {
CallbackRecord callbacks;
synchronized (mLock) {
final long now = System.nanoTime();
// Fetches tasks from the specified type of task chain based on the current point in time
callbacks = mCallbackQueues[callbackType].extractDueCallbacksLocked(now / TimeUtils.NANOS_PER_MS);
if (callbacks == null) {
return; }... }try {
// Execute all tasks removed from the task chain
for(CallbackRecord c = callbacks; c ! =null; c = c.next) { c.run(frameTimeNanos); }}finally{... }}// Task entity class
private static final class CallbackRecord {
public CallbackRecord next;
publicObject action; .public void run(long frameTimeNanos) {
if (token == FRAME_CALLBACK_TOKEN) {
((FrameCallback)action).doFrame(frameTimeNanos);
} else {
// Execute the task((Runnable)action).run(); }}}}Copy the codeAt this point, the “View tree traversal “task pushed by ViewRootImpl was finally executed.
Which extractDueCallbacksLocked () is the method of task chain CallbackQueue, used to get to the current time point need to be performed all tasks:
private final class CallbackQueue {
// Task header
private CallbackRecord mHead;
// Get all tasks prior to the current time node that need to be executed in the current frame
public CallbackRecord extractDueCallbacksLocked(long now) {
CallbackRecord callbacks = mHead;
if (callbacks == null || callbacks.dueTime > now) {
return null;
}
CallbackRecord last = callbacks;
CallbackRecord next = last.next;
// Walk through the task chain, with the present time as the dividing line, from the past task chain
while(next ! =null) {
if (next.dueTime > now) {
last.next = null;
break;
}
last = next;
next = next.next;
}
mHead = next;
// Return all past tasks as a chain
returncallbacks; }}Copy the codeWhen the current frame is drawn, all previous tasks are taken from the task chain with the current moment as the dividing line and executed one by one in chronological order.
Run the above code on paper:
The subscription to the first signal is completed when the first task enters the chain, and the third task is executed after the first signal, so its subscription behaviordoScheduleCallback()It is stored in the main thread message queue until the first signal arrives and then subscribes to the second signal.
doFrame()Thrown to the main thread, it picks up tasks 1 and 2 from the task chain before the current time node and executes them. When the task is complete, the next message “subscribe to next vSYNC signal for Task 3” is fetched from the main thread message queue and executed. When all this is done, the main thread can only sit in a daze, waiting for the next vSYNC signal.
ChoreographerThe task.
Is the lag due to frame drop?
delay
In both cases, the drawing task can be completed in one frame interval. What happens if the task is time-consuming and takes longer than the interval?
After the first vertical signal arrives, tasks 1 and 2 are thrown to the main thread for execution, this time taking slightly longer than one frame interval, causing the function to subscribe to the next signal to be delayed. For the top, miss the secondonVsync()Callback, which means that tasks 1 and 2 missed a display opportunity, and task 3 missed a render opportunity. As for the bottom layer, when the monitor sends a vSYNC signal, it fetches the display from the graphics buffer. This time it doesn’t get the display, so it continues to display the previous frame.
In this case, all drawing tasks in the task chain are displayed deferred.
merge
What if there are other time-consuming tasks in the main thread besides drawing?
doFrame()Is thrown to the main thread message queue, but the main thread is occupied by an I/O taskdoFrame()Never got the chance to execute. And functions that subscribe to subsequent signals can’t be executed, so second and thirdonVsync()Will not be called until the I/O operation completes.
How will delayed doFrame() be different?
public final class Choreographer {
private Choreographer(Looper looper, int vsyncSource) {...// Frame interval = 1 second/refresh rate
mFrameIntervalNanos = (long) (1000000000 / getRefreshRate());
}
private final class FrameDisplayEventReceiver extends DisplayEventReceiver implements Runnable {
private long mTimestampNanos;
@Override
public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {...// Record the arrival time of the vertical signalmTimestampNanos = timestampNanos; . Message msg = Message.obtain(mHandler,this);
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
}
@Override
public void run(a) {
mHavePendingVsync = false;
// The vertical signal is thrown to the main thread along with doFrame() when it arrivesdoFrame(mTimestampNanos, mFrame); }}void doFrame(long frameTimeNanos, int frame) {
synchronized (mLock) {
...
// The time when the current frame is actually drawn
startNanos = System.nanoTime();
// Calculate the current frame drawing delay = current moment - the moment when it should have been drawn
final long jitterNanos = startNanos - frameTimeNanos;
// If the delay is greater than the frame interval
if (jitterNanos >= mFrameIntervalNanos) {
// Count the number of frames skipped and log warning if the number of frames exceeds the threshold
final long skippedFrames = jitterNanos / mFrameIntervalNanos;
if (skippedFrames >= SKIPPED_FRAME_WARNING_LIMIT) {
Log.i(TAG, "Skipped " + skippedFrames + " frames! "
+ "The application may be doing too much work on its main thread.");
}
// Calculates the offset of the frame delay with respect to the frame interval
final long lastFrameOffset = jitterNanos % mFrameIntervalNanos;
// Correct the frame drawing time to align it with the latest arrival vSYNC signalframeTimeNanos = startNanos - lastFrameOffset; }...// Update the last drawing time to the latest vSYNC signal time
mLastFrameTimeNanos = frameTimeNanos;
}
try {
// Render the current frame (the frameTimeNanos passed in is the latest vSYNC signal moment)
doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
} finally{... }... }}Copy the codeBefore drawing a frame, doFrame() calculates the frame delay, which is the difference between the current moment and the moment the frame should have been drawn.
If the frame delay is greater than the frame interval, the draw time frameTimeNanos is corrected to align it to a vsync signal moment prior to the current time. That moment is then passed to doCallbacks() as the basis for picking tasks off the task chain, and all tasks prior to that point are picked off.
At this point, tasks 1, 2, and 3 will all be removed because they are past tasks relative to the subsequent frameTimeNanos, so this doFrame() execution will be performed at the same time as the previous three doFrame() drawing tasks. If the drawing task is executed fast enough, You have the opportunity to subscribe to the next vSYNC signal before it arrives, so that the contents drawn by tasks 1, 2, and 3 are displayed in the next frame, as shown below:
Frame drops occur when the main thread is occupied by time-consuming operations, meaning that frames that should have been rendered and displayed are missed. But for the upper level, nothing is lost, except that the drawing tasks that should have been performed at different frame intervals are combined and displayed together.
What happens if three drawing tasks are time-consuming?
At this point, certain conditions in doCallbacks() fire:
public final class Choreographer {
void doFrame(long frameTimeNanos, int frame) {
synchronized (mLock) {
...
startNanos = System.nanoTime();
final long jitterNanos = startNanos - frameTimeNanos;
// If the delay is greater than the frame interval
if (jitterNanos >= mFrameIntervalNanos) {
...
// Calculates the offset of the frame delay with respect to the frame interval
final long lastFrameOffset = jitterNanos % mFrameIntervalNanos;
// Correct the frame drawing time to align it with the latest vSYNC signalframeTimeNanos = startNanos - lastFrameOffset; }...// Update the last drawing time to the latest vSYNC signal time
mLastFrameTimeNanos = frameTimeNanos;
// If the frame is old relative to the previous frame, skip the current frame and subscribe to the next vSYNC signal
if (frameTimeNanos < mLastFrameTimeNanos) {
scheduleVsyncLocked();
return; }}try {
// Render the current frame
doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
// After the input, animation, and View tree traversal tasks are completed, execute the COMMIT task
doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
} finally{... }... }void doCallbacks(int callbackType, long frameTimeNanos) {
CallbackRecord callbacks;
synchronized (mLock) {
...
// If it is a COMMIT task
if (callbackType == Choreographer.CALLBACK_COMMIT) {
// Computes the frame drawing delay
final long jitterNanos = now - frameTimeNanos;
// If the frame drawing delay is >= 2 times the frame interval
if (jitterNanos >= 2 * mFrameIntervalNanos) {
// Calculates the frame offset and appends a frame interval
final long lastFrameOffset = jitterNanos % mFrameIntervalNanos + mFrameIntervalNanos;
// Set mLastFrameTimeNanos to the second vSYNC signal moment before the current momentframeTimeNanos = now - lastFrameOffset; mLastFrameTimeNanos = frameTimeNanos; }}}... }}Copy the codeBecause the doFrame() execution took longer than 2 frames, when the input, animation, and View tree traversal tasks were completed and the COMMIT task was executed, the frame alignment was triggered and the mLastFrameTimeNanos was offset forward by some time. Align to the second vSYNC signal moment before the end of the drawing task. This is done to prevent more frames from being dropped due to this timeout drawing. Because doFrame() checks the drawing time of the current frame and the previous frame before drawing the current frame:
// If the frame is old relative to the previous frame, skip the current frame and subscribe to the next vSYNC signal
if (frameTimeNanos < mLastFrameTimeNanos) {
scheduleVsyncLocked();
return;
}
Copy the codeIf Choreographer throws a number of draw tasks towards the end of the last doFrame(), the frameTimeNanos of these draw tasks must be smaller than the frameTimeNanos of the last frame. If mLastFrameTimeNanos is set forward without a COMMIT task, the new draw task will miss one execution.