Android click response time

Android users expect applications to respond to their actions in a fraction of the time.

UX studies tell us that response times of less than 100 milliseconds feel instant, while responses of more than 1 second can cause users to lose focus. When the response time approaches 10 seconds, users simply abandon their task.

Measuring response times to user actions is critical to ensuring a good user experience. Clicking is the most common action that an application must respond to. Can we measure the Tap response time?

Tap Response Time is the Time between the user pressing the button and the application apparently responding to the click.

More precisely, it is the time between the finger leaving the touch screen and the display presenting a frame that has a visible response to that click, such as the start of a navigation animation. Tap Response Time does not include any animation Time.

Naive Tap response time

I opened the Navigation Advanced Sample project and added a call to measureTimeMillis() to measure the Tap Response Time when clicking the About button.

aboutButton.setOnClickListener {
  val tapResponseTimeMs = measureTimeMillis {
    findNavController().navigate(R.id.action_title_to_about)
  }
  PerfAnalytics.logTapResponseTime(tapResponseTimeMs)
}
Copy the code

This approach has several drawbacks:

It can return negative time.

It does not scale with the size of the code base.

No consideration is given to the time between the finger leaving the touch screen and the tap listener being invoked.

It does not take into account the time between our completion of the call to navController.navigate () and the display rendering of a frame visible to the new screen.

Negative time

MeasureTimeMillis () calls System.CurrentTimemillis () can be set by the user or the phone network, so time may jump back or forward unpredictably. Elapsed time measurements should not be used with System.currentTimemillis ()

Large code base

Adding measurement code for every meaningful click listener is a daunting task. We need a solution that scales with the size of the code base, which means we need central hooks to detect when meaningful actions are triggered.

Touch pipeline

When your fingers leave the touch screen, the following happens:

• The system_server process receives information from the touch screen and determines which window should receive the MotionEvent.UP touch event. Each window is associated with an input event socket pair: the first socket is owned by System_server to send input events. The first socket is paired with a second socket owned by the application that created the window to receive input events.

• The system_server process sends touch events to the input event socket of the target window.

• The application on the listening socket to receive touch events, and store it in a queue (ViewRootImpl. QueuedInputEvent), and arrange a Choreographer framework to use input events. (The system_server process detects when an input event has been in the queue for more than 5 seconds, at which point it knows it should display the application Non-response (ANR) dialog box.)

• When the Choreographer framework fires, the touch event is dispatched to the root view of the window and then dispatched through its view hierarchy.

• The clicked view receives the MotionEvent.UP touch event and issues a click listener callback. This allows other visual states of the view to be updated before a click operation begins.

• Finally, when the main thread runs the publish callback, the view click listener is invoked.

A lot happens from the time the finger leaves the touch screen to the time the click listener is invoked. Each MotionEvent includes the time at which the event occurred (motionevent.geteventtime ()). If we can access the motionEvent.up event that caused the click, we can measure the actual start Time of the Tap Response Time.

Traverse and render

findNavController().navigate(R.id.action_title_to_about)
Copy the code

• In most applications, the code above starts fragment transactions. The transaction may be immediate (commitNow()) or published (commit()).

• As the transaction executes, the view hierarchy is updated and the layout traversal is scheduled.

• When the layout traversal is performed, a new frame is drawn to a surface.

• It is then combined with frames from other Windows and sent to the display.

Ideally, we want to know exactly when changes to the view hierarchy are actually visible on the display. Unfortunately, as far as I know, there is no Java API, so we have to get creative.

From click to render

Main thread tracing

To clarify this, we enable Java method tracing when the button is clicked.

1. The motionEvent. ACTION_UP event is scheduled and a click is sent to the main thread.

2. The published click runs, and the listener calls navController.navigate () and publishes the fragment transaction to the main thread.

3. The fragment transaction runs, the view hierarchy is updated, and the view traversal is scheduled for the next frame on the main thread.

4. The view traversal runs, and the view hierarchy is measured, laid out, and drawn.

Systrace

In Step 4, the view traverses the draw channel to generate a list of draw commands (called a display list) and sends the list of draw commands to the render thread.

Step 5: The render thread optimizes the display list, adds ripples and other effects, and then uses the GPU to run the drawing command and draw to the buffer (OpenGL surface). When done, the render thread tells the surface thrower (in a separate process) to swap the buffer and place it on the display.

Step 6 (not visible in the Systrace screenshot) : The surfaces of all visible Windows are synthesized by Surface Flinger and Hardware Composer, and the results are sent to the display.

Click response time

We previously defined Tap Response Time as the Time between the user pressing the button and the app’s apparent Response to the click. In other words, we need to measure the total duration through steps 1 through 6.

Step 1: Dispatch up

We defined TapTracker, a touch event blocker. TapTracker stores the time of the last MotionEvent.ACTION_UP touch event. When the published click listener fires, we retrieve the time of its up event by calling Taptracker.currentTap:

object TapTracker : TouchEventInterceptor {

  var currentTap: TapResponseTime.Builder? = null
    private set

  private val handler = Handler(Looper.getMainLooper())

  override fun intercept(
    motionEvent: MotionEvent,
    dispatch: (MotionEvent) -> DispatchState
  ): DispatchState {
    val isActionUp = motionEvent.action == MotionEvent.ACTION_UP
    if (isActionUp) {
      val tapUptimeMillis = motionEvent.eventTime
      // Set currentTap right before the click listener fires
      handler.post {
        TapTracker.currentTap = TapResponseTime.Builder(
          tapUptimeMillis = tapUptimeMillis
        )
      }
    }
    // Dispatching posts the click listener.
    val dispatchState = dispatch(motionEvent)

    if (isActionUp) {
      // Clear currentTap right after the click listener fires
      handler.post {
        currentTap = null
      }
    }
    return dispatchState
  }
}
Copy the code

We then add the TapTracker interceptor to each new window:

class ExampleApplication : Application() { override fun onCreate() { super.onCreate() Curtains.onRootViewsChangedListeners += OnRootViewAddedListener { view -> view.phoneWindow? .let { window -> if (view.windowAttachCount == 0) { window.touchEventInterceptors += TapTracker } } } } }Copy the code

Step 2: Click on the listener and navigation

Let’s define an ActionTracker that will be called when a published click listener is triggered:

object ActionTracker { fun reportTapAction(actionName: String) { val currentTap = TapTracker.currentTap if (currentTap ! = null) { // to be continued... }}}Copy the code

Here’s how we can use it:

aboutButton.setOnClickListener {
  findNavController().navigate(R.id.action_title_to_about)
  ActionTracker.reportTapAction("About")
}
Copy the code

However, we don’t want to add this code to every click listener. Instead, we can add the target listener to the NavController:

navController.addOnDestinationChangedListener { _, dest, _ ->
  ActionTracker.reportTapAction(dest.label.toString())
}
Copy the code

We can add a target listener for each TAB. Or we could add a target listener to each new NavHostFragment instance using a lifecycle callback:

class GlobalNavHostDestinationChangedListener
  : ActivityLifecycleCallbacks {

  override fun onActivityCreated(
    activity: Activity,
    savedInstanceState: Bundle?
  ) {
    if (activity is FragmentActivity) {
      registerFragmentCreation(activity)
    }
  }

  private fun registerFragmentCreation(activity: FragmentActivity) {
    val fm = activity.supportFragmentManager
    fm.registerFragmentLifecycleCallbacks(
      object : FragmentLifecycleCallbacks() {
        override fun onFragmentCreated(
          fm: FragmentManager,
          fragment: Fragment,
          savedInstanceState: Bundle?
        ) {
          if (fragment is NavHostFragment) {
            registerDestinationChange(fragment)
          }
        }
      }, true
    )
  }

  private fun registerDestinationChange(fragment: NavHostFragment) {
    val navController = fragment.navController
    navController.addOnDestinationChangedListener { _, dest, _ ->
      val actionName = dest.label.toString()
      ActionTracker.reportTapAction(actionName)
    }
  }
Copy the code

Step 3: Fragment execution

A call to navController.navigate () does not immediately update the view hierarchy. Instead, a fragment transaction is published to the main thread. When a fragment transaction executes, a view of the target fragment is created and attached. Since all pending fragment transactions are executed once, we added our own custom transaction to take advantage of the runOnCommit() callback. Let’s start to build a utility OnTxCommitFragmentViewUpdateRunner. RunOnViewsUpdated () :

class OnTxCommitFragmentViewUpdateRunner(
  private val fragment: Fragment
) {
  fun runOnViewsUpdated(block: (View) -> Unit) {
    val fm = fragment.parentFragmentManager
    val transaction = fm.beginTransaction()
    transaction.runOnCommit {
      block(fragment.view!!)
    }.commit()
  }
}
Copy the code

Then we pass an instance to ActionTracker. ReportTapAction () :

class GlobalNavHostDestinationChangedListener ... val navController = fragment.navController navController.addOnDestinationChangedListener { _, dest, _ -> val actionName = dest.label.toString() - ActionTracker.reportTapAction(actionName) + ActionTracker.reportTapAction(  + actionName, + OnTxCommitFragmentViewUpdateRunner(fragment) + ) } } }Copy the code

object ActionTracker { - fun reportTapAction(actionName: String) { + fun reportTapAction( + actionName: String, + viewUpdateRunner: OnTxCommitFragmentViewUpdateRunner + ) { val currentTap = TapTracker.currentTap if (currentTap ! = null) { - // to be continued... + viewUpdateRunner.runOnViewsUpdated { view -> + // to be continued... +}}}}Copy the code

Step 4: Frame and view level traversal

When fragment transaction execution, will arrange a view for the next frame traversal, we use the Choreographer. PostFrameCallback () to the link:

object ActionTracker { + + // Debounce multiple calls until the next frame + private var actionInFlight: Boolean = false + fun reportTapAction( actionName: String, viewUpdateRunner: OnTxCommitFragmentViewUpdateRunner ) { val currentTap = TapTracker.currentTap - if (currentTap ! = null) { + if (! actionInFlight & currentTap ! = null) { + actionInFlight = true viewUpdateRunner.runOnViewsUpdated { view -> - // to be continued... + val choreographer = Choreographer.getInstance() + choreographer.postFrameCallback { frameTimeNanos -> + actionInFlight  = false + // to be continued... +}}}}}Copy the code

Step 5: Render the thread

After the view is traversed, the main thread sends the display list to the renderer thread. The render thread performs additional work and then tells the surface Flinger to swap the buffer and place it on the display. We register a OnFrameMetricsAvailableListener to obtain the total frame duration (including the time spent on rendering thread) :

 object ActionTracker {
 ...
         val choreographer = Choreographer.getInstance()
         choreographer.postFrameCallback { frameTimeNanos ->
           actionInFlight = false
-          // to be continued...
+          val callback: (FrameMetrics) -> Unit = { frameMetrics ->
+            logTapResponseTime(currentTap, frameMetrics)
+          }
+          view.phoneWindow!!.addOnFrameMetricsAvailableListener(
+            CurrentFrameMetricsListener(frameTimeNanos, callback),
+            frameMetricsHandler
+          )
         }
       }
     }
   }
+
+  private fun logTapResponseTime(
+    currentTap: TapResponseTime.Builder,
+    fM: FrameMetrics
+  ) {
+    // to be continued...
+  }
Copy the code

Once we have the frame metric, we can determine when the frame buffer is swapped, hence the Tap response time, i.e. the time from motionEvent.action_UP to the buffer swap:

object ActionTracker {
 ...
     currentTap: TapResponseTime.Builder,
     fM: FrameMetrics
   ) {
-    // to be continued...
+    val tap = currentTap.tapUptimeMillis
+    val intendedVsync = fM.getMetric(INTENDED_VSYNC_TIMESTAMP)
+    // TOTAL_DURATION is the duration from the intended vsync
+    // time, not the actual vsync time.
+    val frameDuration = fM.getMetric(TOTAL_DURATION)
+    val bufferSwap = (intendedVsync + frameDuration) / 1_000_000
+    Log.d("TapResponseTime", "${bufferSwap-tap} ms")
   }
 }
Copy the code

SurfaceFlinger

There is no Java API to determine when a composite frame is finally sent to the display by SurfaceFlinger, so I didn’t include that section.