Author: QA-SherryShare Baidu APP technology 2019-12-30

preface

Fluency testing is a deep field in client performance testing technology, so Baidu App brings you a series of articles on the whole process quality monitoring practice of fluency.

It includes: (1) analysis of the current situation of fluency, (2) selection of fluency indicators, (3) online and offline monitoring practice of fluency, and (4) evaluation scheme of fluency competing products. I hope it will be helpful for you to learn and practice fluency performance testing.

background

Why care about fluency? APP is easy to crash, webpage news can not open and other pain points, users will complain or even uninstall. After continuous optimization, improve stability, reduce the rate of white screen, when these pain points are not so painful, we need to focus on the user’s “cool point”, one of the typical problems affecting the cool point is – stuck problem.

Test students by analyzing user feedback, caton under different business scenarios, the influence is found on the user experience, and some conditions user mood is bad, but it’s actually a case by case processing, often because offline cannot retrieval practice, and inadequate provide clues, so you need to separate monitoring index for recall.

On the other hand, with the continuous emergence of high-performance mobile phones, in order to improve user interaction experience, design is no longer a simple pop-box or loading interaction, and more “dazzling” interaction scenes such as dynamic effect are used. Will it be a burden on low-end phones while providing a premium experience? Will it give people not only not “dazzle”, but also the feeling that the card can not be used? Therefore, we desperately need a good set of fluency monitoring standards to balance advanced interaction with lag

Industry Fluency Monitoring Scheme Research (Android)

Introduction to basic fluency concepts

  • Ideal frame rate: 60FPS, subject to monitor refresh rate of 60HZ
  • Ideal frame length: 1/60≈16.6ms
  • Vsync mechanism: Vsync can be simply considered as a timed interrupt. The system sends Vsync Pulse signal every time it needs to draw, and the CPU/GPU immediately processes the drawing after receiving the signal.

Industry Proposal Research

Introduction to basic fluency concepts

  • Frame length based on VSYNC: frame length, SM, SF:

The Choreographer class accepts system VSync signals, performs the View Input, Animation, and Draw operations in sequence each time it receives a VSync signal, and then waits for the next signal to perform the three operations in sequence again. If the second signal arrives and the Draw operation is not completed on time, the screen will not update and will display the contents of the first frame. This means lost frames, lost frames is the cause of the picture stuck. So we can add our own Callback to the Choreographer class, and with the doFrame function of this Callback we can count the number of frames drawn per second (flow value SM), draw time (time between two Doframes (frame length), and frame loss SF.

  • Based on the stars:

Use the UI thread’s Looper to print a log match to get the frame length. This is similar to the VSYNC scheme, except that data loss may occur when the UI thread is heavily blocked. (The impact on UI threads is also a balancing point)

  • Stack monitoring:

Single-open threads periodically grab the stack and upload the nearest stack based on Vsync or Looper when the frame length exceeds a specified threshold. However, because of real-time stack capture by single open thread, the performance of the application itself will degrade, so it is not suitable for long-term large-scale online use.

  • Monitoring precautions (measured experience) :

In the actual test, it is found that when APP is static, especially when webpage is static, SM value may also become low, such as close to 30, and SF value and frame length may exceed the ideal value. The reason is that although the user does not operate the interface, there may also be downloading in the background, animation outside the screen display area and other behaviors. The overall interface display does not show the lag, but it may affect the loading that the user can not perceive with the naked eye.

— The absolute value of the same fluency indicator has different reaction accuracy to the actual user experience in different stages and scenarios. Therefore, it is recommended to monitor different scenes and stages.

The resources

  • Xiao Zhu: Android App Performance Evaluation analysis – Fluency section
  • Htkeepmoving: Mobile APP performance evaluation – Fluency evaluation
  • GT3.1 Simplify your App performance test (2) — principle explanation, tracing the source
  • Wechat reading: Caton monitoring system
  • Tencent PerfDog: PerfDog Performance Dog help documentation
  • Whbsspu: why does the page run smoothly at 60fps?
  • Egos: Introduction to VSync in Android
  • Markzhai ‘s home: BlockCanary – easily find the Android App interface caton culprit (AndroidPerformanceMonitor)