“This is the 27th day of my participation in the First Challenge 2022. For details: First Challenge 2022”

preface

In Android testing, not only function testing, boundary value testing, compatibility testing, performance testing is also indispensable, as shown in the following figure for Android APP testing types.

There are two main types of Android performance testing:

  • ROM system performance test
  • APP application performance test

In the last performance test, we studied CPU, memory, FPS and GPU, four commonly concerned indicators.

In this issue, we will focus on learning from response time, power, flow and other indicators, Let’s go~~

1. Response time

Response time is a measure of how long it takes for an operating APP to achieve the desired effect. If a software fails to load data, the daily life and retention of the software will be affected.

Response time indicator test points focus on three categories:

  • Cold start: The time it takes for the application to start for the first time
  • Hot start: The time it takes for an application not to start for the first time
  • Jump: The time taken to switch between application interfaces

The starting speed test points are as follows:

  • Cold start speed
  • Hot start speed
  • Full start speed
  • Have network start speed
  • No network startup speed

The startup time acceptance criteria are as follows: the cold startup duration is less than 1.5 seconds, and the hot startup duration is less than 1 second

Method for obtaining the startup time

  • Method 1: ADB method
    1. Cold start requires you to confirm in advance whether the application is running in the background or kill the application first
    2. Start application:adb shell am start -W -n package/activity
    3. Stop APP command:adb shell am force-stop package
  • Method 2: Use charls to capture packets and view duration

Response time test solution

scenario methods
Get the time consumed between a single machine event triggering and container startup of Native APP Buried point
Getting NativeAPP to start up completely takes time Check the system. Log
Get the delay time of Native calling PRC request methods Buried point
Gets specific data during the sending of RPC requests Buried point
Get RPC request to return specific data Buried point
The time taken to get the data returned from local parsing Buried point or Traceview tool
Gets the rendering time of the interface Slow camera or bury point

2. Traffic

Nowadays, network types mainly include 2G, 3G, 4G, 5G and WiFi. Different flow control strategies should be adopted for apps using different networks. For example, common video apps mainly use WiFi and cellular networks.

Concepts involved in traffic metrics:

  • Medium load: Normal operation
  • High load: application limit operation

Procedure for obtaining traffic data:

    1. Get the PID of APP:
    • Method one: adb shell ps | grep com. Google. Android. Youtube. XX
    • Method 2: dumpsys package com. Google. Android. Youtube. XX
    1. Get traffic data command:
    • Adb shell cat /proc/Net/dev process id

    Note: Some devices obtain different traffic data cat /proc/net/dev

    1. Tcpdump packet capture analysis

    • Download the tcpdump tool and push it to the device /data/local/ TMP directory

    • On the cli, enter tcpdump -v -i any -s 0 -c 2000 -w/USB/pacakages. pcap

    • Parameters of the tcpdump command are as follows:

      • -v/ -vV / -VVV:, the level of detail of the output data, for the traffic test.
      • -i: indicates the specified interface. Any indicates that no nics are allowed
      • -c: specifies the number of packets to be received. After receiving packets, packet capture stops automatically
      • -s: indicates the maximum number of bytes to be intercepted in each packet. If this parameter is set to 0, the default value is 262144

    • Use Wirshark to open traffic statistics and filter for ip.host=

Attention should be paid to flow test points:

  1. Application startup traffic value
  2. Traffic of the application background for 2H
  3. Application of peak flow during heavy load operation
  4. Apply the average flow in a medium load operation

3. The power

Android devices run a large number of applications, and battery consumption is also a concern for mobile devices.

Pay attention to the following scenarios:

  1. There is no significant difference in standby power consumption before and after APK installation
  2. The average power consumption of common operations, such as standby, operation page and APK startup, is normal
  3. No abnormal power consumption after using APK for a long time

Method of obtaining electric quantity data

  • Method 1: ADB command

    • adb shell dumpsys battery

    Status: 1 indicates the non-charging state, and 2 indicates the charging state

    Level: indicates the electric quantity information

    • The whole equipment power information: adb shell dumpsys batterystats | more
    • Adb shell Dumpsys BatteryStats APK package name

  • Method 2: Use the third-party tool ITest \GT

The quantity of electricity needs to be tested from both software and hardware

  • Hardware: The hardware test engineer needs to use a multimeter and power consumption meter to test the device. Meet market industry standards

  • Software side:

    • Method 1: Use a third-party tool for evaluation
    • Method 2: Obtain battery data on the command end

Temperature of 4.

An abnormal device temperature during the running of an Android device adversely affects user experience and may cause security risks.

Note the following scenarios for temperature indicators:

  1. The peak temperature of the equipment is normal when the equipment is fully loaded
  2. APK plays for a long time, and the average temperature of the device is normal
  3. Normal temperature of the equipment during normal operation, such as clicking and starting APK

Temperature index data and electric quantity index are obtained in the same way

  • adb shell dumpsys battery

  • Temperature: temperature (int type), unit: 0.1 degree

5. Common performance test problems

During performance testing, the following problems are often encountered and their causes:

The problem why
APP connection timeout Network interruption; The APP request interface is abnormal. Procedure
APP flash back Android cache garbage too much; Running programs, resulting in insufficient memory; Version compatibility problems;
Caton, black and white screen The SYSTEM CPU and GPU resources are insufficient. Excessive drawing
collapse APP often appears as a Crash
Poor interaction performance Other APK and frame interference
A memory leak The object created by APK was not released, causing the memory to be occupied
Out of memory The APK memory is insufficient

conclusion

In this issue, the test points and methods of obtaining data related to performance test response time, power consumption and flow are continued from the previous article.

That’s the content of this episode, please give your thumbs up and comments. See you next time