Author: Gao Laixin

Team: Tencent Mobile Quality Center TMQ

background

Canvas/ WebGL tests the rendering capabilities of the browser kernel. Some commonly used test pages in the industry are usually selected as test cases, for example:

Figure 1 Screenshot of webGL test page

Canvas/ WebGL test has always had the problem of unstable test data since it was included in TBS (Tencent Browsing service) pre-launch performance test. In the early stage, the methods such as purchasing small fans, increasing cooling time and writing retest pages were adopted to solve the problem, but the effect was not ideal. From time to time, the test results still found that performance was behind, but the development follow-up analysis revealed that the test data fluctuations caused false positives. This situation affects the work efficiency of both testing and development students.

Problem orientation

The pre-launch performance test of TBS 3.3 once again found webGL lagging behind. The WebglSpacerocks use case is 2.59 frames behind the previous version (TBS3.2). However, after the development of the location of the problem, it is still found that the data fluctuation caused by false positives.

FIG. 2webgl performance test results

This time, the developers and testers decided to get to the bottom of what was causing the fluctuations in the data.

Using perfMon’s performance monitoring tool, we found that CPU downturns and CPU shutdowns due to heat were the main factors affecting our test results.

Figure 3 Developing the problem location email content

The solution

To this end, the test team specially purchased a laptop cooling fan with superior performance to cool the test phone.

Figure 4 Laptop cooling fan

Preliminary verification found good results (the following text is an excerpt of the email) :

“This week’s blink3.0 Canvas/WebGL performance daily monitor results:

The data are basically stable. After adding a radiator and a small fan to assist heat dissipation, the test results of the three phones are close to the results of March and April and have been more or less improved, without retesting items.”

The optimization effect

Daily monitoring history records can show the effects before and after optimization:

1. The fluctuation range of the six use cases is reduced to varying degrees. The fluctuation of GUIMark3 GM3_JS_Compute is reduced from 9 frames to 0.2 frames.

FIG. 5 Comparison of data fluctuation range before and after optimization

2. Use cases that used to be retested are no longer retested. Currently, the retest probability is 0.

FIG. 6 Retest records before and after optimization

Note: retest is required when the monitoring data differs from the last one by 2 frames or more.

Regulate the “effective backwardness” threshold value

Note: only when the threshold value is reached is it considered to be effective and backward, develop follow-up analysis.

Of the six use cases, GUIMark3 GM3_JS_Compute fluctuates the most. So we run 4 tests on GUIMark3 GM3_JS_Compute (10 groups of test values each time) to verify the data fluctuation range:

Fig.7 Data fluctuation after optimization

FIG. 8 Data fluctuation after optimization

Conclusion:

1. The maximum and minimum deviation of 10 groups of data in a single test was 1 frame;

2. The average value of 10 groups of data in multiple tests fluctuates around 0.5 frames;

3. The maximum and minimum deviation of all data in multiple tests was 1.95 frames.

Based on the above test results, the “effective lag” thresholds for different use cases are as follows:

FIG. 9 Set the threshold value of “effective lag” according to the verification result of data fluctuation

Search wechat official number: Tencent Mobile Quality Center TMQ, get more test dry goods!