The background,
1. Multiple: Further expand and understand Perfdog features, with many advantages;
2. Good: Perfdog is used for APP performance analysis, which is easy to use;
3. Fast: quickly find performance bottlenecks for tuning, forming a test closed loop;
** The usage of PerfDog has been clarified previously: Tencent client performance testing tool PerfDog is used
How to test the performance of applets
Perfdog supports multi-process testing.
1. On The Android platform, generally large apps, such as games sometimes run cooperatively with multiple processes (wechat mini games, Wechat apps and games like King of Glory have multiple sub-processes), can select the target sub-process for targeted testing. The default is the main process. King of Glory as shown below:
Wechat mini-game, mini-program test, etc., as follows is wechat mini-game – Romantic Rose Garden:
Note: If the subroutine process name is highlighted, the current subroutine process is at the top level.
2. On iOS platform, APP multi-process is divided into APP Extension and system XPC Server. For example, a live esports software uses the APP Extension process (LABroadcastUpload). Of course, it may also use the system XPC Server service process, such as the general Web browser will use WebKit.
3. Case practice
1. Test summary
| I. Basic information of [XXX] small program performance test requirements | |
| 1. Test requirements | 1. Network type coverage in different scenarios; 2. Response time records in different scenarios; 3. Major performance parameters (FPS, memory, and CPU usage) cannot be recorded in the scenario. |
| 2. Test items | 1. Main performance data of APP (memory, CPU usage, FPS). 2. Response time. 3. Network type overwrite. |
| 3. Test environment | Mobile phone model and version: OnePlus 5 Android 9.0 Installation package: Small Program Experience version V1.0.0.0.QA6 |
| 4. Test location | Home (work from home) |
| 5. Test the network | Mobile (4G) |
| 6. Test method | See Performance Test Plan for details. |
| 7. Pass the standard | If the relevant bugs have been solved, it will pass |
| 8. Test time | 2020/3/19 |
| 9. Tester XX | X |
| 10. Demand communication personnel | XX |
2. Network standards
| scenario | network | Network speed | Preset conditions | Whether or not to perform | The expected results | |
| Scenario 1 | WiFi | Normal network | Excellent network (greater than 1MKBytes/S) | Station test (possible network interference) | no | / |
| Scenario 2 | Good network (greater than 500KBytes/S, smaller than 1000KBytes/S) | Station test (possible network interference) | no | / | ||
| Scenario 3 | Normal network (larger than 300KBytes/S, smaller than 500KBytes/S) | Station test (possible network interference) | no | Phase ii requirements | ||
| Scenario 4 | Weak network | Weak network (greater than 100KBytes/S, smaller than 300KBytes/S) | Station test (possible network interference) | no | Phase ii requirements | |
| Scenario 5 | Weak network (larger than 30KBytes/S, smaller than 100KBytes/S) | Station test (possible network interference) | no | / | ||
| Scenario 6 | Extremely weak network (less than 30KBytes/S) | Station test (possible network interference) | no | / | ||
| Scene 7 | Operators of 4 g | Normal network | The signal is strong | Carrier network | is | The first phase of demand |
| Scenario 8 | Weak network | Weak signal (light signal value <-90dbm) | no | / |
3. Performance test scheme
| Test description | Test whether the performance parameters meet the standards in different scenarios (see scenario description) and in different network environments (WiFi and 4G). If not, it needs to be tuned and validated. | ||||||||||||||||
| Test standard | 1. Small programs refer to the performance standards of the APP end: FPS>25, average CPU usage less than 60%, and memory <1000M. 2. Response time: good within 2S, barely acceptable between 2S and 5S, unacceptable above 8S. | ||||||||||||||||
| Test tools and calculation methods | The test was mainly based on the H5 page of wechat mini program: 1. Scheme 1: Use Tencent Perfdog to record the main scenes one by one (see the operation procedure in different scenarios), and save the data to the local disk and cloud disk. Average value: FPS, memory, and CPU usage. 2. Scheme 2: micro channel small program performance control panel, real-time print performance parameters and save them to the Trace tool. | ||||||||||||||||
| Main Scenario Description | Scenario Description | The network environment | The performance parameters | Whether through | BUG ID | note | |||||||||||
| FPS (Max) | FPS (Min) | FPS (AVG) | Memory (Max) | Memory (Min) | Memory (AVG) | CPU usage (Max) | CPU usage (Min) | CPU usage (AVG) | Response time (Max) | Response time (Min) | Response time (AVG) | ||||||
| The main scenario 1 | Scenario 1 (more details, see column C in [Performance Test Results]) | WiFi (normal)/CARRIER 4G (normal) /WiFi (weak network) | |||||||||||||||
| . | |||||||||||||||||
| N points scene | |||||||||||||||||
| . | Scenario 1 | WiFi (normal)/CARRIER 4G (normal) /WiFi (weak network) | |||||||||||||||
| . | |||||||||||||||||
| . | |||||||||||||||||
| The main scene n | Scenario 1 | WiFi (normal)/CARRIER 4G (normal) /WiFi (weak network) | |||||||||||||||
| . | |||||||||||||||||
| N points scene |
Iv. Performance standards
Note: PerfDog will break down each type of APP and game reference criteria later. Please refer to the 2019 China Mobile Game Quality White Paper: wetest.qq.com/white-paper…
Five, performance report view
1. Report viewing is similar to APP: Tencent client performance testing tool PerfDog is used
2. Record and analyze performance data
| The serial number | priority | The main scene | Points in the scene | The network environment | Results the URL | Whether through | BUG ID | note | ||||||||||
| Response time (AVG) | FPS (Max) | FPS (Min) | FPS (AVG) | Memory (Max) | Memory (Min) | Memory (AVG) | CPU Usage (Max, %) | CPU usage (Min, %) | CPU usage (AVG, %) | |||||||||
| 1 | In the | The entrance | 1. After the APP entry is entered and authorized, jump to XX mini-program. | Operators of 4 g | / | 8.81 | / | / | / | / | / | / | / | / | / | no | 483XXX | It takes a long time for APP to jump small program, and the current check jump is caused by wechat SDK |
| 2 | In the | Home page Display rules | 2. The home page is displayed | Operators of 4 g | 4.85 | |||||||||||||
| 3 | In the | Leave the home page for 1 minute | Leave the home page for 1 minute | Operators of 4 g | Perfdog.qq.com/case_detail… Password: cvPsrZ | / | 1.00 | 10.00 | 3.30 | 340.00 | 320.00 | 327.80 | 11.00 | 4.00 | 7.20 | is | / |
3. Test conclusions
| Test summary | |
| 1. Test overview | 1. Slow jump to applets after wechat authorization (about 8.81s), slow jump to home page after applets authorization (4.85s) : Development check is caused by wechat SDK, and there is no optimized control currently; 2. Slow preview of multimedia resources (depending on the type of resources and the size of the resource is more than 5S) : development check, it uses the built-in components of wechat for parsing, but the slow preview is caused by the network and internal content server, which is expected to be optimized by the internal content server first; 3. Slow compression and uploading of videos (30S of video shooting, 4.47s of compression, 12.21s of upload) : The development, screening and compression use the built-in components of wechat, so there is no room for optimization. Upload using internal content server components, is expected to require internal content server optimization; 4. High memory and CPU usage (maximum memory 968M, maximum CPU usage 88%) : The development check is mainly on the list page, due to a large amount of data, and exit from the list comes with GC. There are no exceptions and no optimization space for now; |
| 2. Known risks/problems | There is no comparison of competing products, which is not reflected in the methodology platform. |
| 3. Test conclusions | The test fails, resulting in slow loading and high performance consumption. For details, see [Test Overview]. |
| 4. Is it recommended | Ask the product owner to evaluate the product quality according to the available data, and then decide whether to use it. |
6. Case analysis
Case 1:
1) BUG description: [Performance] Web disk: when the image is opened for the first time, the preview image on iOS is 16S+ slower than that on Android, so optimization is recommended.
Solution: Picture browsing using wechat picture browsing controls, no room for optimization. Development check some iOS devices load quickly, which should be caused by the slow network or internal content server.
Case 2:
1) BUG description: [Performance] In the editing task of the web disk list sliding memory continued to increase: average memory 729M, maximum memory 1026. The average memory of the home page is 402M, and the maximum memory is 447.
2) Solution: The reason why the memory usage of web disk is higher than that of home page is that each item of web disk has a thumbnail image, and each NDR file meta information contains a lot of content. Therefore, the memory usage will be higher than the web disk. JS has its own garbage collection mechanism, so there is no need to access memory management.
Seven, summary
1. Features of tools
2. Performance test
3. Performance tuning