Although the package is preferential, the data is still very expensive, for users network traffic is money! Users are used to opening the APP traffic statistics function of the system (as shown below). From the perspective of APP, users do not want to see that their APP is a heavy traffic user at a glance, so it is necessary to spend time to know how the APP traffic is lost. However, the traffic statistics function of the system only roughly calculates the total amount of traffic consumed by each APP (time-sharing), but programmers need to conduct a more detailed and multidimensional analysis of APP traffic, so as to optimize APP data traffic in a targeted manner.
Android traffic measurement common methods
| methods | limited | |
|---|---|---|
| Traffic from Linux system files | Android is based on Linux, and the system provides/proc/net/dev,/proc/uid_stat/{uid}And other system files can obtain the traffic statistics of the entire system and specific APP (UID).Numerous articles of the same kind |
It can only obtain the total traffic of the APP in a certain period of time, but cannot carry out a more fine-grained decomposition of the traffic, which request consumes traffic |
| From the Android API | Android itself provides some apis to capture system traffic (from which traffic statistics are set) :TrafficStatsandNetworkStatsManager | In essence, traffic data files are read in the system. Some methods have no time parameters and cannot obtain the traffic in a specific period. In addition, users may need to enable the system permission /root to use the method |
| Agent caught | Network proxy tools are used to intercept each network request response package and analyze the amount of requested data. Common tools include:tcpdump,wiresharkGithub offers an open source package analysis projectpcap2har | Many packet capture software need to configure proxy on the device, do a lot of additional system configuration, very troublesome; The proxy does not support HTTPS, so you need to install forged certificates for each domain name. In addition, the traffic analysis of packet capture tools is not readily available and needs some secondary development |
Note: goose factory flow test scheme of GT performance test tool is summary of the three kinds of methods mentioned above, it can only get a period of time flow size and trend of the application under test, and the GT caught the Wireshark encapsulation, catch all data traffic is by mobile phone card can’t separate application of flow conditions. Therefore, GT tool of Goose factory only stays in the stage of obtaining application traffic and taking it down to see for mobile application traffic, and cannot provide effective traffic optimization guidance strategy.
Fine analysis based on pile insertion
The above mentioned traffic analysis methods, or simply rough traffic statistics, developers still can’t find the application code that consumes traffic; Or packet capture, cumbersome configuration error-prone, high cost of secondary development, high cost of automation. Ideally, developers would do their own logging in the source code, recording the start and end of all network requests and traffic consumption, and then analyzing the log, but manual logging is obviously deadly. Code pegs completely solve these problems from the bottom, modify the bytecode (Dex file) of APK according to rules, imitate manual, insert monitoring code at the place of HTTP request sent by APP every time, collect request data at run time, and analyze traffic. Please see detailed principle: testerhome.com/topics/9264
Appetizer is one of the few tools in the industry that can steadily apply piling technology. It monitors HTTP requests and performs fine-grained multi-dimensional analysis on correctness, performance, and flow, supporting almost all the mainstream HTTP libraries. In fact, most of the libraries are repackages of some basic HTTP libraries (especially the so-called rapid development frameworks that basically use OKHTTP for HTTP requests). So Appetizer intercepts underlying libraries such as HttpURLConnection, Apache HTTP Client, OKHTTP 2/3, Retrofit, Volley, etc. Unlike the packet capture tool, the underlying principle of the packet capture tool is network proxy. HTTPS is designed to prevent the proxy software from seeing the request content. Therefore, the packet capture tool needs to configure forged certificates and other troublesome things. The underlying principle of Appetizer is to collect data just like the data you can see in the source code. Without this problem, the content of HTTPS requests can be perfectly captured.
The method of using Appetizer for pile insertion analysis is very simple: upload APK, insert pile, download the inserted APK, and then run it on the device. After completion, Appetizer analysis can generate test reports. Concrete example, please stamp testerhome.com/topics/8162. Each analysis will generate an analysis report, please poke your eyes to open the report:
This is a screenshot of Appetizer test report. The four functional items on the left are: Statistics, Business process Modeling, detailed timeline, and traffic analysis. So please stamp the fourth
Click into the traffic analysis interface, the horizontal axis is the time, the vertical is the classification criteria, a point represents an HTTP request, a large point represents a large traffic, support left and right drag and pulley scaling, there are six classification criteria to assist traffic analysis
Click on a point to view the details of the network request
Look for traffic killers and optimize
Since it is to analyze traffic, it is natural that we need to know the domain name of the network request made by APP, which way to send and receive data, data type, etc. The refined traffic analysis provided by Appetizer analyzes APP network traffic from different dimensions and suggests optimization strategies from different perspectives.
By Type: Image manipulation? HTTP compression?
Appetizer classifies web requests into images, text, protocols, and other requests based on the type of data they receive. As shown in the above figure, generally speaking, protocol traffic and image traffic dominate APP network requests. Protocol traffic is large and intensive, while image traffic is small and large. Therefore, protocol requests and images are naturally preferentially optimized.
The protocol usually requests some service data, and the returned data type is usually JSON, XML and other text formats. The most effective method is gZIP compression of HTTP requests, with the compression rate of more than 70%. Amway a tool to check whether gzip is enabled, testerhome home for example, compression rate reached 80%.
Image resources are also the major “traffic users” of modern apps, because apps generally need to adapt to the HIGH-DEFINITION experience, and the image of the product is getting bigger and bigger. In fact, the picture format is also very exquisite, choose a more efficient picture processing framework, can automatically transcode the network picture automatically, adapt to the screen, automatic cache, save traffic and save memory to improve the efficiency of picture loading. Picasso, Glide, Fresco. In addition, many apps have a setting to use different resolution (size) images over WIFI and data, which is another design to consider.
Finally, text (web page) conforms to the wave of H5 and gradually becomes a part worth paying attention to in the flow army. Similar protocol, text data is compressed with GZIP, which is fast and high compression rate.
Classification by MIME type: Use modern formats and libraries
MIME Type is the Content-Type field of the HTTP response. It is a more fine-grained division of the request data types, including format types starting with text, image, and application. For image flow, the display cost of different types of images in Android is different, and the cost of using different display methods is also different. For example, PNG images occupy a large amount of memory. If there are too many PNG images, it will be easy to garbage collection or even memory overflow. JPG image memory is small, but the image decoding complex, time-consuming. The developer chooses the best image optimization method based on the type of image in the image above. In addition, it can be recommended that operations use more modern formats (fast decoding, small files, high quality), such as using WebP, using mozJPEG library to improve jpeg image compression rate by 20%, using native MPEG4 format instead of GIF, etc.
By root domain name/subdomain name: Third-party SDK traffic
In APP development, we need to use a lot of SDK services provided by the third party, such as map services, advertising and so on. Appetizer extracts the server domain name from the request URL to obtain the source of APP network request data. From the root domain name analysis result graph, the information of all servers (own domain name and domain name of three parties) that the APP sends network requests can be obtained first, and the server with the most requests can be targeted to monitor the response of servers in different time periods.
Classification by request method: Protocol design that focuses on traffic
The APP client sends HTTP requests. The most common request methods are GET (downstream traffic) and POST (upstream traffic). Request method result graph, feedback to developer APP what data request method is for each network request. From the distribution diagram, you can identify traffic problems caused by different request modes. For example, the common POST reloading problem: when we send the same POST request several times, the result is to create several resources, resulting in invalid consumption of traffic, such problems can be found in the request method classification diagram; A GET request, by definition, should specify a possible cache line for the returned content. Modern HTTP, such as OKHTTP, can recognize the cache requirement and automatically cache local files to reduce unnecessary repeated requests.
Classification by network library: Unified use of efficient HTTP libraries
Common HTTP request libraries in APP include HttpURLConnection, Apache HTTP Client, and OkHttp. Data traffic generated by different request libraries is also different. The compression and caching mechanisms of HttpURLConnection can effectively reduce network access traffic. OkHttp automatically adds Accept-Encoding: gzip to support automatic decompression and caching of HTTP responses on disk. According to the network libraries used by different network requests in the result figure, the optimal HTTP request library is selected when optimizing APP.