Please follow the Wechat official account BaronTalk to get more good articles!
A preface.
Performance problems are one of the main culprits that lead to App user loss. If users encounter problems such as page lag, slow response speed, severe heat, large consumption of traffic and power when using our App, they are likely to uninstall our App. Users are often expensive to acquire, so losing users because of performance issues is something we want to avoid, and it’s our developers’ fault for not doing it right.
Last year, our team completed the reconstruction of the entire project architecture (interested students can refer to my previous article “Anjuke Android Project Architecture Evolution and Android Modularization Exploration and Practice”), which can well support our business at present. In addition, the development efficiency of the team has also been improved to a certain extent, and the quality of the project has also made great progress.
But once the project goes live, are there any performance issues? What’s the user experience like? What are the performance issues that you might encounter in your user’s usage scenarios? Where are the performance weaknesses of our project? We don’t know the answers to any of these questions, which makes it imperative to develop a comprehensive performance monitoring system. This year, our team started to develop our own performance monitoring component APM, hoping to collect online performance data through it, find performance weaknesses, and optimize user experience specifically.
APM Application Performance Management & Monitoring
I will introduce APM principles, framework design and implementation in a series of articles. This is the first article in this series, which focuses on the implementation principles of APM. According to the current plan, the series will be developed in the following aspects:
- Principles: mainly introduces the implementation principle of APM;
- Design: introduces the whole APM framework design;
- Gradle Plugin: Introduces Gradle Plugin in APM project, and how to develop a Gradle Plugin;
- Implementation -Javassist/ASM: an introduction to bytecode manipulation libraries such as Javassist, ASM, and how to use them to insert code at compile time to collect various performance data;
- Implementation – Data storage and reporting: introduces the storage and reporting mechanism and implementation process of APM framework;
- Release integration: Finally, I show you how to publish the library to jCenter() and how to integrate it in a production project.
Here I want to explain to you that the previous articles are often too long in order to try to explain complex issues in an easy to understand way, but also to cover all aspects; Articles such as RxJava Series 6(read RxJava source code from the micro perspective), Dagger2, Anjuke Android project architecture evolution, Android modularity exploration and practice, Binder principle analysis for Android application engineers, etc. The length is usually more than 8,000 to 10,000 words, and it may take nearly half an hour to read the whole text, which is not in line with the current needs of fragmented reading. So keep the rest of your writing to a minimum of 10 minutes.
This is why I want to cover APM as a series, and also to ensure that later introductions to APM will dive into implementation details and avoid generalities.
Basic principles of Android APM
There are many commercial APM platforms in the market, such as the famous NewRelic, as well as domestic listening cloud, OneAPM and so on. The workflow for these platforms is basically the same:
- Firstly, data is collected on the client (Android, iOS, Web, etc.);
- Then collect the data and report it to the server;
- After receiving the data, the server models, stores, mines and analyzes the data, and then visualizes the data for users to use.
The diagram below:
The Android APM framework we introduced is actually a data collection and reporting SDK applied on the Android platform. It mainly consists of three modules:
- The data collection
- Data is stored
- The data reported
Data acquisition is the core of the APM framework.
We can collect data by manually burying points, but this way is heavy workload, inflexible, and can not cover all scenes; So data can only be collected in an automated way. Bytecode piling by modifying bytecode during application build is one way to automate this.
Iii. Android packaging process and bytecode piling principle
Before we talk about bytecode piling, let’s first look at Android’s packaging process, as shown below:
From the above packaging flowchart, we can see that all class files of an App, including class files of the third party, will be packaged into one or more dex files through the dex process.
There are two key steps involved:
- Javac: compile source code files in.java format into class files;
- Dex: Aggregates the files in class format to form one or more dex files.
If we want to modify the bytecode, we just need to iterate through all the bytecode files after Javac and before dex, and filter the changes according to certain rules. Here is the entry point of the bytecode stub.
So how exactly do we intervene in the packaging process to modify the bytecode when a class is converted to a dex file?
The answer is the Transform API
With Android Gradle Plugin 1.5.0 and above, Google officially provides the Transform API as the entry point for bytecode Plugin. All we need to do is implement a custom Gradle Plugin and then modify the bytecode file during compilation. The implementation of Gradle Plugin will be explained in detail later in the article.
4. Modify bytecode
Once the pile entry is found, the next step is to modify the bytecode. The more common frameworks for modifying bytecode are Javassist and ASM.
-
Javassist is an open source library for analyzing, editing, and creating Java bytecode. It provides source-level apis as well as bytecode level apis, source-level apis that use Java coding directly without having to go deep into virtual machine instructions. You can dynamically change the structure of the class or dynamically generate the class.
-
ASM is a Java bytecode manipulation framework. It can be used to dynamically generate classes or enhance the functionality of existing classes. ASM can generate binary class files directly or dynamically change class behavior before the class is loaded into the Java Virtual Machine.
Compared to Javassit, THE ASM API is closer to the bottom and more difficult to use, requiring some knowledge of Java bytecode and virtual machines. The advantages of ASM are performance advantages and more flexibility; Javassist’s implementation makes heavy use of reflection, so performance is low.
In short, ASM is difficult to use, but powerful and efficient. It is the first choice for many non-trace burial points and APM framework.
The details of ASM will be covered later in this series.
In live.
The principle of Android APM is very simple and can be summed up in one sentence:
According to the packaging principle, in the process of class conversion to dex, call Gradle Transform API to walk through the class file, use Javassist, ASM and other frameworks to modify the bytecode, insert our own code to achieve performance data statistics.
All of this is done at compile time.
In fact, the non-trace burying point on Android is also based on the same principle, but the difference is that the hook points are different and the data collected are different. Therefore, the non-trace burying point system can also be realized after mastering the implementation principle of APM.
The principle is simple; the hard part is the implementation details. Such as how to pile the page frame rate, flow, power consumption, and so on. We will cover these details later. As for why they put it in the back… Because I don’t know a lot of things I haven’t done before… 🤣
If you like my posts, follow my BaronTalk, Zhihu.com column, or add a Star on GitHub.
- Wechat official account: BaronTalk
- Zhihu column: zhuanlan.zhihu.com/baron
- GitHub：github.com/BaronZ88
- Personal blog: baronzhang.com
