Tencent open source VasDolly, talk about the principle of Android signature and multi-channel packaging!

One, foreword

Hi, everyone, I am Chengxiang Ink shadow!

When we need to release an App to the application market, we generally need to produce different channel packages for different markets. They use the same set of codes but contain some channel information of their own for data analysis.

A few days ago, The Penguin esports team opened source their own Android Apk multi-channel packaging tool: VasDolly, which is more comprehensive than Meituan’s Walle.

Take this opportunity to explain the differences between different versions of Android’s signature mechanism.

Android signature

2.1 Application Signature

By signing the Apk, developers can prove ownership and control of the Apk, which can be used to install and update their applications. An Apk installed on an Android device will be rejected if the Apk is not signed.

When Apk is installed, the package manager also verifies that the Apk has been signed correctly and that it has not been tampered with through the signing certificate and data digest. Install the device only when the device is secure and does not tamper with the device.

In brief, APK signature has two main functions:

Prove the owner of APK.
Allows the Android market and devices to verify the correctness of APK.

In Android, two application signature schemes are supported:

JAR based signature scheme (V1 scheme).
APK signature scheme v2 (v2 scheme) introduced in Android Nougat (7.0)

Since the signature scheme is being upgraded, v1 scheme must have some defects. Next, let’s take a look at the details of these two schemes.

2.2 V1 Signature Scheme

V1 signature scheme does not protect all content in Apk. If some exceptions are modified, signatures will not be invalid. For example, ZIP metadata.

In this way, when verifying APK signatures, a large number of untrusted (unverified) data structures need to be processed, and the data that is not protected by signatures need to be filtered and discarded before signature verification. That is to say, you can add some content to a signed file that is not protected by the signature, which will increase the risk of attack.

In addition, v1 scheme calculates the data summary separately for the protected original files (uncompressed) inside APK. Therefore, during verification, decompression is also required for each file and then signature verification to verify whether it is tampered. Therefore, when verifying APK signature, all compressed file entries of APK must be decompressed for data summary verification, and these will need to spend more time and memory.

It was because of the flaws in v1 that Android 7.0 introduced APK signature scheme V2.

2.3 V2 Signature Scheme

APK signature scheme V2 is a full file signature scheme, which can protect the signature of all protected parts of APK, so that they can be found to be tampered.

During APK validation, the V2 scheme treats an APK file as a Blob and checks the entire file for signatures. Any changes made to APK, including ZIP metadata, invalidate the APK signature.

When APK signature scheme V2 is used for signature, an APK signature block is inserted into the APK file. The block is located before and adjacent to the ZIP central Directory section. In APK signature Block, v2 signatures and signer identity information are stored in APK signature scheme block V2.

The chunk contains multiple “ID-value” pairs, and is encapsulated in a way that makes it easier to find the chunk in APK. The APK v2 signature is stored as an “ID-value” pair with the ID 0x7109871A.

The figure above is a comparison of APK file structure before and after signature. You can see that there are four parts in the signed APK for V2.

Contents of the ZIP entry.
APK Signing Block.
ZIP central directory.
ZIP Central directory end.

The APK signature scheme v2 is responsible for protecting the integrity of parts 1, 3, and 4, as well as the signed data blocks of the APK signature scheme v2 block contained in Part 2.

The integrity of Parts 1, 3, and 4 is protected by one or more summaries of their contents stored in signed data chunks that are protected by one or more signatures.

The summary information for parts 1, 3, and 4 is computed in a manner similar to a two-level Merkle tree, so it is much more efficient. This is a little bit more complicated, and you’re interested in exploring it a little bit more.

Now that we have the V2 scheme, can we completely discard the V1 scheme and use only the V2 scheme?

Since APK signature v2 was introduced in Android 7.0, APK can be installed in Android 6.0 and later versions. APK should be signed using the JAR signature (v1 signature) and then signed using the V2 scheme.

In the actual test, the situation of using only v2 without v1 can be compiled and installed and run on devices above Android 7.0, but under 7.0, because v2 signatures are not detected and V1 signatures do not exist in APK, during installation, An error with no signature is displayed.

To ensure compatibility issues, v1 signature schemes are also compatible on devices 7.0 and older. Therefore, v2 signature scheme is not a mandatory scheme for the time being.

The priority is: the v2 scheme is verified first. If no verification mechanism exists, the V1 scheme is verified.

2.4 Comparison of the two schemes

At this point, I think I have some understanding of v1 and V2 signature schemes. Since the V2 signature is an upgrade version, it is intended to solve some problems of the V1 signature scheme.

1. Efficiency

In v1, the/meta-INF/manifest.mf file will record the data digest of all files requiring validation signatures, and the data digest is produced from the source files before compression, which will be compressed during the packaging process.

Therefore, in the process of verifying the signature, the original file needs to be decompressed before the data summary can be calculated to verify it, which consumes more time and memory.

V2 signature scheme, is the APK file itself for data summary calculation, that is to say, it only calculates a file (V1 will calculate many files in APK), so there is no operation of decompression APK, efficiency and memory are optimized.

2. Security issues

V1 scheme, which only verifies some files inside APK, does not verify the integrity of APK. Therefore, we can still modify the APK file after signing. For example, meituan’s multi-channel solution is to add an empty file in the/meta-INF/directory and use the filename to identify the channel.

V2 signatures are verified for the entire APK. Therefore, any changes to the APK cannot pass the verification of v2 signatures. In this way, security is higher.

As for the time spent on signature verification, the main influence is the time spent on installation. Here is a copy of laboratory data (Nexus 6P, Android 7.1.1) for reference.

Almost 2. X times difference, v2 signature has a lot of improvement on APK installation.

3. Common multi-channel schemes

The demand point of multiple channels lies in the same App, which needs some differences to distinguish them. The most common scenario is the Domestic market of Android. Different channel packages are used for different markets, and their codes are consistent, but the data of a channel number is different. The advantage of this is that the data is clearer, and we know that the users of that channel are good users.

3.1 Abandoned schemes

Since the signature scheme is being upgraded, the multi-channel scheme needs to be upgraded as well. There was a plan that worked, but it has largely been scrapped. For example, the Gradle Plugin configures different Flavors, unpacks the data using ApkTool, repackages it, and re-signs it.

Each of these options has some limitations. For example, Gradle Flavors, each channel needs to be repackaged, which is very time-consuming, and the CRC value of the generated channel package DEX is inconsistent, which is not good for hot update solutions. In ApkTool’s scheme, the process of unpacking → packaging → re-signing is also time-consuming. Secondly, it is unstable and may fail to unpack after upgrading Gradle Plugin version.

So we’re thinking, for an efficient multi-channel packaging solution, there are a few key items that need to be paid attention to.

Do not break the signature.
Cannot be repackaged.
You have to read information efficiently.

Without breaking the signature, it limits the ability to unpack and re-sign, which is bound to improve efficiency.

Besides VasDolly, the Walle scheme given by Meituan is widely spread in the market. Here, I will briefly explain their principles.

3.2 Common multi-channel schemes under V1 signature

In v1, the signature verification is weak, and Meituan provides a complete solution. Write an empty file to the/meta-INF directory in a packaged signature Apk, using the filename to identify the channel number.

First, using such a scheme does not break the V1 signature, so it is very efficient.

In Tencent’s VasDolly, it actually adds channel information to the EOCD part of Apk files.

APK file is essentially a ZIP package, and EOCD is the third part shown in the figure above, which is not verified by V1 signature and can be used to record our channel information.

3.3 Multi-channel scheme under V2 signature

Under the V2 scheme, there is no difference between Tencent VasDolly and Meituan’s Walle scheme, because the verification is stronger, and there are only so many areas for us to operate.

As mentioned earlier, an APK Signing Block is inserted into the APK file structure after v2 Signing.

In this APK file structure, only block 2, which is the area where APK V2 signature information is recorded, is not all involved in the verification of V2 signature, so everyone is working on this part.

The APK Signing Block contains multiple “id-value” key-value pairs, and the v2 Signing information is recorded in the ID 0x7109871A and does not validate values under other ids.

Thus, a multichannel scheme based on V2 signatures was born: an additional ID- value was added to the APK signature block to record channel information.

4. Commercial multi-channel packaging scheme

Before VasDolly was open source, the multi-channel packaging scheme supporting V2 signature on the market was Walle of Meituan. The following is a simple comparison of their advantages and disadvantages.

Which plan you want to use depends on your actual needs.

Meituan Walle Github address:

https://github.com/Meituan-Dianping/walle

Walle scan code direct

Tencent VasDolly Github

https://github.com/Tencent/VasDolly/

VasDolly scan code direct

Source of some pictures and reference materials:

https://source.android.com/security/apksigning/v2

https://github.com/Tencent/VasDolly/wiki/VasDolly%E5%AE%9E%E7%8E%B0%E5%8E%9F%E7%90%86

https://tech.meituan.com/mt-apk-packaging.html

Today, I reply to “grow up” in the background of the official account, and I will get some learning materials sorted out by me. I can also reply to “add group”, so that we can learn and progress together.