Confusing user manual for Android developers

Writing in the front

Hello, I’m the light source.

This article will be posted on my personal public account and will be posted on my personal blog. If you have any suggestions, please go to the blog comments, and if the blog itself has changes or errata, it will also be updated in the blog.

review

Obfuscations are undoubtedly one of the most important parts of the packaging process and should be turned on in all apps for no particular reason.

First of all, the obfuscation here actually includes the optimization process of code compression, code obfuscation and resource compression. With ProGuard, the obfuscating process removes unused classes, class members, methods, and attributes from the main project and dependent libraries, which helps avoid the bottleneck of 64K methods. At the same time, renaming classes, class members, and methods to meaningless short names increases the difficulty of reverse engineering. With Gradle’s Android plugin, we will remove unused resources and reduce the size of the APK installation package.

This article is composed of two parts. The first part gives best practices for obfuscation, and tries to make obfuscation accessible to even the novice. The second part covers obfuscation as a whole, the syntax and practice of custom obfuscation rules, rules for custom resource retention, and so on.

Android confuses best practices

1. Configurations are confused

Generally, the app Module’s build.gradle file has the following structure by default:

android {
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}Copy the code

It should not be enabled in debug mode because obfuscations take longer to compile. What we need to do is:

1. Change the value of minifyEnabled in release to true to enable obfuscation.

2. Add shrinkResources true to enable resource compression.

The contents of the modified file are as follows:

android {
    buildTypes {
        release {
            minifyEnabled true
            shrinkResources true
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}Copy the code

2. Customize obfuscation rules

A custom obfuscation rule file proguar-rules.pro is generated by default in the App Module. After a lot of research, a best practice for obfuscation rules applicable to most projects is as follows:

- optimizationpasses 5 # # specified compression level cannot skip class members of the non public library - the algorithm adopted in dontskipnonpubliclibraryclassmembers # confusion - optimizations. code/simplification/arithmetic,! field/*,! Class/done / * # confusion in the class, the name also confused - useuniqueclassmembernames # optimization there allowed to access and modify the modifier when members of the class and class - allowaccessmodification # will source file renamed "SourceFile string" - renamesourcefileattribute SourceFile # keep the line Numbers - keepattributes SourceFile, LineNumberTable # keep all implementation Serializable class members-KeepClassMembers * implements Java.io.Serializable {static final Long serialVersionUID; private static final java.io.ObjectStreamField[] serialPersistentFields; private void writeObject(java.io.ObjectOutputStream); private void readObject(java.io.ObjectInputStream); java.lang.Object writeReplace(); java.lang.Object readResolve(); } #Fragment does not need to be registered in androidmanifest.xml, Need additional protection - keep public class * extends android. Support. The v4. App. The fragments - keep public class * extends android. The app. # fragments ** -Dontwarn android.test.** -dontwarn android.test android.support.test.** -dontwarn org.junit.**Copy the code

The only things that are really generic and need to be added are these, but beyond that, each project needs to add a few obfuscation rules according to its own needs:

• Obfuscation rules required by third-party libraries. Formal third-party libraries generally write the required obfuscation rules in the access document, and pay attention to add when using.

• Code that changes dynamically at run time, such as reflection. A typical example is an entity class that converts to JSON. -keep public class **.* model *.** {*; }

• The class called in JNI.

• Method called by JavaScript in WebView

• View constructor, Android :onClick, etc.

3. Check for confusion

Obfuscated packages must be checked to avoid bugs introduced by obfuscation.

On the one hand, you need to look at it at the code level. Using the above configuration for confusion after packaged in < module name > / build/outputs/mapping/release/directory will output the following files:

• Dump. TXT describes the internal structure of all classes in the APK file

• Mapping. TXT provides a comparison table of classes, methods, class members, etc

• Seeds.txt lists the classes and members that were not confused

• Usage.txt lists the removed code

We can look at the seeds.txt file to see if the unconfused classes and members contain all the expected ones, and look at the Usage.txt file to see if any code has been removed by mistake.

On the other hand, you need to check from the testing side. The obfuscated package is fully tested for bugs.

4. Resolve the obfuscation stack

Confusing classes, method names, and so on are hard to read, which certainly makes reverse engineering harder, but it also discourages tracking online crashes. When we get the crash stack information, we find it difficult to locate the crash, so we need to reverse the confusion.

In the < SDK-root >/tools/proguard/ directory, there is an accompanying uninstallation tool (proguardgui.bat for Windows and proguardgui.sh for Mac or Linux).

Take the Window platform as an example. Double-click to run proGuardgui. bat, and you will see a one-line menu on the left. Click ReTrace, select the confusing package corresponding mapping file (after the confusion in the < module name > / build/outputs/mapping/release/path generates mapping. TXT file, It provides a comparison table of classes, methods, class members, etc.), paste the stack trace of crash into the input box, click ReTrace in the lower right corner, and the stack information is displayed.

The above use GUI program to operate, another way is to use the path of the retrace tool through the command line, the command is

retrace.bat|retrace.sh [-verbose] mapping.txt [<stacktrace_file>]Copy the code

Such as:

retrace.bat -verbose mapping.txt obfuscated_trace.txtCopy the code

Notes:

1) All classes involved in AndroidManifest.xml are already held automatically, so there is no need to add this obfuscating rule. (A lot of old obfuscation files are added, no longer necessary)

Proguard-android.txt already has some default obfuscating rules. There is no need to add them to proguard-rules.pro.

Two, confused introduction

“Obfuscation” in Android can be divided into two parts, one is Java code optimization and obfuscation, relying on ProGuard obfuscation to achieve; The other part is resource compression, which removes unused resources from the project and dependent libraries. (Resource compression is not strictly related to confusion, but we generally cover it together.)

1. Code compression

Code obfuscation is a process that includes code compression, optimization, obfuscation and a series of behaviors. As shown above, the obfuscation process has several functions:

1. Compression. Remove invalid classes, class members, methods, attributes, etc.

2. Optimization. Analyze and optimize method binaries; As described in Proguard-android-optimity.txt, optimizations can pose a number of potential risks and are not guaranteed to work correctly on all versions of Dalvik.

3. Confusion. Replace class names, attribute names, and method names with short, meaningless names;

4. Preverification. Add pre-verification information. This prevalidation is available on the Java platform. It is not required on The Android platform and can be removed to speed up obfuscations.

These four processes are enabled by default.

In the Android project we can choose to turn off optimization and preverify with the -dontoptimize and -dontpreverify commands (of course, the default proguard-android. TXT file already contains these two obfuscating commands. No additional developer configuration is required).

2. Resource compression

Resource compression removes unused resources from the project and dependent libraries. This is a great way to reduce the size of the APK package and is generally recommended. To do this, set the shrinkResources property to true in the build.grade file. Note that resource compression only takes effect if code compression is enabled with minifyEnabled True.

Resource compression consists of two processes: merge resources and Remove Resources.

In the Merge Resources process, resources with the same name are considered duplicate resources and are merged. Note that this process is not controlled by the shrinkResources property and cannot be disabled. Gradle must do this because if there are resources with the same name in different projects it will cause an error. Gradle looks for duplicate resources everywhere:

• SRC/main/res/path

• Different build types (Debug, release, and so on)

• Different build channels

• Third-party libraries that the project depends on

Resources are merged in the following priority order:

Dependency -> main -> channel -> build typeCopy the code

For example, if duplicate resources exist in the main folder and in different channels, Gradle will choose to keep the resources in the channel.

If duplicate resources are present at the same level, such as SRC /main/res/ and SRC /main/res/, gradle cannot complete the resource merge and will report a resource merge error.

The “Remove resource” process is defined by its name, and it should be noted that similar code confuses resource removal to define which resources need to be retained, as described below.

Custom obfuscation rules

There is a line of code like this in “Obfuscate configuration” above

proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'Copy the code

This line of code defines obfuscation rules in two parts: the contents of proguards-android.txt in the SDK’s Tools/Proguard/folder, and the contents of proguards-rules.pro, which is placed by default in the module root directory. The former is the default obfuscation file provided by the SDK (see Appendix 1), and the latter is where developers can customize obfuscation rules.

1. Common confusing commands:

• optimizationpasses

• dontoptimize

• dontusemixedcaseclassnames

• dontskipnonpubliclibraryclasses

• dontpreverify

• dontwarn

• verbose

• optimizations

• keep

• keepnames

• keepclassmembers

• keepclassmembernames

• keepclasseswithmembers

• keepclasseswithmembernames

The obfuscation commands used in the first part of the Android Obfuscation Best Practices section are not covered here. For details, see the official website. In particular, there are several commands related to rules that keep related elements out of obfuscation:

2. Rules to keep elements out of confusion

Like:

[hold command] [class] {[member]}Copy the code

“Class” represents a class-related qualification, and it will eventually locate some classes that meet that qualification. Its contents can be used:

• Concrete classes

• Access modifier (public, protected, private)

• Wildcard *, which matches characters of any length without the package name separator (.)

• Wildcard ** that matches any character length and contains the package name separator (.)

• Extends, which specifies the base class of a class

• Implement, matches a class that implements an interface

• $, inner class

“Member” represents a qualification related to a class member, and it will eventually locate some class member that meets that qualification. Its contents can be used:

• Matches all constructors
• Match all fields
• Match all methods

• Wildcard *, which matches characters of any length without the package name separator (.)

• Wildcard ** that matches any character length and contains the package name separator (.)

• The wildcard character is ***, which matches any parameter type

• … To match any type parameter of any length. Such as void test (…). We can match any void test(String a) or void test(int a, String b) method.

• Access modifier (public, protected, private)

For example, if you want to keep all public classes and constructors that inherit Activity in the name.huihui. Test package, you could write:

-keep public class name.huihui.test.** extends Android.app.Activity {
    <init>
}Copy the code

3. Common custom obfuscation rules

• Do not confuse a class

  -keep public class name.huihui.example.Test { *; }Copy the code

• Do not confuse all classes of a package

  -keep class name.huihui.test.** { *; }Copy the code

• Do not confuse subclasses of a class

  -keep public class * extends name.huihui.example.Test { *; }Copy the code

• Do not confuse all classes and their members that have “Model” in their name

  -keep public class **.*model*.** {*; }Copy the code

• Do not confuse the implementation of an interface

  -keep class * implements name.huihui.example.TestInterface { *; }Copy the code

• Do not confuse the constructor of a class

  -keepclassmembers class name.huihui.example.Test { 
    public <init>(); 
  }Copy the code

• Do not confuse specific methods of a class

  -keepclassmembers class name.huihui.example.Test { 
    public void test(java.lang.String); 
  }Copy the code

Custom resource retention rules

1. keep.xml

If shrinkResources true is used to enable resource compression, all unused resources are removed by default. If you need to define which resources must be retained, create an XML file in the res/raw/ directory, such as keep.xml.

The requirements for resource retention can be defined by setting some properties. The configurable properties are:

• Tools :keep Defines which resources need to be reserved (separated by commas (,)).

• Tools :discard Defines the resources to be removed (separated by commas (,)).

• Tools :shrinkMode Enables the strict mode

Normal resource reference checking can be problematic when Resources are retrieved and used with dynamic strings in your code through resources.getidentifier (). For example, the following code causes all resources beginning with “img_” to be marked as used.

String name = String.format("img_%1d", angle + 1);
res = getResources().getIdentifier(name, "drawable", getPackageName());Copy the code

You can set Tools :shrinkMode to strict to enable the strict mode. In this way, only the resources that are actually in use are retained.

This is the configuration associated with custom resource retention rules, as an example:

<? The XML version = "1.0" encoding = "utf-8"? > <resources xmlns:tools="http://schemas.android.com/tools" tools:keep="@layout/l_used*_c,@layout/l_used_a,@layout/l_used_b*" tools:discard="@layout/unused2" tools:shrinkMode="strict"/>Copy the code

2. Remove alternative resources

Some alternative resources, such as strings. XML supported by multiple languages and layout. XML supported by multiple resolutions, can be removed by using resource compression when we do not need to use them and do not want to delete them.

We use the resConfig property to specify the properties that need to be supported, for example

android {
    defaultConfig {
        ...
        resConfigs "en", "fr"
    }
}Copy the code

Other language resources that are not explicitly declared will be removed.

The resources

• Shrink Your Code and Resources

• proguard

• Android Security, decompilation and obfuscation

• Android confounds from beginner to master

• ProGuard for Android code obfuscation

The appendix

1. Proguard-android. TXT File contents

# # package name not mixed cases - dontusemixedcaseclassnames don't skip the public library class - dontskipnonpubliclibraryclasses # # confusion when logging - verbose close preverification Keepattributes *Annotation* # keepattributes *Annotation* # keep all class names with local methods and local method names -keepclasseswithmembernames class * { native <methods>; } public class * extends android.view.View {void set*(***); *** get*(); } class * extends android.app.activity {public void *(android.view.view); } # enumeration-keepclassmembers enum * {**[] $VALUES; public *; } #Parcelable -keepclassmembers class * implements android.os.Parcelable { public static final android.os.Parcelable$Creator CREATOR; R$* {public static <fields>; } - dontwarn android. Support. * * # keep related annotation - keep class. Android support. The annotation. Keep - keep @android.support.annotation.Keep class * {*; } -keepclasseswithmembers class * { @android.support.annotation.Keep <methods>; } -keepclasseswithmembers class * { @android.support.annotation.Keep <fields>; } -keepclasseswithmembers class * { @android.support.annotation.Keep <init>(...) ; }Copy the code

Write in the last

Thank you very much for seeing the end, because my level is limited, if there are mistakes, please do not hesitate to comment, you can directly reply to the public number or go to the blog comment, here in advance thank you.

At the same time, if there is any confusion or discussion in this respect, please feel free to contact me.