@[TOC]

I met ProGuard

Android developers are all familiar with obfuscation, and many of them are confused by it. When the version needs to be issued, load a confused document from the Internet or copy it from other projects and modify it. If it works, it will be left alone. If there is a problem, it will get stuck. This paper tries to make everyone familiar with the rules of confusion, can quickly get started. Knowing what is can also know why.

ProGuard has been integrated since Android Studio2.3. ProGuard is a Java class file confocator that integrates compressors, optimizers, confocators, and prevalidators. ProGuard’s main advantage over other Java obfuscators is probably its compact template-based configuration. A few straightforward command-line options or a simple configuration file are usually all it takes. ProGuard reduces the size of the processed code and provides some potential efficiency gains. It takes only seconds to process megabytes of programs and libraries.

ProGuard is typically used to create more compact code for smaller code archives, faster network transfers, faster loads, and smaller memory footprint. Makes it harder to reverse engineer programs and libraries. Lists invalid code that can be removed from the source code. Reposition and pre-validate existing Java 6 class files to take advantage of Java 6’s faster class loading speed.

Configuring ProGuard obfuscation in Gradle is simple:

android {
    buildTypes {
        release {
            minifyEnabled true
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}
Copy the code

If the minifyEnabled attribute is set to true, obfuscation is enabled. The proguardFiles property specifies the directory where the obfuscated files reside. TXT is the default obturation file in the SDK path. The following ‘proguard-rules.pro’ is our custom obturation file.

ProGuard processes code as follows: the Shrunk compressor detects and removes unused classes, fields, methods, and properties. The Optimize optimizer analyzes and optimizes the method’s bytecode. The Obfuscate Obfuscate Obfuscate uses short, meaningless names to rename the rest of the classes, fields, and methods. These steps make the code base smaller, more efficient, and more difficult to reverse engineer. The final Preverify prevalidator adds the prevalidation information to the class, which is required for Java Micro Edition to reduce Java 6 startup time.

ProGuard,

What’s compressing?

Java source code (.java files) is usually compiled into bytecode (.class files). Bytecodes are more compact than Java source, but they can still contain a lot of unused code, especially if libraries are included. Compressors (such as ProGuard) can analyze bytecode and remove unused classes, fields, and methods. The program remains functionally equivalent, including the information given in the exception stack trace.

What is confusion?

By default, compiled bytecode still contains a lot of debugging information: source file names, line numbers, field names, method names, parameter names, variable names, and so on. This information makes it easy to compile the bytecode directly and reverse engineer the entire program. Sometimes, this is undesirable. Obfuscators such as ProGuard can remove debugging information and replace all names with meaningless character sequences, making it more difficult to reverse engineer code. It also further compresses the code. The program remains functionally equivalent, except for the class name, method name, and line number given in the exception stack trace.

reflection

Reflection poses special problems for automatic handling of code. In ProGuard, classes or class members that are dynamically created or called in code must be specified as entry points, protected with the Keep option. For example, the class.forname () construct can reference any Class at run time. It is often impossible to foresee which classes (and their original names) must be preserved, such as class names that can be read from configuration files. Therefore, they must be specified using the same simply-keep option in the ProGuard configuration.

In addition, if you need to keep certain classes or class members, ProGuard provides some advice. For example, ProGuard will look for structures like “(SomeClass) class.forname (variable).newinstance ()”. These may indicate that the class or interface SomeClass or its implementation may need to be preserved. We can then adjust the obfuscation configuration accordingly.

Confusion options

A confusing document mainly consists of a series of keep options and non-keep options. The keep option tells ProGuard which classes and class members are not to be confused. Non-keep options include input, compression, optimization, obfuscating, general, and other options to tell ProGuard about additional configuration.

Not keep options

  • Input options – skipnonpubliclibraryclasses specified skip the public class, while reading library jar to speed up the processing speed and reduce the memory usage of ProGuard. By default, ProGuard reads non-public public library classes. However, non-public classes are generally irrelevant, as long as they do not affect the actual program code in the input JAR. Ignoring them then speeds up ProGuard without affecting output. Unfortunately, some libraries, including the most recent JSE runtime libraries, contain non-public library classes extended by public library classes. If the class cannot be found because this option is set, ProGuard prints a warning.

    – dontskipnonpubliclibraryclasses specified not to ignore the public library classes. This is the default setting starting from version 4.5.

    – visible dontskipnonpubliclibraryclassmembers specified don’t ignore the package library class members (fields and methods). By default, ProGuard skips these class members when parsing library classes, because program classes typically don’t refer to them. Sometimes, however, program classes and library classes reside in the same package, and they do refer to class members visible to their package. In this case, it might be useful to actually read the class members to ensure that the code is consistent after processing.

  • Compression options Enable compression by default. All classes and class members are removed except those listed with the various -keep options and those on which they directly or indirectly depend. After each optimization step, a compression step is performed, because optimization may again expose unused classes and members. To turn off compression: -dontshrink

  • Optimization is enabled by default. All methods are optimized at the bytecode level. But in some cases, optimization can lead to exceptions, which may change the logic of the program. For example, it removed some special comments and empty loops that it thought were meaningless. To turn off optimization: -dontoptimize

    – optimizationPasss n Specifies the number of optimizations to be performed. By default, a pass is performed once. Multiple passes may lead to further improvements. If no improvement is found after the optimization passes, the optimization ends. Only applicable for optimization.

    – AllowAccessModification Specifies the access modifiers that can enlarge classes and class members during processing. This improves the results of the optimization step.

  • Obfuscation is turned on by default. In addition to the names listed with the various -keep options, classes and class members receive new short random names. Removed internal properties useful for debugging, such as source file names, variable names, and line numbers. Close obfuscate: -Dontobfuscate

    – printMapping [filename] Prints the mapping from the old name to the new name for the renamed class and class members. The mapping is printed to standard output or to a given file.

    – useuniqueclassmembernames this option will need in the name of the class generation only confused confusion. Without this option, more class members are mapped to the same short name, such as “a”, “b”, and so on.

    – dontusemixedcaseclassnames specified at the time of confusion generated mixed case the name of the class, namely all lowercase. By default, obfuscated class names can contain a mixture of uppercase and lowercase characters.

    -keeppackagenames [package_filter] Specifies not to confuse the specified packagename. The optional filter is a comma-separated list of package names. The package name can contain? , * and ** wildcards, or preceded by! . When the main project is different library projects, the class names of different library projects may conflict, such as A.A.A.A. A compilation error occurs when the main project references the obfuscated library AAR:

#Duplicate class a.a.a.a found in modules classes.jar (:libA-release:) and classes.jar (:libB-release:)
Copy the code

Keeppackagenames can avoid this problem by specifying that the packagenames of a library are not confused.

– keepAttributes [attribute_filter] Specifies optional attributes to keep. You can specify attributes using one or more -keepAttributes directives. The optional filter is a comma-separated list of attribute names. Attribute names can include? , * and ** wildcards, or preceded by! . Typical optional attributes include: Exceptions, Signature, InnerClasses, Deprecated, SourceFile, SourceDir, LineNumberTable, LocalVariableTable, LocalVariableTypeTable, Synthetic, EnclosingMethod RuntimeVisibleAnnotations RuntimeInvisibleAnnotations, RuntimeVisibleParameterAnnotations RuntimeInvisibleParameterAnnotations and AnnotationDefault.

For example, at a minimum, the Exceptions, InnerClasses, and Signature attributes should be preserved when working with libraries. The SourceFile and LineNumberTable attributes should also be preserved to produce useful obfuscation stack traces. Finally, if your code relies on annotations, you may want to keep them.

Example:

- keepattributes Exceptions, InnerClasses, Signature # keep internal interfaces or classes, the types of the inner class, generic Signature - renamesourcefileattribute SourceFile # will collapse source log file renamed "SourceFile" - keepattributes SourceFile, confused LineNumberTable # generate useful stack trace - keepattributes * * an Annotation # retain themCopy the code

  • General options

    -verbose Displays more information during processing. If the program terminates due to an exception, this option prints out the entire stack trace, not just the exception message.

    -dontnote [class_filter] specifies not to print comments about potential errors or omissions in the configuration, such as typos in class names or missing options that might be useful. The optional filter class_filter is a regular expression; ProGuard does not print comments for classes that match the optional name.

    -dontwarn [class_filter] specifies that unresolved references and other important issues are not warned. The optional filter class_filter is a regular expression; ProGuard does not print warnings for classes that match the optional name. Ignoring warnings can be dangerous. For example, if you do need to process an unresolved class or class member, the processed code will not work. Use this option only if you know what you are doing!

    – ignoreWarnings Specifies to print any warnings about unresolved references and other important issues, but will continue to be processed in any case. Ignoring warnings can be dangerous. For example, if you do need to process an unresolved class or class member, the processed code will not work. Use this option only if you know what you are doing!

    File filters A file filter is a comma-separated list of file names that can contain wildcards. The following wildcards are supported:? Matches any single character in the name. * Matches a name that does not contain the package separator “. Or any part of the directory separator “/”. ** Matches any part of the name and may contain any number of package or directory separators. For example, “Java / **. Class, javax / **. Class” matches all class files in Java and Javax. “Foo, *bar” matches the name foo and all names ending in bar.

In addition, names can be preceded by a minus sign “!” . Excludes the file name from the matching file name. For example,

! "" **.gif, images/** "# match all files in the images directory, except GIF files. ! "" Foobar,*bar" # matches all names ending in bar except foobar.Copy the code

Keep options

The keep option is used to declare classes and class members to be retained in obfuscation rules, preventing them from being deleted and renamed. The general format is as follows:Copy the code

– Keep option class_specification Class_specification is a template for classes and members that specifies the classes and members to which the KEEP rule applies.

The keep option falls into two categories, depending on whether it can be deleted in the compression phase or renamed in the obfuscation phase: – keep, – keepclassmembers, – keepclasseswithmembers, respectively While retaining the classes and class members, only keep members, the members of the root class according to find meet the conditions of all the classes and don’t need to specify the name of the class, keep the name of the class and member names. The second type, with names, cannot be renamed: – keepnames, – keepclassmembernames, – keepclasseswithmembernames, respectively While retaining the classes and class members are not rename, only keep class members are not rename, root class members according to find meet the conditions of all the classes and don’t need to specify the name of the class, Keep class and member names from being renamed. For the second type, if the class is not called, it is deleted during the compression phase.

As shown in figure:

class_specification

Class_specification is a template for classes and members that specifies the classes and members to which the KEEP rule applies. The format is as follows:

[@annotationtype] [[!]public|final|abstract|@ ...] [!] interface|class|enum classname [extends|implements [@annotationtype] classname] [{ [@annotationtype] [[! ]public|private|protected|static|volatile|transient ...] <fields> | (fieldtype fieldname); [@annotationtype] [[! ]public|private|protected|static|synchronized|native|abstract|strictfp ...] <methods> | <init>(argumenttype,...) | classname(argumenttype,...) | (returntype methodname(argumenttype,...)); [@annotationtype] [[! ]public|private|protected|static ... ] *; ... }]Copy the code

Explain:

  • Square brackets “[]” indicate that its content is optional. The ellipsis “…” Indicates that any number of the preceding items can be specified. A vertical bar “|” define two options. The parentheses “()” group only the parts of the specification that belong to the same part.

  • The class keyword refers to any interface or class. The interface keyword restricts matches to the interface class. The enum keyword restricts matches to enumeration classes. Before an interface or enumeration keyword! Restrict matches to classes that are not interfaces or enumerations, respectively.

  • Each class name must be fully qualified, such as java.lang.String. You can specify a class name as a regular expression containing the following wildcard characters:? Matches any single character in the class name, but not the package separator “.”. * Matches any part of the class name except the package separator.

    "mypackage.*"Matches all classes in mypackage, but does not match all classes in subpackages.Copy the code

    ** Matches any part of the class name and may contain any number of package separators.

    **.test matches all Test classes in all packages except the root package."mypackage.**"Matches all classes in mypackage and its subpackages.Copy the code

  • @annotationtype can be used to limit classes and class members to members annotated with the specified annotationtype. Specify the annotation type just like the class name. Field and method specifications are very similar to those in Java, and the method parameter list for annotation types does not contain parameter names.

  • The class name * represents any class, regardless of its package.

  • < init> Matches any constructor < fields> matches any field < methods> Matches any method * Matches any field or method. Note that there is no return type for the wildcard. Only the < init> wildcard has a parameter list

  • Field and method names can contain the following wildcards:? Matches any single character in the method name. * Matches any part of the method name.

  • A type in a descriptor can contain the following wildcards: % matches any primitive type (” Boolean “, “int”, etc., but not “void”). ? Matches any single character in the name. Matches any part of the name that does not contain the package separator. ** Matches any part of the class name and may contain any number of package separators. Matches any type (primitive or non-primitive, array or non-array). . Matches any number of parameters of any type.

May I have your attention, please? , * and ** wildcards never match the base type. 2. In addition, only *** wildcards can match array types of any dimension. For example, "**get*()" matches "java.lang.object getObject()" but does not match" float getFloat()", nor does it match "java.lang.object [] getObjects()". 3. Constructors can also be specified using the short class name of the constructor (without the package) or the full class name. Like the Java language, the constructor specification has a list of arguments, but no return types. 4. Allow multiple class members to access modifier flags (for example, public static).Copy the code

Other precautions for ProGuard

  • Keep Native methods For native methods, keep their names and class names so that they can be linked to the local library:
-keepclasseswithmembernames class * {
    native <methods>;
}
Copy the code
  • Keep enumerations If your program code contains enumeration classes, you must keep some special methods. Enumerations were introduced in Java 5. The Java compiler converts enumerations into classes with special structures. It is worth noting that these classes contain implementations of static methods that the runtime environment can access through introspection. These must be explicitly specified to ensure that they are not deleted or confused:
-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}
Copy the code


Some technical issues with ProGuard

There are some technical issues you should be aware of when using ProGuard, all of which can be easily avoided or resolved. ProGuard may print some cautions and non-fatal warnings when working with code:

  • Note: … calls ‘(…) Class.forName(variable).newInstance()’

ProGuard lists all Class casts for dynamically created Class instances, such as “(MyClass) class.forname (variable).newinstance ()”. We might need to use an option like “-keep class MyClass” to preserve the mentioned class, or an option like “-keep class * implements MyClass” to preserve its implementation.

  • Note: … accesses a field/method ‘… ‘ dynamically

ProGuard lists many constructs, such as “.getField (” myField “). We might need to figure out where the class member in question is defined and use “-keep class MyClass {MyFieldType myField; } “to keep them. Otherwise, ProGuard may remove or obfuscate class members.

  • Unexpected errors caused by optimizations are usually when ProGuard encounters an accident during the optimization step and may not recover. You can avoid this by using the -dontoptimize option

  • Preserving annotation Classes If you want to preserve classes based on annotations, you might avoid removing the annotation class itself in the compression step. You can use options like “-keep @interface *” to keep all annotation classes explicitly in the program code.

  • The ClassNotFoundException code may be calling class.forname in an attempt to dynamically create the missing Class. ProGuard can only detect constant name parameters, such as class.forname (” mypackage.myclass “). For variable name arguments like class.forname (someClass), you must use the appropriate -keep option to keep all possible classes, for example:

"-keep class mypackage.MyClass" 
"-keep class * implements mypackage.MyInterface".
Copy the code
  • The NoSuchMethodException code may be calling something like myClass.getMethod in an attempt to dynamically find some method. These methods have been confused. So the appropriate -keep option must be used:
"-keep class mypackage.MyClass { void myMethod(); }"
Copy the code

More specifically, if the method reported as missing is a value or valueOf, you may have to keep some enumeration-related methods.

  • By default, the obfuscation step removes all annotations. If your application relies on annotations to function properly, leave them explicitly with “-keep Attributes * Annotation *”.

  • BGF Loops If your code contains empty busy wait loops, the Optimization step of ProGuard may remove them. If it conflicts with the actual logic, the optimization must be turned off with the -dontoptimize option.

  • ClassCastException: class not an enum, or IllegalArgumentException: Class Not an Enum Type Should ensure that special methods of enumeration types are preserved, which the runtime environment calls through introspection.

  • ArrayStoreException: sun reflect. The annotation. EnumConstantNotPresentExceptionProxy may be processing involving the enumeration annotation. Again, you should make sure to keep special methods of enumerated types.

  • For the DEX compiler and Dalvik VM, pre-validation is irrelevant, so we can turn it off with the -dontpreVerify option.

  • – The Optimizations option disables certain arithmetic simplifications that Dalvik 1.0 and 1.5 cannot handle. The Dalvik VM also cannot handle excessive overload (of static fields).

To summarize, this is:

  • Classes or class members that require dynamic access need to be preserved
  • Use optimizations with caution, or even disable them directly with -dontoptimize
  • Preverify is not valid in Android, use -dontpreVerify to turn it off
  • Annotations and enumerations are retained as needed

A generic ProGuard obfuscation file

Finally, a more general reference to ProGuard obfuscation files is provided. We keep all the base classes that might be referenced in the application’s Androidmanifest.xml file. If the manifest file contains additional classes and methods, you may also have to specify them.

We reserve annotations because they might be used by a custom RemoteView.

We’ll use typical constructors to keep all custom View extensions and other classes, because they might be referenced from an XML layout file.

We also leave the required static fields in the Parcelable or Serializable implementation because they can be accessed through introspection.

Finally, we keep the auto-generated static fields of the R class that refer to the inner class so that the calling code can access them through introspection.

If you are using Google’s optional license validation library, you can confuse its code with your own. You must retain its ILicensingService interface for the library to work properly:

-keep public interface com.android.vending.licensing.ILicensingService
Copy the code

If you are using the Android compatibility library, add the following line to let ProGuard know that the library references certain classes that are not available in all versions of the API:

-dontwarn android.support.**
Copy the code

The “Exceptions” attribute must be kept so that the compiler knows which methods may throw Exceptions.

The additional -keep option should be used to specify any other non-public classes or methods only if they are dynamically called.

The “InnerClasses” attribute must also be preserved for any InnerClasses that can be referenced from outside the library. Otherwise, the Javac compiler will not be able to find the inner class.

When compiling in JDK 5.0 and later, you must have the “Signature” property to access generics. Finally, we leave the “Deprecated” attribute and the attribute for generating useful stack traces.

-keepattributes Exceptions,InnerClasses,Signature,Deprecated
Copy the code

In addition, formal third-party libraries will generally write the required obfuscation rules in the access document, which should be added when using third-party libraries. Methods that JavaScript calls in a WebView also need to be preserved. View constructors used for Layout, Android :onClick, etc., also need to be retained.

Optimizationpasses 5 optimizationPasses 5 OptimizationPasses That all lowercase - dontusemixedcaseclassnames # specify not to ignore the public library class - visible dontskipnonpubliclibraryclasses # specified don't ignore the package library class members (fields and methods). Confused - dontskipnonpubliclibraryclassmembers # class also confuse the method name in the # for confusion class generation only confused name - useuniqueclassmembernames # close prospective validation - dontpreverify # Print process logs to output more information during processing -verbose #-dontshrink # disable compression # Specify optimization algorithm - Optimizations! code/simplification/arithmetic,! field/*,! Class /merging/* # disable optimization -dontoptimize # Allows modifiers and class members to be accessed and modified during optimization - AllowAccessModification # 4 Components and subclasses of Application are not confused -keep public class * extends android.app.Activity -keep public class * extends android.app.Application -keep public class * extends android.app.Service -keep public class * extends android.content.BroadcastReceiver -keep public class * extends android.content.ContentProvider -keep public class * extends android.app.backup.BackupAgentHelper -keep public class * Extends android. Preference. Preference # if you are using Google's optional license library, the code can be confused with your own code. Must retain its ILicensingService interface in order to make the normal work of the library - keep public interface com. Android. Vending. Licensing. ILicensingService # will confuse the stack trace source file renamed "SourceFile" - renamesourcefileattribute SourceFile # protection annotation. If your code relies on annotations, you may need to keep annotations, typically using EventBus's events to receive callbacks - keepAttributes *Annotation* # Keep source file names, variable names, and line numbers, To produce useful to confuse the stack trace - keepattributes SourceFile, LineNumberTable # retain abnormal, internal class/interface, generics, Deprecated method is not recommended - keepattributes Exceptions, InnerClasses, Signature, Deprecated, EnclosingMethod # if quotes or v4 v7, the report warned. ProGuard know the library reference for all versions of the API is not available in some classes - dontwarn android. Support. * * # retain native method - keepclasseswithmembernames class * {native <methods>; } # keep custom View classes and constructors so that they can be referenced by XML layout files -keep class * extends Android.view. View {public <init>(android.content.context); public <init>(android.content.Context, android.util.AttributeSet); public <init>(android.content.Context, android.util.AttributeSet, int); } # keepClassMembers public class extends Android.view. View {void set*(***); *** get*(); } # keep the View and its subclasses in the Activity as input methods. Class * extends Android.app. Activity {public void *(android.view.view); } # preserve custom control classes that match the specified constructor type, if written together with the following, So only at the same time have the constructor of the class to meet the two - keepclasseswithmembers class * {public < init > (android. The content. The Context, android. Util. AttributeSet);  } -keepclasseswithmembers class * { public <init>(android.content.Context, android.util.AttributeSet, int); } # keep static members of R files so that calling code can access these fields through introspection - keepClassMembers class **.R$* {public static <fields>; {public static **[] values(); public static ** valueOf(java.lang.String); } # keep class * implements android.os.Parcelable {public static final android.os.Parcelable$Creator *; } # keep all classmembers that implement the Serializable interface - keepClassMembers * implements java.io.Serializable {static final Long serialVersionUID; private static final java.io.ObjectStreamField[] serialPersistentFields; private void writeObject(java.io.ObjectOutputStream); private void readObject(java.io.ObjectInputStream); java.lang.Object writeReplace(); java.lang.Object readResolve(); } #Fragment does not need to be registered in androidmanifest.xml, Need extra protection - keep public class * extends android. Support. The v4. App. The fragments - keep public class * extends android. The app. The fragments Keeppackagenames com.milanac007.* # Keep all files under the specified packagename -keep class com.milanac007.. blecommsdk.**{*; }Copy the code