Serialization and deserialization of GSON

GSON is a great tool for easy serialization and deserialization. But when it comes to confusion, we need to pay attention.

A simple class Item that handles serialization and deserialization

public class Item {
    public String name;
    public int id;
}
Copy the code

Serialization code

Item toSerializeItem = new Item();
toSerializeItem.id = 2;
toSerializeItem.name = "Apple";
String serializedText = gson.toJson(toSerializeItem);
Log.i(LOGTAG, "testGson serializedText=" + serializedText);
Copy the code

Enable log output after obfuscation

I/MainActivity: testGson serializedText={"a":"Apple","b":2}
Copy the code

The property names have been changed to meaningless names, which can be troublesome for some of our subsequent processing.

Deserialization code

Gson gson = new Gson(); Item item = gson.fromJson("{\"id\":1, \"name\":\"Orange\"}", Item.class); Log.i(LOGTAG, "testGson item.id=" + item.id + "; item.name=" + item.name);Copy the code

The corresponding log result is

I/MainActivity: testGson item.id=0; item.name=nullCopy the code

As you can see, the deserialized property value Settings failed after the confusion.

Why is that?

Because deserialization creates an object using reflection, the key of the JSON string is used as the property name, and the value corresponds to the property value.

How to solve

Eliminate confusion between serialized and deserialized classes
Annotate the field with @serializedName

@serializedName (parameter) is implemented with an annotation attribute

In the serialized result, specify the key as the value of parameter.
In the deserialized generated object, used to match key and parameter and assign attribute values.

A simple usage is

public class Item {
    @SerializedName("name")
    public String name;
    @SerializedName("id")
    public int id;
}
Copy the code

Or you could just write it this way

public class Item implements Serializable{
    public String name;
    public int id;
}
Copy the code

Here’s what happens when deserialization is obfuscated by adding the @serializedName annotation. Here’s how to remove the confusion between serialized and deserialized classes.

package com.baidu.bean; public class Item { public String name; public int id; public static class PageConfig { public String type; }}Copy the code

Added an inner class PageConfig to Item.

Here’s the blackboard:

1. The fields in Item, the classes referenced in Item, and the PageConfig inner class in Item all need to be Serializable;

If, instead of implements Serializable, you annotate each field with @serializedName, it is important to note: The @serializedName annotation is required for all fields in Item, all classes referenced by Item, and all inner classes in Item. Otherwise, the @serializedName annotation will cause some weird problems.

The most common practices are:

-keep class com.baidu.bean.** { *; }
Copy the code

Exclude classes in the bean directory, including subdirectories, from confusion

To exclude a class, write:

-keep class com.baidu.bean {*; }

To exclude an inner class from a class, write:

-keep class class com.baidu.bean.Item$PageConfig {*; }

If many entity classes have inner classes, it is recommended to combine them:

-keep class com.baidu.bean.**{ *; } -keep class com.baidu.bean.**$*{ *; }Copy the code

In addition, the following writing method is also possible, mainly above writing method. The specific type of reader can be used according to their needs.

-keep class com.baidu.bean.$** {; }

-keep class com.baidu.bean.$ {*; }

-keep class com.baidu.bean.**$* {*; }

* and ** wildcard configuration appears above, in order to deepen the impression, here is an extension:

Android confuses best practices

1. Confuse the configuration

android{
 
buildTypes {
        release {
            buildConfigField "boolean", "LOG_DEBUG", "false" //不显示log
            minifyEnabled true
            shrinkResources true
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
            signingConfig signingConfigs.config
            }
        }
}
Copy the code

This function is disabled in debug mode because obfuscation takes longer to compile. What we need to do is:

willRelease the minifyEnabledTo change the value oftrue, open confusion;
addshrinkResources trueTo enable resource compression.

BuildConfigField Does not display the log. 4. SigningConfig SigningConfigs. config Configs the signature file

Custom obfuscation rules

Custom obfuscation schemes work for most projects

- optimizationpasses 5 # # specified compression level cannot skip class members of the non public library - the algorithm adopted in dontskipnonpubliclibraryclassmembers # confusion - optimizations. code/simplification/arithmetic,! field/*,! Class/done / * # confusion in the class, the name also confused - useuniqueclassmembernames # optimization there allowed to access and modify the modifier when members of the class and class - allowaccessmodification # will source file renamed "SourceFile string" - renamesourcefileattribute SourceFile # keep the line Numbers - keepattributes SourceFile, LineNumberTable # generics - KeepAttributes Signature # Keeps all classmembers that implement the Serializable interface. - KeepClassMembers class * implements java.io.Serializable {static  final long serialVersionUID; private static final java.io.ObjectStreamField[] serialPersistentFields; private void writeObject(java.io.ObjectOutputStream); private void readObject(java.io.ObjectInputStream); java.lang.Object writeReplace(); java.lang.Object readResolve(); } #Fragment does not need to be registered in androidmanifest.xml, Need additional protection - keep public class * extends android. Support. The v4. App. The fragments - keep public class * extends android. The app. # fragments ** - DontNote junit.runner.** -dontwarn Android.test.** -dontwarn android.support.test.** -dontwarn org.junit.**Copy the code

These are the really generic ones that need to be added, but each project needs to add some obfuscation rules for its own needs:

Obfuscation rules required by third-party libraries. Formal third-party libraries generally write the obfuscation rules in the access document, which should be added when using.

Code that changes dynamically at run time, such as reflection. A good example is an entity class that converts to and from JSON. -keep public class **.* model *.** {*; -keep public class **. }

Classes invoked in JNI.

A method called by JavaScript in a WebView

Layout View constructor, Android :onClick, etc.

Check for confusion

Obfuscated packages must be checked to avoid bugs introduced through obfuscation.

On the one hand, you need to check at the code level. Using the above configuration for confusion after packaged in < module name > / build/outputs/mapping/release/directory will output the following files:

dump.txt

Describes the internal structure of all classes in an APK file

mapping.txt

Provides a comparison table of classes, methods, class members, and so on before and after the obfuscation

seeds.txt

Lists classes and members that are not confused

usage.txt

Lists the code that was removed

We can check the unscrambled classes and members to see if they contain all the desired ones using the seeds.txt file, and then use the usage.txt file to see if any code was mistakenly removed.

On the other hand, you need to check from the testing side. The obfuscated packages are thoroughly tested for bugs.

Solve the obfuscation stack

The confused class and method names are difficult to read, which, of course, will increase the difficulty of reverse engineering, but also cause obstacles to tracking online crash. After we get the stack information of crash, we will find it difficult to locate. In this case, we need to reverse the confusion.

The accompanying undecompress tool (proGuardgui. bat for Windows or ProGuardgui.sh for Mac or Linux) is in < sdK-root >/tools/ proGuard /.

The Window platform is used as an example. After you double-click proGuardgui.bat, you can see a menu on the left. Confused click ReTrace, select the package corresponding mapping file (after the confusion in the/build/outputs/mapping/release/path generates mapping. TXT file, its role is to provide the confusion before and after class, method, class member table), Then paste crash’s stack trace into the input box, click ReTrace in the lower right corner, and the confused stack information will be displayed.

The above is done using a GUI program. Another way is to use the retrace tool in this path to reverse the solution from the command line

retrace.bat|retrace.sh [-verbose] mapping.txt [<stacktrace_file>]

Such as:

retrace.bat -verbose mapping.txt obfuscated_trace.txt

Matters needing attention:

All classes involved in androidmanifest.xml are automatically retained, so there is no need to add this obfuscate rule. (Used in many old obfuscations, no longer necessary)

Proguard-android. TXT already has some default obturation rules, there is no need to add them in proguard-rules.pro

Confuse the introduction

“Obfuscation” in Android can be divided into two parts. One is optimization and obfuscation of Java code, which is realized by proGuard obfuscation. The other part is resource compression, which removes unused resources from projects and dependent libraries (resource compression is not strictly related to confusion, but is generally used together).

Code compression

Code compression process

Code obfuscation is a process involving code compression, optimization and obfuscation. As shown in the figure above, the obfuscation process has the following functions:

Compression. Remove invalid classes, class members, methods, attributes, etc. Optimization. Analyze and optimize method binary code; As described in Proguard-Android-optimize.txt, optimizations pose some potential risks and are not guaranteed to work on all versions of Dalvik. Confusion. Replace class, attribute, and method names with short, meaningless names; Preverification. Add preverification information. This prevalidation works on The Java platform, not on Android, and can be removed to speed up obfuscation. These four processes are enabled by default.

In an Android project, you can turn optimizations and prevalidations off by using -dontoptimize and -dontpreverify. (Of course, the default proGuard-Android. TXT file already contains these obturations, so you don’t need to configure them.)

Resources compression

Resource compression removes unused resources from projects and dependent libraries. This can be useful in reducing the size of APK packages and is generally recommended. Set shrinkResources to true in the build.grade file. Note that resource compression will only take effect if code compression is enabled with minifyEnabled True.

Resource compression consists of merging resources and removing resources.

In the Merge Resource process, resources with the same name are regarded as duplicate resources and merged. Note that this process is not controlled by the shrinkResources property and cannot be disabled. Gradle must do this because it will cause an error if there are resources with the same name in different projects. Gradle looks for duplicate resources everywhere:

SRC /main/res/ path Different build types (debug, release, etc.) Different build channels rely on third-party libraries to merge resources in the following order of priority

Dependencies -> Main -> Channels -> Build typeCopy the code

If a duplicate resource exists in the main folder and in different channels, Gradle chooses to keep the resource in the channel.

At the same time, if duplicate resources occur at the same level, such as SRC /main/res/ and SRC /main/res2/, Gradle cannot complete the resource merge and reports a resource merge error.

The “Remove resource” process is known by name, but it is important to note that, like code, obfuscating resource removal can also define which resources need to be retained, as described below.

Custom obfuscation rules

There is a line like this in the “obfuscation configuration” section above

proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'


Copy the code

This line of code defines that the obturation rule consists of two parts: the contents of proguard-Android. TXT in the tools/proguard/ folder of the SDK and the contents of ProGuard-rules.pro placed by default in the module root directory. The former is the default obfuscation file provided by the SDK, and the latter is where developers customize obfuscation rules.

Common obfuscation instructions

optimizationpasses
dontoptimize
dontusemixedcaseclassnames
dontskipnonpubliclibraryclasses
dontpreverify
dontwarn
verbose
optimizations
keep
keepnames
keepclassmembers
keepclassmembernames
keepclasseswithmembers
Keepclasseswithmembernames more detailed please go to the website

In particular, there are several commands related to the rule of keeping related elements out of confusion:

The command	role
-keep	Prevents classes and members from being removed or renamed
-keepnames	Prevents classes and members from being renamed
-keepclassmembers	Prevents members from being removed or renamed
-keepnames	Prevents members from being renamed
-keepclasseswithmembers	Prevents classes and members that own the member from being removed or renamed
-keepclasseswithmembernames	Prevents classes and members that own the member from being renamed

Rules that keep elements from participating in obfuscation

[Hold command] [class] {[member]}Copy the code

“Class” represents a class-related qualification that will eventually locate some classes that meet that qualification. Its contents can be used:

Concrete classes
Access modifiers (public, protected, private)
Wildcard *, matches any length of character, but does not contain the package name delimiter (.)
The wildcard character ** matches characters of any length and contains the package name delimiter (.)
Extends, which is the base class of a class that you can specify
Implement, matches a class that implements an interface
$, inner class

“Member” represents a qualification associated with a class member, which will eventually locate some class member that meets that qualification. Its contents can be used:

Matches all constructors
Match all domains
Match all methods
Wildcard *, matches any length of character, but does not contain the package name delimiter (.)
The wildcard character ** matches characters of any length and contains the package name delimiter (.)
The wildcard character *** matches any parameter type
… Matches any type of argument of any length. Such as void test (…). I can match any void test(String a) or void test(int A, String b).
Access modifiers (public, protected, private) For example, if you want to preserve all public classes and constructors that inherit from the Activity in the com.biaobiao.test package, you can write:

 -keep public class com.biaobiao.test.** extends Android.app.Activity {
    <init>
}
Copy the code

Common custom obfuscation rules

Don’t mess with a class

-keep public class com.biaobiao.example.Test { *; }


Copy the code

Do not confuse all classes of a package

-keep class com.biaobiao.test.** { *; }}Copy the code

Do not confuse subclasses of a class

-keep public class * extends com.biaobiao.example.Test { *; }
 
Copy the code

Do not confuse all classes and their members whose class names contain “Model”

-keep public class * extends com.biaobiao.example.Test { *; }
Copy the code

Do not confuse the implementation of an interface

-keep class * implements com.biaobiao.example.TestInterface { *; }
Copy the code

Do not obfuscate the constructor of a class

-keepclassmembers class com.biaobiao.example.Test { 
    public <init>(); 
}
Copy the code

Do not confuse specific methods of a class

-keepclassmembers class com.biaobiao.example.Test { public void test(java.lang.String); }}Copy the code

Do not confuse inner classes of a class

-keep class com.biaobiao.example.Test$* {
        *;
 }
Copy the code

Customize resource retention rules

1. keep.xml

With shrinkResources true enabled, all unused resources are removed by default. If you need to define which resources must be reserved, create an XML file in the res/raw/ path, such as keep.xml.

You can define resource retention requirements by setting the following properties:

keepDefine which resources need to be reserved (separated by commas)
discardDefine which resources need to be removed (separated by commas)
shrinkModeTurn on strict mode
When the code passesResources.getIdentifier()When retrieving and using resources using dynamic strings, ordinary resource reference checking can be problematic. For example, the following code causes all resources starting with “img_” to be marked as used.

When the code passesResources.getIdentifier()When retrieving and using resources using dynamic strings, ordinary resource reference checking can be problematic. For example, the following code causes all resources starting with “img_” to be marked as used.

String name = String.format("img_%1d", angle + 1);
res = getResources().getIdentifier(name, "drawable", getPackageName());
Copy the code

We can set Tools :shrinkMode to strict to enable strict mode so that only resources that are actually used are reserved.

The above is the configuration related to custom resource retention rules, as an example:

<? The XML version = "1.0" encoding = "utf-8"? > <resources xmlns:tools="http://schemas.android.com/tools" tools:keep="@layout/l_used*_c,@layout/l_used_a,@layout/l_used_b*" tools:discard="@layout/unused2" tools:shrinkMode="strict"/>Copy the code

Remove alternative resources

Some alternative resources, such as strings. XML with multi-language support and layout. XML with multi-resolution support, can be removed by resource compression when they are not needed and do not want to be deleted.

We use the resConfig attribute to specify the attributes that need to be supported. For example, some alternative resources, such as strings. XML with multi-language support and layout.xml with multi-resolution support, can be removed by resource compression when they are not needed and do not want to be deleted.

We use the resConfig property to specify the properties that need to be supported, for example

android {
    defaultConfig {
        ...
        resConfigs "en", "fr"
    }
}
Copy the code

Other language resources that are not explicitly declared are removed.

Finally, I attached a confusing scheme in the actual project

Proguard-android. TXT File contents

# OptimizationPasses 5 # Do not use case when blending Blend to class called lowercase - dontusemixedcaseclassnames # specified not to ignore the public library classes - dontskipnonpubliclibraryclasses # don't do the divverify, preverify is one of the four steps of proguard, Android does not require PreVerify, and removing this step can speed up obfuscation. - DontPreVerify -verbose # Avoid obfuscating generics - KeepAttributes Signature # Preserve annotations without obfuscating - KeepAttributes *Annotation*,InnerClasses # Google Recommendation Algorithm - Optimizations! code/simplification/arithmetic,! code/simplification/cast,! field/*,! Class/done / * # avoid confusion the Annotation, inner classes, generics, anonymous classes - keepattributes * * an Annotation, InnerClasses, Signature, EnclosingMethod # Rename an exception is thrown when the file name - renamesourcefileattribute SourceFile # when they throw an exception code line number - keepattributes SourceFile, LineNumberTable # ** -dontwarn android.support.** # Keep the extension public class * extends android.support.v4.** -keep public class * extends android.support.v7.** -keep public class * extends Android. Support. The annotation. * * # retain R the following resources - keep class * * * R ${*; } # keep four components, - Keep public class * extends Android.app. Activity - Keep public class * extends android.app.Appliction -keep public class * extends android.app.Service -keep public class * extends android.content.BroadcastReceiver -keep public class * extends android.content.ContentProvider -keep public class * extends android.preference.Preference -keep public class com.android.vending.licensing.ILicensingService # The method parameters that remain in the Activity are the methods of the View, Keepclassmembers class extends Android.app.activity {public void keepClassMembers extends Android.app.activity {public void keepClassMembers extends Android.app.activity {public void keepClassMembers extends Android.app.activity {public void keepClassMembers extends Android.app.activity {public void keepClassMembers extends Android.app.activity *(android.view.View); Keepclassmembers class * {void *(**On*Event); keepclassMembers class * {void *(**On*Event); void *(**On*Listener); } # retain local native method is not be confused - keepclasseswithmembernames class * {native < the methods >; {public static **[] values();} # keepClassMembers enum * {public static **[] values(); public static ** valueOf(java.lang.String); } # keep class * implements android.os.Parcelable {public static final android.os.Parcelable$Creator *; } -keepclassmembers class * implements java.io.Serializable { static final long serialVersionUID; private static final java.io.ObjectStreamField[] serialPersistentFields; private void writeObject(java.io.ObjectOutputStream); private void readObject(java.io.ObjectInputStream); java.lang.Object writeReplace(); java.lang.Object readResolve(); } # assume no side effects: delete android. Util. The Log output Log - assumenosideeffects class android. Util. Log {public static * * * v (...). ; public static *** d(...) ; public static *** i(...) ; public static *** w(...) ; public static *** e(...) ; } # Keep Keep the name of the class and method annotations - Keep, allowobfuscation @ interface. Android support. The annotation. Keep - Keep @android.support.annotation.Keep class * -keepclassmembers class * { @android.support.annotation.Keep *; } #3D map before V5.0.0: -dontwarn com.amap.api.** -dontwarn com.autonavi.** -keep class com.amap.api. } -keep class com.autonavi.**{*; } -keep class com.amap.api.maps.**{*; } -keep class com.autonavi.amap.mapcore.*{*; } -keep class com.amap.api.trace.**{*; } #3D map after V5.0.0: -keep class com.amap.api.maps.**{*; } -keep class com.autonavi.**{*; } -keep class com.amap.api.trace.**{*; } # location -keep class com.amap.api.location.**{*; } -keep class com.amap.api.fence.**{*; } -keep class com.autonavi.aps.amapapi.model.**{*; } # search -keep class com.amap.api.services.**{*; } #2D map -keep class com.api.maps2d.**{*; } -keep class com.amap.api.mapcore2d.**{*; } # keep class com.amap.api.navi.**{*; } -keep class com.autonavi.**{*; } # Retain generic type information for use by reflection by converters and adapters. -keepattributes Signature # Retain  service method parameters when optimizing. -keepclassmembers,allowshrinking,allowobfuscation interface * { @retrofit2.http.* <methods>; } # Ignore annotation used for build tooling. -dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement # Ignore JSR 305 annotations for embedding nullability information. -dontwarn javax.annotation.** # JSR 305 annotations are for embedding nullability information. -dontwarn javax.annotation.** # A resource is loaded with a relative path so the package of this class must be preserved. -keepnames class okhttp3.internal.publicsuffix.PublicSuffixDatabase # Animal Sniffer compileOnly dependency to ensure APIs are compatible with older versions of Java. -dontwarn org.codehaus.mojo.animal_sniffer.* # OkHttp platform used only on JVM and when Conscrypt dependency is available. - dontwarn okhttp3. Internal. Platform. ConscryptPlatform # fastjson confusion - keepattributes Signature - dontwarn com.alibaba.fastjson.** -keep class com.alibaba.**{*; } -keep class com.alibaba.fastjson.**{*; } -keep public class com.ninstarscf.ld.model.entity.**{*; }Copy the code

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Android development confusion after Gson serialization, deserialization of those pits

Serialization and deserialization of GSON

Why is that?

How to solve

Android confuses best practices

1. Confuse the configuration

Custom obfuscation rules

Check for confusion

Solve the obfuscation stack

Confuse the introduction

Code compression

Resources compression

Custom obfuscation rules

Common obfuscation instructions

Rules that keep elements from participating in obfuscation

Common custom obfuscation rules

Customize resource retention rules

1. keep.xml

Remove alternative resources

Finally, I attached a confusing scheme in the actual project

Android development confusion after Gson serialization, deserialization of those pits

Serialization and deserialization of GSON

Why is that?

How to solve

Android confuses best practices

1. Confuse the configuration

Custom obfuscation rules

Check for confusion

Solve the obfuscation stack

Confuse the introduction

Code compression

Resources compression

Custom obfuscation rules

Common obfuscation instructions

Rules that keep elements from participating in obfuscation

Common custom obfuscation rules

Customize resource retention rules

1. keep.xml

Remove alternative resources

Finally, I attached a confusing scheme in the actual project

Related Posts

Flutter learning Roadmap

OkHttp 4 source code (7) – Summary

Android 9.0 silent installation implementation