preface
In fact, I ran into the problem of JSON parsing under Kotlin very early, and I didn’t have time to sort it out until now. The most widely used JSON library in Android development is Gson. There is no doubt that Gson is a very mature library, but after moving to Kotlin, Gson has two problems. The default value of class field is invalid, and non-empty types can be assigned null. In fact, both cases are mentioned for the same reason in Gson Issue #1550. In this article, we analyze the problem from phenomenon -> cause -> solution.
The sample code has been uploaded to Github
The phenomenon of
Where all fields have default values
@JsonClass(generateAdapter = true)
data class DefaultAll(
val name: String = "me".val age: Int = 17
)
fun testDefaultAll(a) {
val json = "" "{} "" "
val p1 = gson.fromJson(json, DefaultAll::class.java)
println("gson parse json: $p1")
val p2 = moshi.adapter(DefaultAll::class.java).fromJson(json)
println("moshi parse json: $p2")}/ / the result
// gson parse json: DefaultAll(name=me, age=17)
// moshi parse json: DefaultAll(name=me, age=17)
Copy the codeYou can see that neither Gson nor Moshi has a problem in this case
Some fields have default values
@JsonClass(generateAdapter = true)
data class DefaultPart(
val name: String = "me".val gender: String = "male".val age: Int
)
fun testDefaultPart(a) {
// There must be an age field. Moshi will not allow null age to be used for null security
val json = """{"age": 17}"""
val p1 = gson.fromJson(json, DefaultPart::class.java)
println("gson parse json: $p1")
val p2 = moshi.adapter(DefaultPart::class.java).fromJson(json)
println("moshi parse json: $p2")}/ / the result
// gson parse json: DefaultPart(name=null, gender=null, age=17)
// moshi parse json: DefaultPart(name=me, gender=male, age=17)
Copy the codeIn this case, GSON ignores the name and gender defaults and sets a null value for the non-empty type, which is not what is expected. Moshi had no effect.
Cause analysis,
Gson loses the default value
When Gson deserializes an object
- First try to get the constructor with no arguments
- On failure, the constructors for List, Map, and so on are tried
- Finally, let’s use the unsafe.newinstance wrapper. (This wrapper doesn’t call the constructor and therefore won’t be called by any object initialization code.)
This is obviously the case because Gson failed to get the class’s no-argument constructor and ended up in the unsafe scheme. Let’s take a look at the Java equivalent of the Kotlin code to find out. AS tools -> kotlin -> show kotlin bytecode You can view the compiled bytecode of kotlin. You can view the corresponding Java code after decompile.
All fields have default values
public final class DefaultAll {
@NotNull
private final String name;
private final int age;
@NotNull
public final String getName(a) {
return this.name;
}
public final int getAge(a) {
return this.age;
}
public DefaultAll(@NotNull String name, int age) {
Intrinsics.checkNotNullParameter(name, "name");
super(a);this.name = name;
this.age = age;
}
// $FF: synthetic method
public DefaultAll(String var1, int var2, int var3, DefaultConstructorMarker var4) {
if ((var3 & 1) != 0) {
var1 = "me";
}
if ((var3 & 2) != 0) {
var2 = 17;
}
this(var1, var2);
}
public DefaultAll(a) {
this((String)null.0.3, (DefaultConstructorMarker)null); }}Copy the codeAs you can see in this case, the class generates the null constructor, but instead of assigning values, it calls the synthetic Method, an additional generated auxiliary constructor, to assign default values to the fields. The penultimate argument of synthetic Method is an int, which is used to mark which fields are assigned with the default values. In the order of field declaration, the corresponding flag values are 2^n, that is, 1, 2, 4, 8….
Because the null constructor exists and the default value is assigned, gson is used normally in this case.
Some fields have default values
public final class DefaultPart {
@NotNull
private final String name;
@NotNull
private final String gender;
private final int age;
@NotNull
public final String getName(a) {
return this.name;
}
@NotNull
public final String getGender(a) {
return this.gender;
}
public final int getAge(a) {
return this.age;
}
public DefaultPart(@NotNull String name, @NotNull String gender, int age) {
Intrinsics.checkNotNullParameter(name, "name");
Intrinsics.checkNotNullParameter(gender, "gender");
super(a);this.name = name;
this.gender = gender;
this.age = age;
}
// $FF: synthetic method
public DefaultPart(String var1, String var2, int var3, int var4, DefaultConstructorMarker var5) {
// The minimum value not 0 indicates that the first default value field has no value in json, and the default value is required
if ((var4 & 1) != 0) {
var1 = "me";
}
if ((var4 & 2) != 0) {
var2 = "male";
}
this(var1, var2, var3); }}Copy the codeIn this case, the class does not generate a null constructor, so Gson instantiates with Unsafe, and the natural default does not take effect. The null constructor is actually generated only if all fields have default values.
Moshi normal working reasons
The Moshi library is owned by Square and originally led by Jake Wharton, a Kotlin fan. It’s not hard to assume that Moshi is compatible with Kotlin, and he is.
When Moshi serializes/deserializes, it creates the corresponding JsonAdapter based on the reflection of each class and uses it to perform specific operations. In addition, It supports the annotationProcessor to generate JsonAdapter for each class in advance during the compile time, which improves performance in space and time. Let’s start with the code generated by the annotator to see what it does, and select the DefaultPartJsonAdapter generated by the DefaultPart class.
public class DefaultPartJsonAdapter(
moshi: Moshi
) : JsonAdapter<DefaultPart>() {
private val options: JsonReader.Options = JsonReader.Options.of("name"."gender"."age")
private val stringAdapter: JsonAdapter<String> = moshi.adapter(String::class.java, emptySet(),
"name")
private val intAdapter: JsonAdapter<Int> = moshi.adapter(Int: :class.java, emptySet(), "age")
@Volatile
private var constructorRef: Constructor<DefaultPart>? = null
public override fun toString(a): String = buildString(33) {
append("GeneratedJsonAdapter(").append("DefaultPart").append(') ')}public override fun fromJson(reader: JsonReader): DefaultPart {
var name: String? = null
var gender: String? = null
var age: Int? = null
// Complement 32 1
var mask0 = -1
reader.beginObject()
while (reader.hasNext()) {
when (reader.selectName(options)) {
0- > {// The name field is not null. If the value in json is NULL, an exception is thrown
name = stringAdapter.fromJson(reader) ?: throw Util.unexpectedNull("name"."name", reader)
/ /... 1111 &... 1110 =... 1110, the lowest value 0 indicates that the first default field exists in json
mask0 = mask0 and 0xfffffffe.toInt()
}
1 -> {
gender = stringAdapter.fromJson(reader) ?: throw Util.unexpectedNull("gender"."gender",
reader)
/ /... 1110 &... 1101 =... 1100, lower low 0 indicates that the second default field exists in JSON and does not need to be default, and so on
mask0 = mask0 and 0xfffffffd.toInt()
}
2 -> age = intAdapter.fromJson(reader) ?: throw Util.unexpectedNull("age"."age", reader)
-1- > {// Unknown name, skip it.
reader.skipName()
reader.skipValue()
}
}
}
reader.endObject()
if (mask0 == 0xfffffffc.toInt()) {
// If all default fields exist in JSON, then we call the constructor to assign them to the values in JSON, ignoring the default values
return DefaultPart(
name = name as String,
gender = gender as String,
// The age field is not empty. If there is no corresponding key in json, throw an exceptionage = age ? :throw Util.missingProperty("age"."age", reader)
)
} else {
// If a field with default values does not exist in Json, then pass flag reflection to call the synthetic constructor and fill in the default values
@Suppress("UNCHECKED_CAST")
val localConstructor: Constructor<DefaultPart> = this.constructorRef ? : DefaultPart::class.java.getDeclaredConstructor(String::class.java, String::class.java,
Int: :class.javaPrimitiveType, Int: :class.javaPrimitiveType,
Util.DEFAULT_CONSTRUCTOR_MARKER).also { this.constructorRef = it }
returnlocalConstructor.newInstance( name, gender, age ? :throw Util.missingProperty("age"."age", reader),
mask0,
/* DefaultConstructorMarker */ null)}}}Copy the codeAnd what I’m doing is actually very simple, I’m writing comments in my code
- If the default value field has a key in the JSON to be parsed. if the default value field has a key in the JSON to be parsed. if the default value field has a key in the JSON to be parsed. if the default value field has a key in the JSON to be parsed. if the default value field has a key in the JSON to be parsed. if the default value field has a key in the JSON to be parsed. if the default value field has a key in the JSON to be parsed. if the default value field has a key, the key is 0
- If it is true, the default value is ignored and the instance is generated using the json field. If it is false, the synthetic constructor can only reflect the call. The synthetic constructor assigns default value fields according to flag bits
In short, Moshi achieves compatibility by following Kotlin’s mechanics.
The solution
After all this analysis, it’s obvious how to avoid invalid defaults
- When the class is defined, all fields are given a default value so that GSON works properly
- The use of Moshi library
Other problems, Json value is null
In normal cases, only the Object field should be null in the Json data returned by the backend, but in reality, there are cases where the String /list type is null.
- In Java, the null value overwrites the default value. When used, the null value in the get method is fine.
- However, in Kotlin, if the field is declared as non-null, the non-null field is given a null value after serialization with GSON, although this is obviously not expected because the null security check is done in the compiler and no exception is reported.
- When the JSON value corresponding to the non-empty field is NULL, a JsonDataException is thrown. When the corresponding key does not exist, the same processing is done
All of this logic seems reasonable, but in real development, there are unexpected cases of null values, and it is unlikely that we will declare all fields as nullable, so it is probably a good idea to resolve the null values in Json to default values.
Gson custom parsing replaces null value
Gson custom parsing uses a TypeAdapterFactory or a single TypeAdapter. The following example replaces the null value in Json with an empty String and an empty List for fields declared as String and List with a custom parser
class GsonDefaultAdapterFactory: TypeAdapterFactory {
override fun <T : Any> create(gson: Gson, type: TypeToken<T>): TypeAdapter<T>? {
if (type.type == String::class.java) {
return createStringAdapter()
}
if (type.rawType == List::class.java || type.rawType == Collection::class.java) {
return createCollectionAdapter(type, gson)
}
return null
}
/** * replace null with an empty List */
private fun <T : Any> createCollectionAdapter(
type: TypeToken<T>,
gson: Gson
): TypeAdapter<T>? {
val rawType = type.rawType
if(! Collection::class.java.isAssignableFrom(rawType)) {
return null
}
val elementType: Type = `$Gson$Types`.getCollectionElementType(type.type, rawType)
val elementTypeAdapter: TypeAdapter<Any> =
gson.getAdapter(TypeToken.get(elementType)) as TypeAdapter<Any>
return object : TypeAdapter<Collection<Any>>() {
override fun write(writer: JsonWriter, value: Collection<Any>?{ writer.beginArray() value? .forEach { elementTypeAdapter.write(writer, it) } writer.endArray() }override fun read(reader: JsonReader): Collection<Any> {
val list = mutableListOf<Any>()
// Replace null with an empty list
if (reader.peek() == JsonToken.NULL) {
reader.nextNull()
return list
}
reader.beginArray()
while (reader.hasNext()) {
val element = elementTypeAdapter.read(reader)
list.add(element)
}
reader.endArray()
return list
}
} as TypeAdapter<T>
}
/** * Replace null with an empty string */
private fun <T : Any> createStringAdapter(a): TypeAdapter<T> {
return object : TypeAdapter<String>() {
override fun write(writer: JsonWriter, value: String?). {
if (value == null) {
writer.value("")}else {
writer.value(value)
}
}
override fun read(reader: JsonReader): String {
// Replace null with ""
if (reader.peek() == JsonToken.NULL) {
reader.nextNull()
return ""
}
return reader.nextString()
}
} as TypeAdapter<T>
}
}
Copy the codeThe test code
val gson: Gson = GsonBuilder()
.registerTypeAdapterFactory(GsonDefaultAdapterFactory())
.create()
data class Person(
val name: String,
val friends: List<Person>
)
fun testGsonNullValue(a) {
// There must be an age field. Moshi will not allow null age to be used for null security
val json = """{"name":null, "friends":null}"""
val p1 = gson.fromJson(json, Person::class.java)
println("gson parse json: $p1")}Copy the codeGson Parse JSON: Person(name=, friends=[]) meets expectations
Moshi Custom parsing replaces null value
Custom parsing in Moshi through JsonAdapter or JsonAdapterFactory, here I directly take Moshi standard JsonAdapter to modify
public final class MoshiDefaultAdapterFactory {
private MoshiDefaultAdapterFactory(a) {}public static final JsonAdapter.Factory FACTORY = new JsonAdapter.Factory() {
@Override
publicJsonAdapter<? > create( Type type, Set<? extends Annotation> annotations, Moshi moshi) {if(! annotations.isEmpty())return null; .if (type == String.class) return STRING_JSON_ADAPTER;
return null; }}; .static final JsonAdapter<String> STRING_JSON_ADAPTER = new JsonAdapter<String>() {
@Override
public String fromJson(JsonReader reader) throws IOException {
// Replace null with ""
if(reader.peek() ! = JsonReader.Token.NULL) {return reader.nextString();
}
reader.nextNull();
return "";
}
@Override
public void toJson(JsonWriter writer, String value) throws IOException {
writer.value(value);
}
@Override
public String toString(a) {
return "JsonAdapter(String)"; }}; }Copy the codeReplace null with an empty List Adapter
/ * * *@author greensun
* @date 2021/6/2
* @descNull converted to empty collection changed from {@linkCom. Squareup. Moshi. CollectionJsonAdapter} * < p > * if field declarations for the Collection, json value is null, the kotlin under the declared type is not empty cases are abnormal, Here an empty Collection is filled with */
public abstract class MoshiDefaultCollectionJsonAdapter<C extends Collection<T>, T> extends JsonAdapter<C> {
public static final Factory FACTORY =
new Factory() {
@Override
publicJsonAdapter<? > create( Type type, Set<? extends Annotation> annotations, Moshi moshi) { Class<? > rawType = Types.getRawType(type);if(! annotations.isEmpty())return null;
if (rawType == List.class || rawType == Collection.class) {
return newArrayListAdapter(type, moshi);
} else if (rawType == Set.class) {
return newLinkedHashSetAdapter(type, moshi);
}
return null; }};private final JsonAdapter<T> elementAdapter;
private MoshiDefaultCollectionJsonAdapter(JsonAdapter<T> elementAdapter) {
this.elementAdapter = elementAdapter;
}
static <T> JsonAdapter<Collection<T>> newArrayListAdapter(Type type, Moshi moshi) {
Type elementType = Types.collectionElementType(type, Collection.class);
JsonAdapter<T> elementAdapter = moshi.adapter(elementType);
return new MoshiDefaultCollectionJsonAdapter<Collection<T>, T>(elementAdapter) {
@Override
Collection<T> newCollection(a) {
return newArrayList<>(); }}; }static <T> JsonAdapter<Set<T>> newLinkedHashSetAdapter(Type type, Moshi moshi) {
Type elementType = Types.collectionElementType(type, Collection.class);
JsonAdapter<T> elementAdapter = moshi.adapter(elementType);
return new MoshiDefaultCollectionJsonAdapter<Set<T>, T>(elementAdapter) {
@Override
Set<T> newCollection(a) {
return newLinkedHashSet<>(); }}; }abstract C newCollection(a);
@Override
public C fromJson(JsonReader reader) throws IOException {
C result = newCollection();
if (reader.peek() == JsonReader.Token.NULL) {
// Null returns an empty collection
reader.nextNull();
return result;
}
reader.beginArray();
while (reader.hasNext()) {
result.add(elementAdapter.fromJson(reader));
}
reader.endArray();
return result;
}
@Override
public void toJson(JsonWriter writer, C value) throws IOException {
writer.beginArray();
for (T element : value) {
elementAdapter.toJson(writer, element);
}
writer.endArray();
}
@Override
public String toString(a) {
return elementAdapter + ".collection()"; }}Copy the codeThe test code
val moshi: Moshi = Moshi.Builder()
.add(KotlinJsonAdapterFactory())
.add(MoshiDefaultAdapterFactory.FACTORY)
.add(MoshiDefaultCollectionJsonAdapter.FACTORY)
.build()
@JsonClass(generateAdapter = true)
data class Person(
val name: String,
val friends: List<Person>
)
fun testMoshiNullValue(a) {
val json = """{"name":null, "friends":null}"""
val p2 = moshi.adapter(Person::class.java).fromJson(json)
println("moshi parse json: $p2")}Copy the codeMoshi Parse JSON: Person(name=, friends=[]), as expected
conclusion
Best practices:
- Assign default values to all defined fields using Gson + Custom parsing filters out unexpected null values
- With Moshi, custom parsing filters out unexpected null values