Fastjson fatal defect

preface

Over the weekend, a bug in the same public account fastjson was discovered by several tech bloggers, and this time it caused the service to crash! A serious Bug in FastJson has emerged shortly after the previous Bug incident. At present, many projects use FastJson to convert objects to JSON data, and need to update the version and redeploy, which can be said to be time-consuming and laborious. At the same time, it also brings me new thinking: facing a large number of powerful open source libraries, we can not blindly introduce into the project, any unstable factor in many open source frameworks is enough to destroy a project. During the weekend, I will learn the excellent open source framework Gson, which also has the function of converting object JSON to each other, at home. I plan to use Gson gradually in future projects where FastJson is used. I will record the learning summary, hoping it will be helpful to my friends.

All the code fragments involved in this article are in the warehouse below, if you are interested, please refer to them:

https://github.com/wrcj12138aaa/gson-actions

Version support:

– JDK 8

– gson 2.8.5

– junit 5.5.1

– Lomok 1.18.8

Gson profile

Before introducing Gson, we can take a look at the description of Gson on the official wiki to see what it is.

Gson is a Java library that can be used to convert Java Objects into their JSON representation. It can also be used to Convert a JSON string to an Equivalent Java object.

As you can see from the description, Gson is a Java library for converting Java objects to and from JSON-formatted string data. It was initially widely used within Google on the Android platform and Java server. Since open source in 2008, It has become another widely used open source Framework for Google. Up to now (2019.09.08), it has more than 1W6 stars on GitHub. And Ali open source FastJson and so on.

In terms of usage, Gson provides a simple API fromJson/toJson to convert Java toJson, and can generate compact, readable JSON string output. It also supports complex object conversion and rich custom representation. Enough to meet most of our JSON data processing needs in daily development.

We usually to convert between the object and JSON string called the Serialization and Deserialization (Serialization/Deserialization). The process of converting an object to a JSON string is called serialization, and the process of converting a JSON string to an object is called deserialization.

Basic use of Gson

The com.google.gson.Gson object is also the key object of Gson framework. The public API provided by Gson framework has a variety of serialization and de-sequence methods.

There are two main ways to create Gson objects:

• usenewKeyword directly created:Gson gson = new Gson()
• Built from GsonBuilder objects:Gson gson = new GsonBuilder().create()

In general, Gson objects created in the above two ways behave the same for serialization and de-sequence operations, but the second way to build Gson objects allows additional behavior customization, such as formatting the output content of JSON strings, whether to serialize null values, and so on.

Java serialization

Serialization of simple objects

We can use the following example to see the different effects of serializing Java objects in the two ways described above:

public class ResultTest { @Test void test_serialization() { Gson gson = new Gson(); Result Result = new Result(200, "success ", null); String json = gson.toJson(result); System.out.println("json is " + json); Gson buildedGson = new GsonBuilder().setPrettyPrinting().serializeNulls().create(); String buildedJson = buildedGson.toJson(result); System.out.println("buildedJson is " + buildedJson); } class Result { private int code; private String message; private Object data; public Result(int code, String message, Object data) { this.code = code; this.message = message; this.data = data; }}}Copy the code

Run the test case and you can see the following log output on the console:

Gson objects can be seen from the results, the default behavior serialized object will be null field to ignore, and execute com. Google. Gson. GsonBuilder# serializeNulls methods will allow Gson object serialization null field; And normal serialized JSON string is compact format, save the string memory, using com. Google. Gson. GsonBuilder# setPrettyPrinting method after the final output of JSON string format is easier to read. Of course, in addition to these two methods, GsonBuilder provides a number of apis for customizing serialization and deserialization behavior, which we’ll cover later.

JosnObject generate JSON

In addition to the above method of converting the object of the custom class toJson, you can also use the JsonObject provided by the Gson framework to build a normal object, and then use the toJson method to generate a JSON string. Add the following test class to the original test class, and run the test class to view the result as follows

@Test void test_jsonObject_serialization() { Gson gson = new Gson(); JsonObject jsonObject = new JsonObject(); jsonObject.addProperty("code", 400); Jsonobject.addproperty ("message", "parameter error "); String toJson = gson.toJson(jsonObject); String exceptedJson = "{\" code \ ": 400, \" message \ ": \" parameter error \ "} "; Assertions.assertEquals(exceptedJson, toJson); //true }Copy the code

JsonObject uses addProperty(Value) to add String, Number, Boolean, Character. Because the internal is to call com. Google. Gson. JsonObject# add, and will value encapsulation became a JsonPrimitive object, and then save the internal variables set custom LinkedTreeMap members; If you need to add other objects to a JsonObject, you need to add a JsonElement object directly using the Add (String Property, JsonElement Value) method. JsonElement is an abstract class. Both JsonObject and JsonPrimitive inherit from JsonElement, so we end up using the new JsonObject object as an attribute object on the original JsonObject:

Gson gson = new Gson(); JsonObject jsonObject = new JsonObject(); / /... JsonObject nestJsonObject = new JsonObject(); nestJsonObject.addProperty("username", "one"); nestJsonObject.addProperty("score", 99); jsonObject.add("data", nestJsonObject); String toJson2 = gson.toJson(jsonObject); System.out.println(toJson2); / / {" code ": 400, the" message ":" parameter error ", "data" : {" username ":" one ", "score" : 99}}Copy the code

JSON deserialization

Deserialization of simple objects

Com.google.gson.gson #fromJson (String JSON, Class classOfT) Attempts to convert a JSON string to an object of the specified Class. If the conversion fails, a JsonSyntaxException is thrown. We can add a new test case to the original code and run it to see what happens:

@test void test_deserialization() {String json = "{\"code\":400,\"message\":\" parameter error \"}"; Result result = new Gson().fromJson(json, Result.class); Assertions.assertEquals(400, result.code); // true Assertions. AssertEquals (" parameter error ", result.message); // true }Copy the code

Deserialize Map

Instead of serializing JSON strings into custom Java objects, we can convert them into Map collections, which Gson provides and is easy to use:

@Test
void test_map() {
    String jsonString = "{'employee.name':'one','employee.salary':10}";
    Gson gson = new Gson();
    Map map = gson.fromJson(jsonString, Map.class);
    assertEquals(2, map.size());
        assertEquals("one", map.get("employee.name"));
    assertEquals(Double.class, map.get("employee.salary").getClass());
}Copy the code

It should be noted that the real type of Map object after transformation is not HashMap, but Gson custom collection LinkedTreeMap, which implements Map interface, stores key-value pairs, optimizations in addition and deletion implementation, and uses the order of storing key-value pairs as traversal order. So what’s put in first is iterated first. In addition, all numeric data in JSON strings will be converted to Double, while true/false data will be converted to Boolean, Specific judgment basis can refer to the com. Google. Gson. Internal. Bind. ObjectTypeAdapter# read method implementation.

Convert JSON to Array and List

Convert JSON Array

When we are converting JSON data to an array, we can convert it toJson data in a similar way as ordinary objects. Use fromJson to convert an array type to an array of the corresponding type.

@Test void test_array() { Gson gson = new Gson(); int[] ints = {1, 2, 3, 4, 5}; String[] strings = {"abc", "def", "ghi"}; String s = gson.toJson(ints); / / [1, 2, 3, 4, 5] assertEquals (" [1, 2, 3, 4, 5] ", s); // true String s1 = gson.toJson(strings); // ["abc", "def", "ghi"] assertEquals("[\"abc\",\"def\",\"ghi\"]", s1); String[] strings1 = gson.fromJson(s1, String[].class); assertEquals(strings.length, strings1.length); // true assertEquals(strings[0], strings1[0]); / / true int [] ints2 = gson. FromJson ([1, 2, 3, 4, 5], int [] class); assertEquals(1, ints2[0]); // true assertEquals(5, ints2[4]); // true }Copy the code

JSON conversion List

To convert List data to JSON data, use Gson the same way you would for Array data; Converting JSON data into a List object is slightly different. To convert a JSON array data into a List of custom classes, we write as follows:

@Test
public void givenJsonString_whenIncorrectDeserializing() {
    Gson gson = new Gson();
    String inputString = "[{\"id\":1,\"name\":\"one\"},{\"id\":2,\"name\":\"two\"}]";
    List<Person> outputList = gson.fromJson(inputString, List.class);
        outputList.get(0).getId();
}Copy the code

Unfortunately, running this code throws a ClassCastException, described as follows:

java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap cannot be cast to com.one.learn.Person
...Copy the code

From the above description, we can see that after executing fromJson, the deserialized List element is of type LinkedTreeMap, not Person. So when the ID attribute is accessed as a Person object, a ClassCastException is thrown. FromJson (String json, Type typeOfT) : fromJson(String json, Type typeOfT

@Test
public void givenJsonString_whenCorrectDeserializing_() {
  Gson gson = new Gson();
  String inputString = "[{\"id\":1,\"name\":\"one\"},{\"id\":2,\"name\":\"two\"}]";
  Type type = new TypeToken<List<Person>>(){}.getType();
  List<Person> outputList = gson.fromJson(inputString, type);
  int id = outputList.get(0).getId();
  assertEquals(1, id); // true
  assertEquals("one", outputList.get(0).getName()); // true
}Copy the code

The Type object in this method is obtained through the getType method of the TypeToken object and is the generic Type associated with the TypeToken object. TypeToken is a class introduced by Gson to support generics to solve the problem that Java cannot provide generic type representation. As the constructor of TypeToken is protected, it cannot be constructed directly. Use new TypeToken>() {}.getType().

Gson advanced usage

Once you’ve touched on the basic uses of Gson, let’s move on to other uses of Gson.

Deserialization of generic objects

In the last section, we briefly touched on Gson’s support for generics. To show how powerful it can be, let’s adjust the Result class above to accept generic parameters:

class Result<T> { private int code; private String message; private T data; public Result(int code, String message, T data) { this.code = code; this.message = message; this.data = data; }}Copy the code

Parse a JSON string with an embedded object into a Result object as follows:

@ Test void test_genric_object () {String json = "{\" code \ ": 200, \" message \ ": \ \" operation successful ", \ "data \" : {\ username \ "" : \"one\",\"avater\": \"image.jpg\"" + "}}"; Type type = new TypeToken<Result<User>>(){}.getType(); Result<User> result = new Gson().fromJson(json, type); Assertions.assertEquals(200, result.code); Assertions.assertEquals("one", result.data.getUsername()); Assertions.assertEquals("image.jpg", result.data.getAvater()); } class User { private String username; private String avater; public String getUsername() { return username; } public String getAvater() { return avater; }}Copy the code

The concrete generic type Result is obtained using a TypeToken object, passed in to the fromJson method and deserialized based on the corresponding type.

Custom serialization

If we want to perform special processing on some fields of Java objects, such as hiding the serialization of certain fields and formatting the data of fields, we can implement the JsonSerializer interface to customize the serialization logic. For example, if we need to handle the DateSerializer class in a specific format, we can declare the following:

class DateSerializer implements JsonSerializer<Date> { SimpleDateFormat dateTime = new SimpleDateFormat("yyyy-MM-dd"); @Override public JsonElement serialize(Date src, Type typeOfSrc, JsonSerializationContext context) { return new JsonPrimitive(dateTime.format(src)); }}Copy the code

Then, before building the Gson object, use GsonBuilder to register the DateSerializer instance as follows:

Gson gson = new GsonBuilder().registerTypeAdapter(Date.class, new DateSerializer()).create();
Copy the code

This way, whenever a field of type Date is serialized, the custom serialize method outputs the Date in YYYY-MM-DD format, as shown in the following example:

@Test
void test_dateSerializer() {
  MyObject myObject = new MyObject(new Date(), "one");
  Gson gson = new GsonBuilder().registerTypeAdapter(Date.class, new DateSerializer()).create();
  String json = gson.toJson(myObject);
  String exceptedJson = "{\"date\":\"2019-09-08\",\"name\":\"one\"}";
  Assertions.assertEquals(exceptedJson, json); // true
}

class MyObject {
    private Date date;
    private String name;

    public MyObject(Date date, String name) {
        this.date = date;
        this.name = name;
    }

    public MyObject() {
    }
}Copy the code

Custom deserialization

Similar to the implementation of custom deserialization, if you want to customize deserialization logic, you need to also implement a JsonDeserializer interface for the implementation of custom deserialization logic. Let’s say we have a JSON string with the contents {“CODE”: 400, “MESSAGE”: “Parameter error “}, need to be deserialized into the Result object mentioned above, because the field name is different, in order to achieve the corresponding conversion, need to define the ResultDeserializer class, the concrete implementation is as follows:

class ResultDeserializer implements JsonDeserializer<Result> { @Override public Result deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException { JsonObject object = json.getAsJsonObject(); Result<Object> result = new Result<>(object.getAsJsonPrimitive("CODE").getAsInt(),object.getAsJsonPrimitive("MESSAGE").getAsString(), null); return result; }}Copy the code

The next step is to use GsonBuilder to register ResultDeserializer instance and generate the corresponding Gson object, which will take effect during deserialization operation:

@test void test_resultDeserializer() {//language=JSON String JSON = "{\"CODE\ : 400,\"MESSAGE\": \" parameter error \"}"; Gson gson = new GsonBuilder().registerTypeAdapter(Result.class, new ResultDeserializer()) .create(); Result result = gson.fromJson(json, Result.class); Assertions.assertEquals(400, result.code); // true Assertions. AssertEquals (" parameter error ", result.message); // true }Copy the code

Gson common annotations

In addition to the apis that Gson provides for developers to use, there are also a number of special annotations available. Here are some of the most commonly used annotations in Gson.

@Expose

This annotation can only be used on fields to indicate whether the corresponding field will be exposed during serialization or deserialization. There are two properties serialize and deserialize, both of which are true by default. When a field is annotated @expose (serialize = true, deserialize = false), it indicates that the field is only visible at serialization time and that assignment is ignored during deserialization. Need extra attention is that @ Expose annotation is only valid at the time of constructing Gson GsonBuilder way, and must be called before build excludeFieldsWithoutExposeAnnotation method, Otherwise, parsing has no effect on the annotated field. Here’s an example:

@Test
void test_expose() {
    MySubClass subclass = new MySubClass(42L, "the answer", "Verbose field not to serialize");
    MyClass source = new MyClass(1L, "foo", "bar", subclass);
    Gson gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().create();
    String s = gson.toJson(source);
    System.out.println(s);
        // {"name":"foo","subclass":{"id":42,"description":"the answer","otherVerboseInfo":"Verbose field not to serialize"}}
}

@Data
@AllArgsConstructor
class MyClass {
    private long id;
    @Expose(serialize = false, deserialize = true)
    private String name;
    private transient String other;
    @Expose
    private MySubClass subclass;
}

@Data
@AllArgsConstructor
class MySubClass {
    @Expose
    private long id;
    @Expose
    private String description;
    @Expose
    private String otherVerboseInfo;
}Copy the code

Transient keyword fields in Gson are not serialized and deserialized by default, which is consistent with Java native serialization and deserialization operations.

@Since

This annotation is used to mark the version of the corresponding field or type, allowing Gson to specify the version number for serialization and deserialization operations. This annotation is useful when JSON data on a Web service has multiple versions of fields in its entity class.

Similarly, this annotation is valid only for Gson objects built using GsonBuilder and when setVersion is used to specify the version number, only the corresponding version fields in the object will be parsed. Here is an example:

public class VersioningSupportTest { @Test void test() { VersionedClass versionedObject = new VersionedClass(); Gson Gson = new GsonBuilder().setVersion(1.0).create(); String jsonOutput = gson.toJson(versionedObject); System.out.println(jsonOutput); // {"newField":"new","field":"old"}}} class VersionedClass {@since (1.1) private final String newerField; @since (1.0) private Final String newField; private final String field; public VersionedClass() { this.newerField = "newer"; this.newField = "new"; this.field = "old"; }}Copy the code

@SerializedName

This annotation is simple and useful to use. @serializedName specifies the name of the member field to be serialized and deserialized, so that we can adjust the inconsistency between JSON data and the corresponding entity class field name. Here is how to use it:

public class JSONFieldNamingSupportTest {
    private class SomeObject {
        @SerializedName("custom_naming")
        private final String someField;
        private final String someOtherField;

        public SomeObject(String a, String b) {
            this.someField = a;
            this.someOtherField = b;
        }
    }

    @Test
    void test() {
        SomeObject someObject = new SomeObject("first", "second");
        String jsonRepresentation = gson.toJson(someObject);
        System.out.println(jsonRepresentation);
          // {"custom_naming":"first","someOtherField":"second"}
        SomeObject someObject1 = gson.fromJson(jsonRepresentation, SomeObject.class);
        System.out.println(someObject1);
          // SomeObject{someField='first', someOtherField='second'}
    }
}Copy the code

@JsonAdapter

Is different from the above comments, @ JsonAdapter role only in class, main effect is to replace GsonBuilder. RegisterTypeAdapter method of implementation, You can use @jsonAdapter (aClass. Class) to specify the JsonDeserializer object or JsonSerializer object. And priority than GsonBuilder. RegisterTypeAdapter higher priority, because just registerTypeAdapter method reduces execution to annotate, here no longer presentation, You can see this effect by using it directly on the Result class from the custom deserialization section above.

conclusion

This paper mainly summarizes the use of serialization and anti-sequence operations of Gson framework, and introduces a variety of features of Gson usage, hoping to help those who feel headache in dealing with JSON data.

The resources

• https://github.com/google/gson/blob/master/UserGuide.md
• https://www.jianshu.com/p/e740196225a4
• https://juejin.cn/post/6844903577098387464
• https://www.baeldung.com/gson-deserialization-guide
• https://www.baeldung.com/gson-string-to-jsonobject

● Spring-boot-starter-GRPC performance test and selection of different serialization methods

● Spring Boot 2 integrates log4J2 logging framework

● Java interview key points summary set of core reference answers

● Java interview key points summary of the framework of reference answers

● How to protect your password

● Spring Boot RabbitMQ – Priority queue

● Visibility, orderliness, happens-before

● Get started with JUnit 5 for Java unit Testing

Some Spring Boot advanced interview questions

● Spring Boot 2

● Core introduction to Prometheus, the next generation application monitoring indicator collector

● The @async annotation in Spring implements asynchronous calls to “methods”

● To learn concurrent programming, a thorough understanding of these three core is the key

● Cache abstraction layer Spring cache actual operation

● Brief on design pattern principles

● There are three sources of concurrency bugs, so keep your eyes open

Responsibility chain design mode explanation

● Java Web interview questions about Spring MVC, do not collect blood loss!

● Kafka is used in Micronaut microservices

● Besides Zuul and Spring Cloud Gateway, there are more excellent microservices gateways

● RSocket — an alternative to Http

● Numbers in Java and how to determine if a string is a number

● You need these to learn Spring Security and Apache Shiro well

This article is published by OpenWrite!