Marshal and Unmarshal

Introduction to the

Json(Javascript Object Nanotation) is a data exchange format commonly used for data transfer at the front and back ends. Either side converts the data into a JSON string, and the other side parses the string into a corresponding data structure, such as a string, strcut object, etc.

implementation

Json Marshal: Encodes data into Json strings

type Stu struct {
    Name  string `json:"name"`
    Age   int
    HIgh  bool
    sex   string
    Class *Class `json:"class"`
}

type Class struct {
    Name  string
    Grade int
}

func main(a) {
    // Instantiate a data structure to generate a JSON string
    stu := Stu{
        Name: "Zhang",
        Age:  18,
        HIgh: true,
        sex:  "Male",}// Pointer variable
    cla := new(Class)
    cla.Name = "Class 1"
    cla.Grade = 3
    stu.Class=cla

    //Marshal failed when err! =nil
    jsonStu, err := json.Marshal(stu)
    iferr ! =nil {
        fmt.Println("Error generating JSON string")}//jsonStu is of []byte type, converted to string for easy viewing
    fmt.Println(string(jsonStu))
}
Copy the code

Results:

{"name":"Zhang"."Age": 18."HIgh":true."class": {"Name":"Class 1"."Grade": 3}}Copy the code

Why can’t a named initial lowercase be exported? Answer: Because Golang syntax dictates that lowercase properties are private and uppercase properties are public

You can see from the results

• Any exportable member (variable starts with a capital letter) can be converted to JSON. The member variable sex cannot be converted to JSON because it is not exportable.

• If the variable has a JSON label, such as json:” Name “next to Name, then the converted JSON key uses the label” Name “. Otherwise, the variable Name is used as the key, such as “Age”, “HIgh”.

• Bool is also a value that can be converted directly to JSON. Channels, complex, and functions cannot be encoded as JSON strings. Of course, the circular data structure didn’t work either, causing Marshal to fall into an infinite loop.

• Pointer variables that are automatically converted to the value they point to when encoded, such as CLA variables. (Of course, the Stu struct member Class has the same effect if it is of Class struct type. But Pointers are faster and save memory.

Finally, it’s important to note that json is a pure string when encoded as a string.

The above member variables are all known types and can only receive specified types. For example, string Name can only assign string data. But sometimes for the sake of generality, or for simplicity of code, we want to have a type that accepts various types of data and encodes json. This uses the interface{} type.

The interface{} type is an empty interface, that is, an interface with no methods. Each type of GO implements this interface. Therefore, any other type of data can be assigned to the interface{} type.

type Stu struct {
    Name  interface{} `json:"name"`
    Age   interface{}
    HIgh  interface{}
    sex   interface{}
    Class interface{} `json:"class"`
}

type Class struct {
    Name  string
    Grade int
}

func main(a) {
    // Same as the previous example. }Copy the code

Results:

{"name":"Zhang"."Age": 18."HIgh":true."class": {"Name":"Class 1"."Grade": 3}}Copy the code

As you can see from the result, any string, int, bool, pointer, etc., can be assigned to interface{} and coded normally, as in the previous example.

Add: In real projects, data structures encoded as JSON strings are often of the sliced type. A slice of type []StuRead is defined below

// Get it right

// Method 1: only declare, no memory allocation
var stus1 []*StuRead

// Method 2: Allocate memory with initial value 0
stus2 := make([]*StuRead,0)

// Error
// New () can only instantiate a struct object, whereas []StuRead is a slice, not an object
stus := new([]StuRead) stu1 := StuRead{member assignment... } stu2 := StuRead{member assignment... }// Slices created by methods 1 and 2 can successfully append data
// Mode 2 should allocate 0 length, append will automatically increase. If the initial length is specified, the value will not automatically increase when the length is insufficient, resulting in data loss
stus1 := appen(stus1,stu1,stu2)
stus2 := appen(stus2,stu1,stu2)

// Successful encoding
json1,_ := json.Marshal(stus1)
json2,_ := json.Marshal(stus2)
Copy the code

When decoding, define the corresponding slice to accept

Json Unmarshal: Decodes the Json string into the appropriate data structure

Let’s decode the above example

type StuRead struct {
    Name  interface{} `json:"name"`
    Age   interface{}
    HIgh  interface{}
    sex   interface{}
    Class interface{} `json:"class"`
    Test  interface{}}type Class struct {
    Name  string
    Grade int
}

func main(a) {
    // The "quotes" in the json character must be escaped with \, otherwise the compilation fails
    //json string is the same as above, but the key is resized and the sex data is added
    data:="{\" name \ ": \" zhang SAN \ ", \ "Age \" : 18, \ "high \" : true, \ "sex \" : \ "m \", \ "CLASS \" : {\ "name \" : \ \ "" 1 CLASS, \" GradE \ ": 3}}"
    str:=[]byte(data)

    //1. The first argument to Unmarshal is a JSON string, and the second argument is the data structure to accept JSON parsing.
    // The second argument must be a pointer, otherwise parsed data cannot be received, such as stu is still empty StuRead{}
    Stu :=new(StuRead)
    stu:=StuRead{}
    err:=json.Unmarshal(str,&stu)

    // Parsing failure will result in error, such as json string format is incorrect, missing ", missing}, etc.
    iferr! =nil{
        fmt.Println(err)
    }

    fmt.Println(stu)
}
Copy the code

Results:

{zhang SAN 18true<nil> map[GradE: 1] <nil>}Copy the code

Conclusion:

• Json string parsing requires a “receiver body” to accept parsed data, and the receiver must pass a pointer when Unmarshal occurs. Otherwise, no parsing error is reported, but the data cannot be assigned to the recipient body. For example, StuRead{} receive is used here.

• When parsing, the receiver can define itself. The key in the JSON string automatically finds the matching item in the receiver body for assignment. The matching rules are:

  • Find the JSON tag that is the same as key, and assign the corresponding variable (for example, Name) to the tag.
  • If there is no JSON tag, look for variables with the same name as key, such as Age, from top to bottom. Or a variable whose name is the same as key regardless of case. Such as HIgh, Class. The first match is assigned, and any subsequent matches are ignored. (Provided that the variable must be exportable, i.e., capitalized).

• Non-exportable variables cannot be parsed (e.g., the sex variable, whose value is nil even though there is k-v in the JSON string with key sex)

• When there is an item in the receiver that cannot be matched in the JSON string, parsing automatically ignores the item and retains its original value. For example Test, leave the null value nil.

• You’ll notice that the Class variable doesn’t seem to resolve as expected. This is because Class is a variable of type interface{}, and value in json string with key Class is a compound structure, not a simple type data that can be parsed directly (e.g., “zhang SAN”, 18, true, etc.). Therefore, when parsing, json automatically parses data with value as a composite structure into items of type MAP [string]interface{}, since the specific type of variable Class is not specified. That is, the struct Class object has nothing to do with the Class variable in StuRead, and therefore nothing to do with the json parsing.

Let’s take a look at the resolved types of these interface{} variables

func main(a) {
    // Same as the previous JSON parsed code. fmt.Println(stu)// Prints the variable type before json parsing
    err:=json.Unmarshal(str,&stu)
    fmt.Println("--------------json after parsing -----------")... fmt.Println(stu)// Prints the variable type after json parsing
}

// Print the variable type using reflection
func printType(stu *StuRead){
    nameType:=reflect.TypeOf(stu.Name)
    ageType:=reflect.TypeOf(stu.Age)
    highType:=reflect.TypeOf(stu.HIgh)
    sexType:=reflect.TypeOf(stu.sex)
    classType:=reflect.TypeOf(stu.Class)
    testType:=reflect.TypeOf(stu.Test)

    fmt.Println("nameType:",nameType)
    fmt.Println("ageType:",ageType)
    fmt.Println("highType:",highType)
    fmt.Println("sexType:",sexType)
    fmt.Println("classType:",classType)
    fmt.Println("testType:",testType)
}
Copy the code

Results:

nameType: <nil> ageType: <nil> highType: <nil> sexType: <nil> classType: <nil> testType: <nil> --------------json after parsing ----------- nameType: String ageType: float64 highType: bool sexType: <nil> classType: map[string]interface {} testType: <nil>Copy the code

You can see from the results

• Interface {} type variables are printed nil until json is parsed, except that there is no specific type. This is the nature of empty interfaces (interface{} types).

• After json parsing, value in the JSON string, as long as it is “simple data”, is assigned to the default type. For example, “Zhang SAN” is assigned to the Name variable as string. The number 18 corresponds to float64, and true corresponds to bool.

Simple data: indicates the data that cannot be parsed twice. For example, name and SAN can be parsed only once. “Compound data” : data such as “CLASS\” :{\ “naME\” :\ “1 CLASS\”,\ “GradE\” :3} can be parsed twice or even multiple times because its value is a separate PARsed JSON. That is, the first parse key is the value of CLASS, and the second parse key is the value of naME and GradE

For “compound data”, if the configuration item in the receiver is declared as interface{}, go will be resolved to map[string]interface{} by default. If we want to parse directly into a struct Class object, we can define the item corresponding to the receiving body as the struct type. As follows:

type StuRead struct{...// Common struct type
Class Class `json:"class"`
// Pointer type
Class *Class `json:"class"`
}
Copy the code

Stu prints the result

Class type: {Zhang SAN 18true<nil> {1 Class 3} <nil>} *Class type: {zhang SAN 18true <nil> 0xc42008a0c0 <nil>}
Copy the code

It can be seen that pass in the pointer type Class, the Class variables of stu is a pointer, we can be accessed directly by the pointer’s data, such as stu. Class. The Name/stu. Class. Grade

What if you don’t want to specify a Class variable as a specific type and still want to keep the interface{} type, but want that variable to be parsed into a struct Class object?

The demand is likely to exist

There is a way we can define this variable as json.rawmessage

type StuRead struct {
    Name  interface{}
    Age   interface{}
    HIgh  interface{}
    Class json.RawMessage `json:"class"` // Notice here
}

type Class struct {
    Name  string
    Grade int
}

func main(a) {
    data:="{\" name \ ": \" zhang SAN \ ", \ "Age \" : 18, \ "high \" : true, \ "sex \" : \ "m \", \ "CLASS \" : {\ "name \" : \ \ "" 1 CLASS, \" GradE \ ": 3}}"
    str:=[]byte(data)
    stu:=StuRead{}
    _:=json.Unmarshal(str,&stu)

    // Note here: quadratic parsing!
    cla:=new(Class)
    json.Unmarshal(stu.Class,cla)

    fmt.Println("stu:",stu)
    fmt.Println("string(stu.Class):".string(stu.Class))
    fmt.Println("class:",cla)
    printType(&stu) // Implement the function in the previous example
}
Copy the code

The results of

Stu: {Zhang SAN 18true [123 34 110 97 77 69 34 58 34 49 231 143 173 34 44 34 71 114 97 100 69 34 58 51 125]}
string(stu.Class): {"naME":"Class 1"."GradE":3} class: &{1 class 3} nameType: String ageType: float64 highType: bool classType: json.RawMessageCopy the code

You can see from the results

• In the receiver, a variable declared as jSON. RawMessage retains its original json value during JSON parsing, that is, it is not automatically parsed as map[string]interface{}. {” naME “:” Class 1 “, “GradE” :3}

• As you can also see from the printed type, the variable Class is of type json.rawmessage when the json is first parsed. At this point, we can do a secondary JSON parsing of the variable, since the value is still a separate and parsable complete JSON string. We just need to define a new receiver, such as json.unmarshal (stu.class, clA)