This article is a memento of the hair I scratched out during those hours by documenting the holes I stepped in while doing YAML parsing.

Scene reappearance

Yesterday I found a problem with parsing a YAML file I had changed earlier: I needed to read from a YAML configuration file and parse into a struct. However, some parameters are not resolved and are empty. Take a look at the code (note: the code below is abstracted and simplified from my real problem) :

My YAML configuration file:

kind: PersonalInfo
name: he
age: 18
Copy the code

Structure definition for receive configuration:

type Config struct {
  TypeMeta `json:",inline"`
  Name     string `json:"name"`
  Words    string `json:"words"` 
}

type TypeMeta struct {
   Kind string `json:"kind,omitempty"` 
}
Copy the code

I started by using the Go-YAMl package directly.

func TestParseYaml(t *testing.T) {
   ymlFilePath := "conf.yaml"    
   ymlFile, err := os.Open(ymlFilePath)
   iferr ! =nil {
      t.Errorf("open yaml file(%s) failed: %v", ymlFilePath, err)
      return
  }
   defer ymlFile.Close()

   ymlContent, err := ioutil.ReadAll(ymlFile)
   iferr ! =nil {
      t.Errorf("read yaml file(%s) failed: %v", ymlFilePath, err)
      return
  }

  var config Config
  err = yaml.Unmarshal(ymlContent, &config)
   iferr ! =nil {
      t.Errorf("yaml format error: %v", err)
      return
  }

   t.Logf("%#v", config)
}
Copy the code

The output is as follows:

{TypeMeta:{Kind:} Name:he Age:18}
Copy the code

Looking at the result of this analysis, I can’t help thinking — where is my kind?? Who ate my kind??

Cause analysis,

I tried to modify my struct, tested it several times, and found that all the data that was not parsed was in anonymous variables.

But if my struct parses a JSON string, all fields, including fields in anonymous variables, will parse properly.

What is it that affects my ability to parse yamL data?

After a bunch of data analysis, I finally found hua point — struct label has been ignored by me!

In my struct, only THE JSON label is added, and there is no yamL related label. Therefore, in the process of YAML conversion, all fields are not labeled. An anonymous member without an inline tag is ignored during parsing.

Modify the validation

Based on the above analysis, I modified my struct type definition to add yamL’s inline tag to anonymous variables:

type Config struct {
  TypeMeta `json:",inline" yaml:",inline"`
  Name     string `json:"name"`
  Age      int `json:"age"` 
}

type TypeMeta struct {
   Kind string `json:"kind,omitempty"` 
}
Copy the code

Executing my parsing function again gives me the following result:

{TypeMeta:{Kind:PersonalInfo} Name:he Age:18}
Copy the code

Tears, I finally got the right result!

Final plan

While adding yamL tags to structs gives you the desired output, looking at our code, tags to structs are almost always JSON, and you don’t label YAML individually. If I were to label the struct I’m using separately with yamL, it would be a little out of style. And structs are almost never tagged with yamL, so the next time someone else adds an anonymous member and forgets to tag with yamL, it’s going to be pretty cool.

So, after some deliberation, I decided to first convert the YAML content I read from the configuration file to JSON, and then use JSON’s unmarshal method to convert json to struct.

The final method of parsing YAML is as follows:

type Config struct {
  TypeMeta `json:",inline"`
  Name     string `json:"name"`
  Age      int `json:"age"` 
}

type TypeMeta struct {
   Kind string `json:"kind,omitempty"` 
}


func TestParseYaml(t *testing.T) {
   ymlFilePath := "conf.yaml"    
   ymlFile, err := os.Open(ymlFilePath)
   iferr ! =nil {
      t.Errorf("open yaml file(%s) failed: %v", ymlFilePath, err)
      return
  }
   defer ymlFile.Close()
   ymlContent, err := ioutil.ReadAll(ymlFile)
   iferr ! =nil {
      t.Errorf("read yaml file(%s) failed: %v", ymlFilePath, err)
      return
  }
   data, err := yaml.ToJSON(ymlContent)

   var config Config
  err = json.Unmarshal(data, &config)
   iferr ! =nil {
      t.Errorf("yaml format error: %v", err)
      return
  }

   t.Logf("%+v", config)
}
Copy the code

Output result:

{TypeMeta:{Kind:PersonalInfo} Name:he Age:18}Copy the code