Introduction to the

Every developer, whether front-end or back-end, uses JSON regularly in their daily work. JSON is a very simple data interchange format. Compared to XML, it is flexible, lightweight, and easy to use. JSON is also recommended by RESTful apis. Sometimes we just want to read certain fields in JSON. If you parse by hand, reading layer by layer, this becomes tedious. Especially if there is a deep level of nesting. Today we introduce Gojsonq. It makes it easy to manipulate JSON.

Quick to use

First installation:

$ go get github.com/thedevsaddam/gojsonq
Copy the code

After using:

package main

import (
  "fmt"

  "github.com/thedevsaddam/gojsonq"
)

func main(a) {
  content := `{ "user": { "name": "dj", "age": 18, "address": { "provice": "shanghai", "district": "xuhui" }, "hobbies":["chess", "programming", "game"] } }`

  gq := gojsonq.New().FromString(content)
  district := gq.Find("user.address.district")
  fmt.Println(district)

  gq.Reset()

  hobby := gq.Find("user.hobbies.[0]")
  fmt.Println(hobby)
}
Copy the code

The operation is very simple:

  • First callgojsonq.New()To create aJSONQThe object;
  • You can then use methods of this type to query properties.

In this code we read district and the first element of the hobbies array. Layers are separated by periods (.). If it is an array, the element with index index is read through.[index] after the property field. This allows for very flexible reads.

Notice one detail: after the query, we manually called the Reset() method once. Because the JSONQ object internally records the current node when it calls the Find method, the next query starts with the node it last looked up. Jq. That is to say if we commented the Reset (), the second a Find () method actually Find is user address. District. User. Hobbies. [0], nature will return nil. In addition, GojsonQ provides another way. If you want to save some state information for the current query, you can call JSONQ’s Copy method to return an object in its original state that shares the underlying JSON string and the parsed object. The above gq.reset () can be replaced with the following line:

gpCopy := gp.Copy()
Copy the code

You can then use gpCopy to query hobbies.

This is one of the features of the GojsonQ library, but it’s also one of the most frustrating things for beginners to watch out for. In fact, many of the methods provided by JSONQ change the current node, as we’ll see more clearly later.

The data source

In addition to loading from strings, JSONQ also allows reading from files and IO.Reader. Using the JSONQ object’s File and Reader methods, respectively:

func main(a) {
  gq := gojsonq.New().File("./data.json")

  fmt.Println(gq.Find("items.[1].price"))}Copy the code

The effect is the same as the following program:

func main(a) {
  file, err := os.OpenFile("./data.json", os.O_RDONLY, 0666)
  iferr ! =nil {
    log.Fatal(err)
  }

  gq := gojsonq.New().Reader(file)

  fmt.Println(gq.Find("items.[1].price"))}Copy the code

For later demonstration purposes, I constructed a data.json file:

{
  "name": "shopping cart"."description": "List of items in your cart"."prices": ["2400"."2100"."1200"."400.87"."89.90"."150.10"]."items": [{"id": 1."name": "Apple"."count": 2."price": 12
    },
    {
      "id": 2."name": "Notebook"."count": 10."price": 3
    },
    {
      "id": 3."name": "Pencil"."count": 5."price": 1
    },
    {
      "id": 4."name": "Camera"."count": 1."price": 1750
    },
    {
      "id": null."name": "Invalid Item"."count": 1."price": 12000}}]Copy the code

Advanced query

Gojsonq is unique in that it can do conditional queries like SQL, you can choose which fields to return, and you can do some aggregate statistics.

Field mapping

If you only care about a few fields in an object, you can use Select to specify which fields to return and not return the rest:

func main(a) {
  r := gojsonq.New().File("./data.json").From("items").Select("id"."name").Get()
  data, _ := json.MarshalIndent(r, ""."")
  fmt.Println(string(data))
}
Copy the code

Only id and name fields are output:

$ go run main.go
[
  {
    "id": 1,
    "name": "Apple"
  },
  {
    "id": 2,
    "name": "Notebook"
  },
  {
    "id": 3,
    "name": "Pencil"
  },
  {
    "id": 4,
    "name": "Camera"
  },
  {
    "id": null,
    "name": "Invalid Item"
  }
]
Copy the code

To make it more intuitive, I’ve embellished the output with json.marshalindent ().

Select id,name From items…

Here’s a look at the From method, which moves the current node to the specified location. It was also stated that the current node position is recorded. For example, in the above code we first move the current node to items, and the subsequent queries and aggregations are directed against this array. The Find method actually calls From inside:

// src/github.com/thedevsaddam/gojsonq/jsonq.go
func (j *JSONQ) Find(path string) interface{} {
  return j.From(path).Get()
}

func (j *JSONQ) From(node string) *JSONQ {
  j.node = node
  v, err := getNestedValue(j.jsonContent, node, j.option.separator)
  iferr ! =nil {
    j.addError(err)
  }
  // ============= Note this line and remember the current node location
  j.jsonContent = v
  return j
}
Copy the code

Finally, you must call Get(), which executes the query after all the conditions are combined and returns the result.

Conditions of the query

With Select and From, how can there be no Where? Gojsonq provides so many Where methods that we can only look at a few.

The first is Where(key, op, val), which is a generic Where condition that indicates whether key and val satisfy the OP relationship. Op has nearly 20 built-in versions and supports customization. For example, = equals,! = for unequal, startsWith for whether val prefixes the key field, and so on;

Many other conditions are specific to Where: having (key, val) is equivalent to Where(key, “in”, val), WhereStartsWith(key, val) is equivalent to Where(key, “startsWith”, val).

By default, all Where conditions are joined with And. We can use OrWhere to make it join with Or:

func main(a) {
  gq := gojsonq.New().File("./data.json")

  r := gq.From("items").Select("id"."name").
    Where("id"."=".1).OrWhere("id"."=".2).Get()
  fmt.Println(r)

  gq.Reset()

  r = gq.From("items").Select("id"."name"."count").
    Where("count".">".1).Where("price"."<".100).Get()
  fmt.Println(r)
}
Copy the code

The first query above looks for the record with id 1 or 2. The second query looks for records where count is greater than 1 and price is less than 100.

Specifies the offset and number of returned entries

Sometimes we want to paginate the display, returning the first three items on the first query and the next three records on the second query. We can use the Offset and Limit methods of the JSONQ object to specify the Offset and the number of items to return:

func main(a) {
  gq := gojsonq.New().File("./data.json")

  r1 := gq.From("items").Select("id"."name").Offset(0).Limit(3).Get()
  fmt.Println("First Page:", r1)

  gq.Reset()

  r2 := gq.From("items").Select("id"."name").Offset(3).Limit(3).Get()
  fmt.Println("Second Page:", r2)
}
Copy the code

To see the results:

$ go run main.go
First Page: [map[id:1 name:Apple] map[id:2 name:Notebook] map[id:3 name:Pencil]]
Second Page: [map[id:4 name:Camera] map[id:<nil> name:Invalid Item]]
Copy the code

Aggregate statistics

We can also do simple statistics on some fields, calculate sum, average, maximum, minimum, etc. :

func main(a) {
  gq := gojsonq.New().File("./data.json").From("items")

  fmt.Println("Total Count:", gq.Sum("count"))
  fmt.Println("Min Price:", gq.Min("price"))
  fmt.Println("Max Price:", gq.Max("price"))
  fmt.Println("Avg Price:", gq.Avg("price"))}Copy the code

It counts the total quantity, lowest price, highest price and average price of goods.

None of the methods of the aggregate statistics class changes the pointing of the current node, soJSONQObjects can be reused!

You can also group and sort data:

func main(a) {
  gq := gojsonq.New().File("./data.json")

  fmt.Println(gq.From("items").GroupBy("price").Get())
  gq.Reset()
  fmt.Println(gq.From("items").SortBy("price"."desc").Get())
}
Copy the code

Other formats

By default, GoJsonQ parses data in JSON format. We can also set up other format parsers so that GoJsonQ can handle data in other formats:

func main(a) {
  jq := gojsonq.New(gojsonq.SetDecoder(&yamlDecoder{})).File("./data.yaml")
  jq.From("items").Where("price"."< =".500)
  fmt.Printf("%v\n", jq.First())
}

type yamlDecoder struct{}func (i *yamlDecoder) Decode(data []byte, v interface{}) error {
  bb, err := yaml.YAMLToJSON(data)
  iferr ! =nil {
    return err
  }
  return json.Unmarshal(bb, &v)
}
Copy the code

The above code uses the YAML library and requires additional installation:

$ go get github.com/ghodss/yaml
Copy the code

Any parser that implements gojsonq.Decoder can be set to GoJsonQ, so that any format of processing can be implemented:

// src/github.com/thedevsaddam/gojsonq/decoder.go
type Decoder interface {
  Decode(data []byte, v interface{}) error
}
Copy the code

conclusion

Gojsonq also has some advanced features, such as customizing Where operation types, taking the first, last, NTH value, and so on. Interested in your own research ~

If you find a fun and useful Go library, please Go to GitHub and submit issue😄

reference

  1. Gojsonq GitHub:github.com/thedevsadda…
  2. GitHub: github.com/darjun/go-d…

I

My blog

Welcome to follow my wechat public account [GoUpUp], learn together, progress together ~

This article is published by OpenWrite!