An overview of the

In the business system development, especially in the background management system, the data displayed in the list page comes from multiple data sources, and the list page needs to support pagination, how to solve it?

The problem

As shown in the figure above, data sources may come from different DB databases, from different API interfaces, or from a combination of DB and API.

I don’t have a good solution. If I receive such demand, I must first communicate with the demander to see if it is reasonable to pagination.

There are only two options:

  • Data is synchronized periodically, with the query data first summarized in one place and then query paging.
  • Paging in memory, the query data is first stored in memory, and then query paging.

If you paginate one data source, other fields are fetched from other data sources, which is easier to handle.

If multiple data sources are fused and then paging, use periodic data synchronization or in-memory paging.

Regular data synchronization scheme can be designed according to the actual situation of the synchronization frequency, as for synchronization to ES/MySQL/MongoDB you can decide.

For the in-memory paging scheme, I share two small methods below for your reference.

PHP method

= $data [0 = > [' name '= > name of "1", "age" = > "age 1]", 1 = > [' name' = > 2 "name", "age" = > "age 2"], 2 = > [' name '= > "name 3", 'age' = > "age 3", 3 = > [' name '= > 4 "name", "age" = > "age 4"], 4 = > [' name' = > "name 5", "age" = > "age 5]", 5 = > [' name '= > 6 "name", 'age' = > "age 6, 6 = > [' name '= > 7" name ", "age" = > "age 7"], 7 = > [' name' = > "name 8", "age" = > "age 8"], 8 = > [' name '= > "name nine", 'age' = > "age 9, 9 = > [' name '= >" name 10 ", "age" = > "age 10"],]; /** * array page * @param array $arrayData array data * @param int $page number of pages * @param int $pageSize Number of pages per page * @return array */ function arrayToPageData($arrayData = [], $page = 1, $pageSize = 10) { $arrayData = array_values((array)$arrayData); $pageData['list'] = array_slice($arrayData, ($page - 1) * $pageSize, $pageSize); $pageData['pagination']['total'] = count($arrayData); $pageData['pagination']['currentPage'] = $page; $pageData['pagination']['prePageCount'] = $pageSize; return $pageData; } echo json_encode(arrayToPageData($data, 2, 3));Copy the code

Output:

{" a list ": [{" name" : "name 4", "age" : "age 4"}, {" name ":" name 5 ", "age" : "age 5"}, {" name ":" name of 6 ", "age" : }], pagination: {"total": 10, "currentPage": 2, "prePageCount": 3}}Copy the code

Go way

package main import ( "encoding/json" "fmt" ) type User []struct { Name string `json:"name"` Age string `json:"age"` } type Pagination struct { Total int `json:"total"` CurrentPage int `json:"currentPage"` PrePageCount int `json:"prePageCount"` } type ListPageData struct { List User `json:"list"` Pagination Pagination `json:"pagination"` } Func main () {jsonStr: = ` [{" name ":" name 1 ", "age" : "age 1"}, {" name ":" name 2 ", "age" : "age 2"}, {" name ":" name 3 ", "age" : "Age 3"}, {" name ":" name 4 ", "age" : "age 4"}, {" name ":" name 5 ", "age" : "age 5"}, {" name ":" name of 6 ", "age" : "age 6"}, {" name ":" name 7 ", "age" : "Age 7"}, {" name ":" name of eight ", "age" : "age 8"}, {" name ":" name nine ", "age" : "age 9"}, {" name ":" name 10 ", "age" : }] 'var user user err := json.Unmarshal([]byte(jsonStr), &user) if err! = nil { fmt.Println(err.Error()) } page := 2 pageSize := 3 pageData := ArraySlice(user, page, pageSize) listPageData := ListPageData{} listPageData.List = pageData listPageData.Pagination.Total = len(user) listPageData.Pagination.CurrentPage = page listPageData.Pagination.PrePageCount = pageSize jsonData, _ := JsonEncode(listPageData) fmt.Println(jsonData) } func JsonEncode(v interface{}) (string, error) { bytes, err := json.Marshal(v) if err ! = nil { return "", err } return string(bytes), nil } func ArraySlice(u User, page int, pageSize int) User { offset := (page - 1) * pageSize if offset > int(len(u)) { panic("offset: the offset is less than the length of u") } end := offset + pageSize if end < int(len(u)) { return u[offset:end] } return u[offset:] }Copy the code

Output:

{" a list ": [{" name" : "name 4", "age" : "age 4"}, {" name ":" name 5 ", "age" : "age 5"}, {" name ":" name of 6 ", "age" : }], pagination: {"total": 10, "currentPage": 2, "prePageCount": 3}}Copy the code

summary

If you have a better solution, feel free to leave a comment

Recommended reading

  • Development process specification
  • Git branch design specification
  • API interface design specification
  • What exactly are the front-line technology managers in charge of?
  • It’s not just ability that gets a person promoted, it’s trust