When the amount of data is large, the data is divided by database and table to share the burden of reading and writing. After dividing the database into tables, the more troublesome problem is the query. If you do not query directly according to the sharding key, you need to query multiple tables.

In some complex business scenarios, such as order search, in addition to order number, user, merchant and other commonly used search criteria, there may be time, goods, etc.

At present, the common practice is to synchronize data to search framework such as ES for query, and then query the complete data in the specific data table through the search result, usually the primary key ID, and then assemble and return to the caller.

For example, the following code queries the article information and then queries the user’s nickname based on the user ID in the article.

List<ArticleBO> articleBos = articleDoPage.getRecords().stream().map(r -> {
    String nickname = userManager.getNickname(r.getUserId());
    return articleBoConvert.convertPlus(r, nickname);
}).collect(Collectors.toList());
Copy the code

If the article has 10 pieces of data, then the interface provided by the user service needs to be invoked 10 times, and the operation is invoked synchronously.

Of course, we can also use parallel flow to achieve concurrent calls, the code is as follows:

List<ArticleBO> articleBos = articleDoPage.getRecords().parallelStream().map(r -> {
    String nickname = userManager.getNickname(r.getUserId());
    return articleBoConvert.convertPlus(r, nickname);
}).collect(Collectors.toList());
Copy the code

The advantage of parallel streams is that the code doesn’t have to change very much. Note that if parallel streams are used, it is best to define a separate ForkJoinPool.

In addition to using parallel streams, you can also use batch query to improve performance and reduce the number of RPC calls, the code is as follows:

List<Long> userIds = articleDoPage.getRecords().stream().map(article -> article.getUserId()).collect(Collectors.toList());
Map<Long, String> nickNameMap = userManager.queryByIds(userIds).stream().collect(Collectors.toMap(UserResponse::getId, UserResponse::getNickname));
List<ArticleBO> articleBos = articleDoPage.getRecords().stream().map(r -> {
    String nickname = nickNameMap.containsKey(r.getUserId()) ? nickNameMap.get(r.getUserId()) : CommonConstant.DEFAULT_EMPTY_STR;
    return articleBoConvert.convertPlus(r, nickname);
}).collect(Collectors.toList());
Copy the code

If you use CompletableFuture to implement asynchronous concurrent calls, you can also use the native CompletableFuture directly, but the arrangement ability is not so strong. Here we choose a parallel scheduling based on CompletableFuture packaging box, look at me before this article in detail: mp.weixin.qq.com/s/3EE8ccydK…

A bit of encapsulation provides a more user-friendly utility class to implement the logic of calling multiple interfaces concurrently.

The first method applies to, for example, finding out a batch of ids from ES, then querying real data in the database or calling RPC according to the IDS, and finally obtaining a Map, which can obtain corresponding data according to the Key.

Internally, there are multiple concurrent calls that wait until all the results are returned.

public Object aggregationApi() { long s = System.currentTimeMillis(); List<String> ids = new ArrayList<>(); ids.add("1"); ids.add("2"); ids.add("3"); Map<String, UserResponse> callResult = AsyncTemplate.call(ids, id -> { return userService.getUser(id); }, u -> u.getId(), COMMON_POOL); long e = System.currentTimeMillis(); System.out.println(" time: "+ (e-s) + "ms"); return ""; }Copy the code

Another scenario is API aggregation, where multiple interfaces are called in parallel to assemble the results.

List<AsyncCall> params = new ArrayList<>();
AsyncCall<Integer, Integer> goodsQuery = new AsyncCall("goodsQuery", 1);
params.add(goodsQuery);
AsyncCall<String, OrderResponse> orderQuery = new AsyncCall("orderQuery", "100");
params.add(orderQuery);
UserQuery q = new UserQuery();
q.setAge(18);
q.setName("yinjihuan");
AsyncCall<UserQuery, UserResponse> userQuery = new AsyncCall("userQuery", q);
params.add(userQuery);
AsyncTemplate.call(params, p -> {
    if (p.getTaskId().equals("goodsQuery")) {
        AsyncCall<Integer, Integer> query = p;
        return goodsService.getGoodsName(query.getParam());
    }
    if (p.getTaskId().equals("orderQuery")) {
        AsyncCall<String, OrderResponse> query = p;
        return orderService.getOrder(query.getParam());
    }
    if (p.getTaskId().equals("userQuery")) {
        AsyncCall<UserQuery, UserResponse> query = p;
        return userService.getUser(query.getParam());
    }
    return null;
});
Copy the code

AsyncCall defines parameters and response types. The response result is automatically set to AsyncCall after execution. In the call method, you need to do the corresponding processing logic according to taskId. Different taskId calls different interfaces.

Source code reference: github.com/yinjihuan/k…

About the author * : Yin Jihuan, a simple technology lover, the author of Spring Cloud Micro-service – Full Stack Technology and Case Analysis, Spring Cloud Micro-service Introduction combat and Progress, the founder of the public account Monkey World.