We know that Node.js is full of asynchracy and later promises and async/await to solve the “callback hell” problem. Let’s take a look at how Promise and async/await simplify writing CONCURRENT JS code, and finally give a Go code that does the same thing.

The problem

One of the common things to do in code development is to request an API, and possibly further gain access to a new interface based on the API’s return results. Here we construct a problem: get the ID, title, date, author’s nickname, and first responder’s nickname for the top 10 topics at https://cnodejs.org/. Cnodejs provides apis, and the first two interfaces here at https://cnodejs.org/api will do the job. First use https://cnodejs.org/api/v1/topics interface to get to the top 10 switchable viewer, and then take out each topic id to access the get/topic / : id theme details interface, it can obtain the response data.

Simple implementation

There are many ways to initiate a web request, but we used the AXIos library here, which has several benefits, including support for both Node.js and Browser.

We implement a version directly with “state of the art” async/await:

1const axios = require("axios"); 2 3async function getFirst10TopicsIncludeFirstReplyAuthor() { 4 const response = await axios.get( 5 "https://cnodejs.org/api/v1/topics?limit=10" 6 ); 7 const json = response.data; 8 const first10 = json.data.map(topic => { 9 return {10 id: topic.id,11 title: topic.title,12 date: topic.create_at,13 author: topic.author.loginname14 }; 15}); 1617 for (let topic of first10) {18 const response = await axios.get(19 `https://cnodejs.org/api/v1/topic/${topic.id}`20 ); 21 const json = response.data; 22 const firstReply = json.data.replies[0]; 23 topic.firstReplyAuthor = firstReply && firstReply.author.loginname; 24 }2526 return first10; 27}2829getFirst10TopicsIncludeFirstReplyAuthor().then(data => console.log(data));Copy the code

concurrent

The above code is simple and direct, using async/await. Asynchronous code looks basically synchronous and is straightforward to understand. Make a request for 10 topics, then make a request for each topic to get the first response data, then piece it together and return it. Since later requests require the ID returned by the first request, they can’t be sent until the first request comes back, which is fine. But the next 10 requests are completely independent, so they can be made concurrently, which greatly reduces the time. For example, if each request takes 1s, the total cost of the above code is 1(the first request) + 10(the next 10 requests) = 11s, whereas if the second request is completely concurrent, it only takes 1(the first request) + 1(the next 10 requests simultaneously) = 2s!!

Since the network request is greatly affected by the network speed, it is not conducive to our accurate analysis of the problem, and to avoid the impact of a large number of requests on the Cnodejs service, we use setTimout locally to simulate the time spent by the network request.

The above code is equivalent in concurrency to the following code:

2function mockAPI(result, time = 1000) {3 return new Promise((resolve, resolve) reject) => { 4 setTimeout(() => { 5 resolve(result); 6 }, time); 7}); 8} 910async function get10Topics() {11 const t1 = Date.now(); 12 const result = []; 13 const total = await mockAPI(10); 14 for (let i = 1; i <= total; i += 1) {15 const r = await mockAPI(i); 16 result.push(r); 17 }18 const t2 = Date.now(); 19 console.log(`total cost: ${t2 - t1}ms.`); 20 return result; 21}2223get10Topics().then(data => console.log(data));Copy the code

After execution, it is found that it is indeed around 11s:

1➜ test-js git:(master) qualify node p1.js2total cost: 11037ms.3[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]Copy the code

Promise.all can initiate multiple promises at the same time and returns an array containing the results of each Promise when all of them are completed.

2function mockAPI(result, time = 1000) {3 return new Promise((resolve, resolve) reject) => { 4 setTimeout(() => { 5 resolve(result); 6 }, time); 7}); 8} 910async function get10Topics2() {11 const t1 = Date.now(); 12 const total = await mockAPI(10); 13 const promises = []; 14 for (let i = 1; i <= total; i += 1) {15 promises.push(mockAPI(i)); 16 }17 const result = await Promise.all(promises)18 const t2 = Date.now(); 19 console.log(`total cost: ${t2 - t1}ms.`); 20 return result; 21}2223get10Topics2().then(data => console.log(data));Copy the code

The time, as we said, is reduced to 2s!

1➜ test-js git:(Master) Qualify node p2.js2Total cost: 2005ms.3[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]Copy the code

Current limiting

The second method above has greatly improved efficiency, and the more requests there are, the more efficiency there is. From the previous analysis, if getting the top 100 topics, the first serial method needs 101s and the second method still needs 2s!!

If you think about it, what’s wrong with the second method is that it’s too concurrent! Ten requests might be fine, but if you have 100 concurrent requests, it’s going to have some impact on the server. If you have 1,000, 10,000, it’s going to be even more of a problem, and at some point, it’s going to exceed the number of connections the operating system is allowed to open, and it’s going to have a big impact on the client itself.

So we need to limit the maximum number of concurrent requests. For example, if we limit the maximum number of concurrent requests to 3, then 10 requests are about 3 in groups of 3, and there will be 4 groups in total (the last group has only 1). The total time is 5s, which is also more than 50% improvement over 11s. One way to do this is as follows:

1async function get10Topics3() { 2 const t1 = Date.now(); 3 const total = await mockAPI(10); 4 const MAX_CURRENCY = 3; 5 const result = []; 6 for (let i = 1; i <= total; i += MAX_CURRENCY) { 7 const promises = []; 8 for (let j = i; j < i + MAX_CURRENCY && j <= total; j += 1) { 9 promises.push(mockAPI(j)); 10 }11 const r = await Promise.all(promises); 12 result.push(... r); 13 }14 const t2 = Date.now(); 15 console.log(`total cost: ${t2 - t1}ms.`); 16 return result; 17}1819get10Topics3().then(data => console.log(data));Copy the code

Take a look at the results:

1➜ test-js git:(master) qualify node p3.js2total cost: 5012ms.3[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]Copy the code

Any other questions?

One More Step

The above approach, which takes advantage of concurrency while limiting it so that it doesn’t exhaust system resources, seems perfect. But what if each request takes a different amount of time? Get10Topics3 is implemented in groups of three, waiting for all three to complete before proceeding to the next set of requests. So if one of the three tasks takes a long time, and the other two tasks are completed, you could have moved on to the new task, but now you have to wait for the third task to complete before starting the new task. Even if all three tasks take different amounts of time, the first one has to wait for the second and the third, and the second one has to wait for the third, and the whole system is dragged down by the slowest task. For example, the first task needs 1s, the second task needs 2s, and the third task needs 3s, then get10Topics3 requires 3s for each group of tasks, 3 * 3 = 9s, and the last group of tasks only needs 1s, the total need 1 + 3 + 3 + 1 = 11s. Of course, this is much faster than the time required for full serial 1 + 1 + 2 + 3 + 1 + 2 + 3 + 1 = 20s.

2function mockAPI(result, time = 1000) {3 console.log(result, time); 4 return new Promise((resolve, reject) => { 5 setTimeout(() => { 6 resolve(result); 7 }, time); 8}); 9}1011async function get10Topics4() {12 const t1 = Date.now(); 13 const total = await mockAPI(10); 14 const MAX_CURRENCY = 3; 15 const result = []; 16 for (let i = 1; i <= total; i += MAX_CURRENCY) {17 const promises = []; 18 for (let j = i; j < i + MAX_CURRENCY && j <= total; j += 1) {19 const costtime = j % 3 === 0 ? 3 : j % 3; // The first task is 1s, the second 2 is, the third 3s... 20 promises.push(mockAPI(j, costtime * 1000)); 21 }22 const t3 = Date.now(); 23 const r = await Promise.all(promises); 24 const t4 = Date.now(); 25 console.log(`promise ${i} cost: ${t4 - t3}ms`); 26 result.push(... r); 27 }28 const t2 = Date.now(); 29 console.log(`total cost: ${t2 - t1}ms.`); 30 return result; 31}3233get10Topics4().then(data => console.log(data));Copy the code

Running results:

1➜ test-js git:(Master) Qualify node p4.js 210 1000 31 1000 42 2000 53 3000 6Promise 1 cost: 3002ms 74 1000 85 2000 96 300010promise 4 cost: 2999ms117 1000128 2000139 300014promise 7 cost: 3002ms1510 100016promise 10 cost: 1005ms17total cost: 11030ms.18[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]Copy the code

It’s easy to think of a way to reduce the amount of time you have to wait for each other by putting short tasks together and executing them concurrently. For example, if you put 4 1s together, 3 2s together, and 3 3s together, the total time required is: 1 + 1 + 2 + 3 + 1 = 8s, which is a little higher. On the one hand, we don’t know exactly how long each task will take until the actual task is started and completed. On the other hand, it’s impossible that there will always be a group of tasks that take the same amount of time, and even in extreme cases, each task will take a different amount of time.

If you think about it, all you need to do is build a task pool, start with three tasks, and when each task comes back, don’t wait for the other two tasks, just see if there are any more tasks in the pool, and do them until all of them are complete.

Since there are no semaphores in Node.js to synchronize work between “threads”, this is done by recursive manipulation of public variables, if there is a better way for readers to leave a message to the author. Note, “Concurrently modifying shared variables is the root of all evil, there are data race issues, fortunately JS is single-threaded, so there is no such problem.

2function mockAPI(result, time = 1000) {3 console.log(result, time); 4 return new Promise((resolve, reject) => { 5 setTimeout(() => { 6 resolve(result); 7 }, time); 8}); 9}1011const start = Date.now(); 12function worker(tasks, result) {13 const task = tasks.shift(); 14 if (! Task) {15 // Task ends 16 return; 17 }18 const costtime = task % 3 === 0 ? 3 : task % 3; // The first task is 1s, the second 2 is, the third 3s... 19 return mockAPI(task, costtime * 1000).then(r => {20 console.log(`${r} completes at time: ${Date.now() - start}`); 21 result.push(r); 22 return worker(tasks, result); 23}); 24}2526async function get10Topics5() {27 const t1 = Date.now(); 28 const total = await mockAPI(10); 29 const MAX_CURRENCY = 3; 30 const result = []; 3132 const tasks = []; 33 for (let i = 1; i <= total; i += 1) {34 tasks.push(i); 35 }3637 const promises = []; 38 for (let i = 0; i < MAX_CURRENCY; i += 1) {39 promises.push(worker(tasks, result)); 40 }4142 const r = await Promise.all(promises); 43 const t2 = Date.now(); 44 console.log(`total cost: ${t2 - t1}ms.`); 45 return result; 46}4748get10Topics5().then(data => console.log(data));Copy the code

Run the code to see the result:

1➜ test-js git:(Master) Eligible Node p5.js 210 1000 31 1000 42 2000 53 3000 61 at time: 2s, by worker0 74 1000 82 completes at time: 3s, by worker1 95 2000104 completes at time: 3s, by worker0116 3000123 completes at time: 4s, by worker2137 1000145 completes at time: 5s, by worker1158 2000167 completes at time: 5s, by worker2179 3000186 completes at time: 6s, by worker01910 1000208 completes at time: 7s, by worker12110 completes at time: 7s, by worker0229 completes at time: 8s, by worker223total cost: 8032ms.24[ 1, 2, 4, 3, 5, 7, 6, 8, 10, 9 ]Copy the code

As you can see, worker0, worker1, and Worker2 are started simultaneously. Worker0 completes Task1 in 2s (since 1s is the first API call), and instead of waiting, it continues to work on Task4. Then worker0 completes task4 and starts task6. Then worker2 completes task3 and starts task7. It can be seen that each worker is scrambling to complete the task until all tasks are completed, which takes a total of 8s.

Re-implement the concurrent access API

Here I change the original serial access API code to concurrent execution, without limiting the flow, readers can be modified from the previous analysis into limiting the flow version, as a small exercise.

1const axios = require("axios"); 2 3function getFirst10TopicsIncludeFirstReplyAuthor() { 4 return axios 5 .get("https://cnodejs.org/api/v1/topics?limit=10") 6 .then(function(response) { 7 const json = response.data; 8 const first10 = json.data.map(topic => { 9 return {10 id: topic.id,11 title: topic.title,12 date: topic.create_at,13 author: topic.author.loginname14 }; 15}); 1617 const promises = first10.map(data => {18 return axios19 .get(`https://cnodejs.org/api/v1/topic/${data.id}`)20 .then(response => {21 const json = response.data; 22 const firstReply = json.data.replies[0]; 23 return {24 id: json.data.id,25 firstReplyAuthor: firstReply && firstReply.author.loginname26 }; 27}); 28}); 29 return Promise.all(promises).then(rs => {30 const map = rs.reduce((acc, e) => {31 acc.set(e.id, e); 32 return acc; 33 }, new Map()); 34 for (let topic of first10) {35 topic.firstReplyAuthor = map.get(topic.id).firstReplyAuthor; 36 }37 return first10; 38}); 39 })40 .catch(function(error) {41 console.log(error); 42}); 43}4445getFirst10TopicsIncludeFirstReplyAuthor().then(data => console.log(data));Copy the code

Go language implementation

Actually, Go has nothing to do with Promise. It’s just that I’ve been making things in Go recently, so it’s a comparison. Go is very easy to implement the function of limiting the flow, here directly pasted code, do not do too much analysis.

 1package main 2 3import ( 4    "fmt" 5    "time" 6) 7 8const start = time.Now().Unix() 910func mockAPI(result int, duration time.Duration) int {11    fmt.Println(result, duration)12    time.Sleep(duration)13    return result14}1516func worker(id int, jobs <-chan int, result chan<- int) {17    for job := range jobs {18        t := job % 319        if t == 0 {20            t = 321        }22        r := mockAPI(job, (time.Duration)(t)*time.Second)23        diff := time.Now().Unix() - start24        fmt.Printf("%d completes at time: %ds, by worker%d\n", r, diff, id)25        result <- r26    }27}28func main() {29    t1 := time.Now().Unix()3031    jobs := make(chan int, 10)32    result := make(chan int, 10)33    total := mockAPI(10, 1*time.Second)3435    const MaxCurrency = 336    for i := 0; i < MaxCurrency; i++ {37        go worker(i, jobs, result)38    }3940    for i := 1; i <= total; i++ {41        jobs <- i42    }43    close(jobs)4445    rs := make([]int, total)46    for i := 0; i < total; i++ {47        r := <-result48        rs[i] = r49    }5051    t2 := time.Now().Unix()52    fmt.Printf("total cost: %ds.\n", (t2 - t1))53    fmt.Println(rs)54}Copy the code

The output is as follows:

1➜ chap8 go run currency-rate-limit2.go 210 1s 31 1s 42 2s 53 3s 61 at time: 2s, by worker0 74 1s 84 completes at time: 3s, by worker0 92 completes at time: 3s, by worker2105 2s116 3s123 completes at time: 4s, by worker1137 1s147 completes at time: 5s, by worker1158 2s165 completes at time: 5s, by worker0179 3s186 completes at time: 6s, by worker21910 1s208 completes at time: 7s, by worker12110 completes at time: 7s, by worker2229 completes at time: 8s, by worker023total cost: 8s.24[1 4 2 3 7 5 6 8 10 9]Copy the code

The resources

• https://developer.mozilla.org/zh-CN/docs/Web/JavaScript/Reference/Global_Objects/Promise

• https://yar999.gitbooks.io/gopl-zh/content/ch8/ch8-06.html