Faster Async functions and promises

JavaScript’s asynchronous processes have long been considered not fast enough, and worse, debugging in real-time scenarios such as NodeJS can be a nightmare. However, this is changing, and this article will explain in detail how we optimized the async functions and promises in V8 (and some other engines), along with the development experience optimizations.

Tip: Here’s a video you can read in conjunction with the article.

A new approach to asynchronous programming

From Callbacks to promises to async functions

Before Promises became part of the JavaScript standard, callbacks were heavily used in asynchronous programming. Here’s an example:

function handler(done) {
  validateParams((error) = > {
    if (error) return done(error);
    dbQuery((error, dbResults) = > {
      if (error) return done(error);
      serviceCall(dbResults, (error, serviceResults) = > {
        console.log(result);
        done(error, serviceResults);
      });
    });
  });
}
Copy the code

Deeply nested callbacks like the one above are often referred to as “callback black holes” because they make code less readable and less maintainable.

Fortunately, promises are now part of the JavaScript language, and the following implementations do the same:

function handler() {
  return validateParams()
    .then(dbQuery)
    .then(serviceCall)
    .then(result= > {
      console.log(result);
      return result;
    });
}
Copy the code

Recently, JavaScript has supported async functions. The asynchronous code above can be written as synchronous code like this:

async function handler() {
  await validateParams();
  const dbResults = await dbQuery();
  const results = await serviceCall(dbResults);
  console.log(results);
  return results;
}
Copy the code

With async functions, the code becomes more concise, the logic of the code and the data flow become more controllable, of course, the underlying implementation is still asynchronous. (Note that JavaScript is still single-threaded, and async functions do not open new threads.)

Callback from event listener to async iterator

ReadableStreams are also particularly common in NodeJS as another form of asynchronomy. Here’s an example:

const http = require('http');
http.createServer((req, res) = > {
  let body = ' ';
  req.setEncoding('utf8');
  req.on('data'.(chunk) = > {
    body += chunk;
  });
  req.on('end'.() = > {
    res.write(body);
    res.end();
  });
}).listen(1337);
Copy the code

This code is a bit difficult to understand: the data stream in chunks can only be retrieved through a callback, and the end of the data stream must be handled in the callback. If you fail to understand that the function ends immediately but the actual processing must take place in the callback, you may introduce a bug.

Fortunately, there is a cool async iterator introduced in the ES2018 feature that simplifies the code above:

const http = require('http');
http.createServer(async (req, res) => {
  try {
    let body = ' ';
    req.setEncoding('utf8');
    for await (const chunk of req) {
      body += chunk;
    }
    res.write(body);
    res.end();
  } catch {
    res.statusCode = 500;
    res.end();
  }
}).listen(1337);
Copy the code

You can place all data processing logic in an async function using for await… Of deiterates chunks instead of handling them separately in the ‘data’ and ‘end’ callbacks, and we also add try-catch blocks to avoid unhandledRejection problems.

You can use these features in build environments today! Async functions have been fully supported since Node.js 8 (V8 V6.2 / Chrome 62), and async iterators since Node.js 10 (V8 V6.8 / Chrome 68).

Async performance optimization

From V8 V5.5 (Chrome 55 & Node.js 7) to V8 V6.8 (Chrome 68 & Node.js 10), we’ve been working on improving the performance of asynchronous code, and the results are good so far, so you can use these new features with confidence.

The doxBee benchmark shown above reflects performance with heavy use of Promise, and the vertical axis represents execution time, so smaller is better.

The PARALLEL benchmark, on the other hand, reflects performance with heavy use of Promise.all(), and the results are as follows:

Promise.all performance improved eight times!

However, the above tests are only small DEMO level tests, and the V8 team is more concerned with optimizing the actual user code.

This is a test based on popular HTTP frameworks that make heavy use of promises and async functions. This table shows the number of requests per second, so unlike previous tables, the larger the number, the better. As you can see from the table, the performance of Node.js 7 (V8 V5.5) to Node.js 10 (V8 V6.8) has improved considerably.

The performance improvement depends on three factors:

  • TurboFan, the new optimized compiler 🎉
  • Orinoco, the new garbage collector 🚛
  • A bug in Node.js 8 causes the await to skip some microticks 🐛

When TurboFan was enabled in Node.js 8, we got a huge performance boost.

We also introduced a new garbage collector called Orinoco, which removes garbage collection from the main thread, thus helping to speed up requests.

Finally, node.js 8 introduced a bug that sometimes causes await to skip micro-ticks, which actually improves performance. This bug was caused by an unintended violation of the specification, but it gives us some ideas for optimization. Let’s explain a little bit here:

const p = Promise.resolve();
(async() = > {await p; console.log('after:await'); }) (); p.then(() = > console.log('tick:a'))
 .then(() = > console.log('tick:b'));
Copy the code

The code above first creates a promise p with the completed state, then anticipates the result, and chains both then. What will the final output of console.log be?

Since p is done, you might expect it to print ‘after:await’ first, followed by the remaining two ticks. In fact, the result in Node.js 8 is:

Although the above results were expected, they did not meet the specifications. Node.js 10 corrects this behavior by executing the then chain first, then the async function.

This “correct behavior” may seem unusual and even surprise many JavaScript developers, but it’s worth explaining in more detail. Before we explain, let’s start with some basics.

Tasks vs. MicroTasks

At one level, there are tasks and microtasks in JavaScript. Tasks handle events such as I/O and timers one at a time. Microtasks are designed for delayed execution of async/await and promise, with each task executed last. The queue of microtasks is cleared before returning to the event loop.

Learn more about Tasks, Microtasks, Queues, and Schedules in the Browser by Jake Archibald. The task model in Node.js is very similar to this.

Async function

According to MDN, the async function is a function that executes asynchronously and implicitly returns a PROMISE as a result. From a developer’s perspective, async functions make asynchronous code look like synchronous code.

One of the simplest async functions:

async function computeAnswer() {
  return 42;
}
Copy the code

This function returns a promise, and you can use the returned value just like any other promise.

const p = computeAnswer();
/ / to Promise
p.then(console.log);
// prints 42 on the next turn
Copy the code

You can only get the value returned by promise p after the next microtask is executed. In other words, the code above is semantically equivalent to the result obtained by using promise.resolve:

function computeAnswer() {
  return Promise.resolve(42);
}
Copy the code

The real power of async functions comes from await expressions, which can make a function execution pause until a promise has been accepted, and then resume execution when it is fulfilled. The completed promise is treated as the value of the await. Here’s an example to illustrate this behavior:

async function fetchStatus(url) {
  const response = await fetch(url);
  return response.status;
}
Copy the code

FetchStatus suspends upon encountering await and resumes execution when the FETCH promise has completed, which is somewhat equivalent to directly chaining the promise returned by fetch.

function fetchStatus(url) {
  return fetch(url).then(response= > response.status);
}
Copy the code

The chained handler contains the code that preceded the await.

Normally you would put a Promise after “await”, but it can be followed by any JavaScript value. If it is not followed by a Promise, it will be converted to a Promise, so “await” 42 “will have the following effect:

async function foo() {
  const v = await 42;
  return v;
}
const p = foo();
/ / to Promise
p.then(console.log);
// prints `42` eventually
Copy the code

More interestingly, await can be followed by any “thenable”, such as any object that contains a THEN method, even if it is not a promise. So you can implement an interesting class to record execution time:

class Sleep {
  constructor(timeout) {
    this.timeout = timeout;
  }
  then(resolve, reject) {
    const startTime = Date.now();
    setTimeout(() = > resolve(Date.now() - startTime),
               this.timeout); }} (async() = > {const actualTime = await new Sleep(1000);
  console.log(actualTime); }) ();Copy the code

Let’s take a look at how the V8 specification handles await. Here’s a simple async function foo:

async function foo(v) {
  const w = await v;
  return w;
}
Copy the code

When it executes, it wraps v as a promise, pauses until the promise is complete, then assigns w to the completed promise, and finally async returns this value.

The mystery of theawait

First, V8 marks the function as recoverable, meaning that execution can be paused and resumed (from the await perspective). Then, a so-called IMPLICIT_PROMISE is created (to convert the value generated in the Async function to a Promise).

Then comes the interesting stuff: genuine await. First, the value following the await is converted to a promise. The handler then binds the promise to resume the main function after the promise is complete, at which point the async function is suspended, returning implICIT_PROMISE to the caller. Once the promise is complete, the function resumes and takes the value w from the Promise. Finally, implICIT_Promise is marked as accepted with W.

In simple terms, the await V initialization step consists of the following:

  1. thevTurn into a promiseawaitThe back).
  2. Bind handlers are used for later recovery.
  3. Pause the async function and returnimplicit_promiseTo the person who dropped it.

Let’s step by step, assuming that the await is followed by a promise and that the final completed state has a value of 42. The engine then creates a new promise and uses the value after the await as the value of resolve. Using standard PromiseResolveThenableJob the promise will be put into the next cycle.

The engine then creates another promise called Throwaway. It’s called that because there’s nothing else linked to it, it’s just used inside the engine. Throwaway Promises are chained to promises that contain recovery handlers. Promise.prototype.then() Eventually, the async function pauses and gives control to the caller.

The caller will continue, and eventually the call stack will be cleared, and the engine will start executing the microtask: Run before PromiseResolveThenableJob is ready, the first is a PromiseReactionJob, its job is just on the value of the transfer to await encapsulates a layer of promise. The engine then goes back to the microtask queue, because the microtask queue must be emptied before it can return to the event loop.

Then another PromiseReactionJob, waiting for the promise we are awaiting (we mean 42 here) to complete, and then assigning the action to the Throwaway Promise. The engine continues back to the microtask queue because there is one last microtask left.

Now the second PromiseReactionJob communicates the decision to the Throwaway Promise and resumes async function execution, finally returning 42 from the await.

In summary, the engine creates two additional promises for each await (even if the Rvalue is already a promise) and requires at least three microtasks. Who knew that a simple await could have so many redundant operations? !

So let’s look at what actually causes redundancy. The first line wraps a promise, and the second line acts as the value v after the promose await wrapped in resolve. These two lines produce one redundant promise and two redundant microtasks. It’s not a good deal if V is already a promise (which it is most of the time). “Await” 42 in some special scenarios, that does need to be encapsulated as a promise.

For this reason, you can use the promiseResolve operation to handle this, and only encapsulate promises when necessary:

If the incoming parameter is a Promise, it is returned intact, encapsulating only the necessary promises. This operation eliminates an additional promise and two microtasks if the value is already promose. This feature can be turned on in V8 (starting with V7.1) with the –harmony-await-optimization parameter, and we’ve made a proposal to ECMAScript that looks likely to merge soon.

Here is the simplified execution of await:

Thanks to the magic of promiseResolve, now we just pass V and don’t care what it is. Then, as before, the engine will create a throwaway promise and put it into the PromiseReactionJob. In order to restore the async function on the next tick, it will pause the function and return itself to the user who dropped it.

When everything is finished, the engine will run the microtask queue and execute the PromiseReactionJob. This task will pass the promise result to throwaway and restore the async function to get 42 from the await.

Despite being used internally, the engine creating the Throwaway Promise can feel like something is wrong. As it turns out, Throwaway Promises are simply designed to meet the requirements of the specification’s performPromiseThen.

This is a recent proposed change to ECMAScript, and the engine no longer needs to create throwaways most of the time.

Compare the performance of await on Node.js 10 with that of the optimized await (which should be placed on Node.js 12) :

Async /await performs better than handwritten promise codes. The key point is that we have reduced some of the unnecessary overhead in async functions, not only in the V8 engine, but also in other JavaScript engines.

Development experience optimization

In addition to performance, JavaScript developers are also concerned with problem locating and fixing, which is not always easy in asynchronous code. Chrome DevTools now supports asynchronous stack tracing:

This is a useful feature when developing locally, but not once the application is deployed. When debugging, you can only see the Error#stack information in the log file, which does not contain any asynchronous information.

The recent zero-cost asynchronous stack trace we’ve been working on has Error#stack containing the async function call information. “Zero cost” sounds exciting, right? How can Chrome DevTools come to zero cost when it has significant overhead? For example, calling bar in foo throws an exception after “await” a promise:

async function foo() {
  await bar();
  return 42;
}
async function bar() {
  await Promise.resolve();
  throw new Error('BEEP BEEP');
}
foo().catch(error= > console.log(error.stack));
Copy the code

This code runs in node.js 8 or Node.js 10 as follows:

$ node index.js Error: BEEP BEEP at bar (index.js:8:9) at process._tickCallback (internal/process/next_tick.js:68:7) at Function.Module.runMain  (internal/modules/cjs/loader.js:745:11) at startup (internal/bootstrap/node.js:266:19) at bootstrapNodeJSCore (internal/bootstrap/node.js:595:3)Copy the code

Note that foo itself is not in the stack trace, despite the fault of a call in foo(). If the application is deployed in a cloud container, this can make it difficult for developers to locate problems.

Interestingly, the engine knows what to continue after bar has finished: after the await in the foo function. As it happens, this is where Foo stops. The engine can use this information to reconstruct an asynchronous stack trace. With the above optimizations, the output will look like this:

$ node --async-stack-traces index.js Error: BEEP BEEP at bar (index.js:8:9) at process._tickCallback (internal/process/next_tick.js:68:7) at Function.Module.runMain  (internal/modules/cjs/loader.js:745:11) at startup (internal/bootstrap/node.js:266:19) at bootstrapNodeJSCore (internal/bootstrap/node.js:595:3) at async foo (index.js:2:3)Copy the code

In the stack trace, the topmost function appears first, followed by some asynchronous call stacks, followed by the stack information of the bar context in foo. This feature can be enabled via the –async-stack-traces parameter in V8.

However, if you compare the stack information in Chrome DevTools above, you will see that the asynchronous part of the stack trace is missing the call point information for Foo. We take advantage of the fact that await recovery is the same as paused location, but this is not the case for Promise#then() or Promise#catch(). See Mathias Bynens’ post await beats Promise#then() to learn more.

conclusion

There are two optimizations that make async functions faster:

  • Two additional microtasks were removed
  • removedthrowaway promise

In addition, we have improved the development and debugging experience of await and promise.all () with zero cost asynchronous stack tracing.

We also have some JavaScript developer-friendly performance advice:

Use async and await rather than hand-written promise code, and use promises provided by the JavaScript engine rather than implementing them yourself.

The article can be reproduced at will, but please keep the original link.

If you are passionate enough to join ES2049 Studio, please send your resume to caijun.hcj(at)alibaba-inc.com.