Read the source code of CO module from Generator

This article is about the JS asynchronous principle and implementation of the fourth article, the first three are:

SetTimeout and setImmediate Mediate Mediate Do not mediate. This article will help you understand the Event Loop

Read EventEmitter source code of Node.js from publish and subscribe mode

Hand-write A Promise/A+ that passes the official 872 test cases flawlessly

This article will mainly talk about the application and implementation principle of Generator, then we will read the source code of CO module, and finally we will mention async/await.

All examples of this article are available on GitHub: github.com/dennis-jian…

Generator

Asynchronous programming has always been one of the cores of JS, and the industry has been exploring different solutions to optimize asynchronous programming, from “callback hell” to publish-subscribe models to promises. While promises are good enough that they don’t fall into “callback hell,” there are a lot of nested layers and a bunch of THEN’s that can’t always be written straight down like synchronous code. Generator is a scheme introduced in ES6 to further improve asynchronous programming, so let’s take a look at its basic usage.

Basic usage

A function that adds * becomes a Generator function and returns an iterator object, as in the following code:

// gen is a generator function
function* gen() {
  let a = yield 1;
  let b = yield a + 2;
  yield b + 3;
}

let itor = gen();   // The generator function returns an iterator object called itor.
Copy the code

The ES6 specification states that an iterator must have a next method, which returns an object with two attributes: done and value. Done indicates whether the iterator content has been completed, true when it is completed, false otherwise, and value indicates the value returned by the current step. In generator usage, execution is paused each time the yield keyword is encountered, and when next of the iterator is called, the value of the expression following yield is used as the value of the returned object, such as the result of the generator above:

We can see that the first call to Next returns the value of the expression after the first yeild, which is 1. Note that the entire iterator is currently paused at the first yield, assignment to variable A is not performed, assignment to variable A is not performed until next is called, and execution continues until the second yield. So what value should I put on a? From the code, the value of a should be the yield statement’s return value, but yield itself does not return a value, or the return value is undefined. If assigning a value to a needs to be passed in manually the next time we call next, we pass a 4, and 4 will be assigned to a as the yield statement’s return value:

You can see that the expression a + 2 after the second yield is 6, because the 4 we passed in was used as the yield return, and then a + 2 is 6.

Let’s continue next and run through the iterator:

The first value returned by next is NaN because we did not pass a parameter when we called next, that is, b is undefined, undefined + 3 is NaN. The value of next should be the value of the function return, but since we did not write return, the default value is return undefined, and done will be set to true after execution.

throw

Another iterator method is throw. This method can throw an error outside the function body and catch it inside the function, as in the above example:

function* gen() {
  let a = yield 1;
  let b = yield a + 2;
  yield b + 3;
}

let itor = gen();  
Copy the code

Instead of executing next, we’ll throw an error:

This error is thrown directly to the outermost layer because we did not catch it. We can catch it inside the function body.

function* gen() {
  try {
    let a = yield 1;
    let b = yield a + 2;
    yield b + 3;
  } catch (e) {
    console.log(e); }}let itor = gen();  
Copy the code

Then throw again:

As you can see from this graph, the error is caught in the function, and it goes to the catch, and there’s only one console sync in there, and the whole function just runs, so done becomes true, and of course the catch can yield and then execute with next.

return

The iterator returns a method that terminates the iterator and sets done to true. This method takes the iterator value as shown in the above example:

function* gen() {
  let a = yield 1;
  let b = yield a + 2;
  yield b + 3;
}

let itor = gen();  
Copy the code

This time we call return directly:

yield*

Simply put, yield* simply means that a generator calls another generator in a generator, but it does not occupy a next, but runs directly into the generator being called.

function* gen() {
  let a = yield 1;
  let b = yield a + 2;
}

function* gen2() {
  yield 10 + 5;
  yield* gen();
}

let itor = gen2();  
Copy the code

The first time we call next, the value of course is 10 + 5, which is 15, and the second time we call next, we actually yield*, which is the equivalent of calling gen, and then performing its first yield, which is 1.

coroutines

The Generator implements coroutines, a concept smaller than threads. A process can have multiple threads, a thread can have multiple coroutines, but a thread can only have one coroutine running at a time. This means that if the current coroutine can execute, such as synchronous code, then execute it. If the current coroutine can’t execute temporarily, such as an asynchronous file read operation, then suspend it and execute another coroutine. When the result of the coroutine comes back, it can execute it again. Yield is essentially the equivalent of suspending the current task and starting from there the next call. The concept of coroutines has been around for years, and many other languages have their own implementations. Generator is equivalent to a JS implementation coroutine.

Asynchronous application

The basic usage of the Generator was described earlier. Let’s use it to handle an asynchronous event. I’ll use the example I used in the previous article. Three network requests, request 3 depends on the result of request 2, and request 2 depends on the result of request 1. With a callback it looks like this:

const request = require("request");

request('https://www.baidu.com'.function (error, response) {
  if(! error && response.statusCode ==200) {
    console.log('get times 1');

    request('https://www.baidu.com'.function(error, response) {
      if(! error && response.statusCode ==200) {
        console.log('get times 2');

        request('https://www.baidu.com'.function(error, response) {
          if(! error && response.statusCode ==200) {
            console.log('get times 3'); }})}})});Copy the code

We use Generator to solve callback hell this time:

const request = require("request");

function* requestGen() {
  function sendRequest(url) {
    request(url, function (error, response) {
      if(! error && response.statusCode ==200) {
        console.log(response.body);

        // Notice that the external iterator itor is referenced hereitor.next(response.body); }})}const url = 'https://www.baidu.com';

  // Use yield to initiate three requests, and continue next after each request succeeds
  const r1 = yield sendRequest(url);
  console.log('r1', r1);
  const r2 = yield sendRequest(url);
  console.log('r2', r2);
  const r3 = yield sendRequest(url);
  console.log('r3', r3);
}

const itor = requestGen();

// Manually call the first next
itor.next();
Copy the code

In this example, we write a request method in the generator, which initiates the network request, calls next for yield after each successful network request, and then manually calls next in the outer layer to trigger the process. This is actually like a tail call, which can be written this way, but references the iterator itor outside the requestGen, which is highly coupled and hard to reuse.

Thunk function

In order to solve the problem of high coupling and poor reuse, there is the thunk function. The thunk function is a bit convoluted to understand, so I’ll write the code first and then analyze its execution order step by step:

function Thunk(fn) {
  return function(. args) {
    return function(callback) {
      return fn.call(this. args, callback) } } }function run(fn) {
  let gen = fn();
  
  function next(err, data) {
    let result = gen.next(data);
    
    if(result.done) return;
    
    result.value(next);
  }
  
  next();
}

// Use the thunk method
const request = require("request");
const requestThunk = Thunk(request);

function* requestGen() {
  const url = 'https://www.baidu.com';
  
  let r1 = yield requestThunk(url);
  console.log(r1.body);
  
  let r2 = yield requestThunk(url);
  console.log(r2.body);
  
  let r3 = yield requestThunk(url);
  console.log(r3.body);
}

// Start running
run(requestGen);
Copy the code

The Thunk function in this code returns several layers of functions. Let’s peel them down one by one:

RequestThunk is the return value of the Thunk operation, and the argument is requestThunk.

function(. args) {
  return function(callback) {
    return request.call(this. args, callback);// Note that request is called}}Copy the code

The run function takes a generator as an argument, so let’s see what it does:
1. Run calls the generator, gets the iterator gen, and then customizes a next method and calls the next method. I’ll call this custom Next local Next for the sake of differentiation
2. Local next will call the generator’s next. The generator’s next is actually yield requestThunk(URL). The parameter is the URL we passed in.
```
function(callback) {
  return request.call(this, url, callback);   
}
Copy the code
```
3. Check if the iterator has finished iterating, and if not, continue to call the function in step 2, which actually goes to request. In this case, local Next is passed in, which also serves as a callback to Request.
4. This callback calls Gen. next, which allows the generator to proceed, and takes the callback’s data, so that r1 in the generator actually gets the return value of the request.

The Thunk function is one such function that can execute Generator automatically. Because of the wrapper of the Thunk function, we can yield the return value of asynchronous code in Generator just like synchronous code.

Co module

The CO module is a popular module that can also automatically execute generators. Its yield supports thunk and Promise, so let’s take a look at its basic use and then examine its source code. Official GitHub: github.com/tj/co

The basic use

Support thunk

Earlier we talked about the thunk function, so let’s start with the thunk function. The thunk function is still used, but since co supports thunk as a function that accepts only callback functions, we need to adjust it:

// The same thunk function as before
function Thunk(fn) {
  return function(. args) {
    return function(callback) {
      return fn.call(this. args, callback) } } }// Convert request to thunk
const request = require('request');
const requestThunk = Thunk(request);

// The converted requestThunk can now be used directly
// requestThunk(url)(callback)
// But our co receives thunk as fn(callback)
// Let's switch
// baiduRequest is also a function. The URL has already been passed. It only needs a callback function as an argument
// baiduRequest(callback)
const baiduRequest = requestThunk('https://www.baidu.com');

// Introduce co execution, whose parameter is a Generator
// The return value of co is a Promise, and we can use then to get its result
const co = require('co');
co(function* () {
  const r1 = yield baiduRequest;
  const r2 = yield baiduRequest;
  const r3 = yield baiduRequest;
  
  return {
    r1,
    r2,
    r3,
  }
}).then((res) = > {
  {r1, r2, r3}}
  console.log(res);
});
Copy the code

Supporting Promise

Co officially recommends that yield be followed by Promise. Thunk is supported, but may be removed in the future. Using Promise, our code was actually much simpler to write, just using fetch instead of wrapping Thunk.

const fetch = require('node-fetch');
const co = require('co');
co(function* () {
  // Fetch returns a Promise
  const r1 = yield fetch('https://www.baidu.com');
  const r2 = yield fetch('https://www.baidu.com');
  const r3 = yield fetch('https://www.baidu.com');
  
  return {
    r1,
    r2,
    r3,
  }
}).then((res) = > {
  {r1, r2, r3}
  console.log(res);
});
Copy the code

Source code analysis

This article source analysis based on co module 4.6.0 version, source: github.com/tj/co/blob/…

If you look closely at the source code, you’ll see that it doesn’t have a lot of code, more than two hundred lines, half of which are used for yield checking to see if it’s a Promise, and if it’s not a Promise, it’ll still be converted to a Promise, so even if you pass thunk after yield, it’ll still be converted to a Promise. The code for transforming promises is relatively independent and simple. I won’t expand it in detail here, but I’ll focus on the core method co(Gen). Here’s my copy of the simplified code with the comments removed:

function co(gen) {
  var ctx = this;
  var args = slice.call(arguments.1);

  return new Promise(function(resolve, reject) {
    if (typeof gen === 'function') gen = gen.apply(ctx, args);
    if(! gen ||typeofgen.next ! = ='function') return resolve(gen);

    onFulfilled();

    function onFulfilled(res) {
      var ret;
      try {
        ret = gen.next(res);
      } catch (e) {
        return reject(e);
      }
      next(ret);
      return null;
    }

    function onRejected(err) {
      var ret;
      try {
        ret = gen.throw(err);
      } catch (e) {
        return reject(e);
      }
      next(ret);
    }

    function next(ret) {
      if (ret.done) return resolve(ret.value);
      var value = toPromise.call(ctx, ret.value);
      if (value && isPromise(value)) return value.then(onFulfilled, onRejected);
      return onRejected(new TypeError('You may only yield a function, promise, generator, array, or object, '
        + 'but the following object was passed: "' + String(ret.value) + '"')); }}); }Copy the code

From the overall structure, the parameter of CO is a Generator, and the return value is a Promise. Almost all the logical codes are in the Promise, which is why we use “then” to get the result.
The Promise takes the Generator out and executes it, resulting in an iterator gen
Manually invoke ondepressing once to start the iteration
1. onFulfilledReceiving a parameterresThe first call does not pass this parameter, which is mainly used to receive the result of subsequent then returns.
2. And then callgen.next, notice that this returns the ret of the form {value, done}, and passes the ret locally to next
It then executes local next, which receives the yield return value {value, done}.
1. This first checks if the iteration is complete, and if so, the entire Promise resolve is directly resolved.
2. Here the value is the value of the expression following yield, which can be either thunk or promise
3. Convert value to promise
4. The transformed promise is taken out and executed, and the successful callback comes firstonFulfilled
This is the second time that onFulfilled. The parameter res passed in this time is the result of the last asynchronous promise, and our fetch is the data that we fetch back, which is passed to the second Gen. next. The effect is that the assignment in our code is given to the r1 variable before the first yield. Then proceed to the local next, which essentially implements the second asynchronous Promise. The promise’s success callback continues to call Gen.next, and so on until done becomes true.
A final look at the onRejected () method, which is an error branch of the asynchronous promise, calls Gen.throw so that we can use try… catch… Get the error. It is important to note that Gen. Throw continues to call next(ret), because there may be yield on the catch branch of the Generator, such as an error-reported network request, in which case the iterator does not necessarily end.

async/await

Finally async/await, let’s see how to use it:

const fetch = require('node-fetch');

async function sendRequest () {
  const r1 = await fetch('https://www.baidu.com');
  const r2 = await fetch('https://www.baidu.com');
  const r3 = await fetch('https://www.baidu.com');
  
  return {
    r1,
    r2,
    r3,
  }
}

// Note that async also returns a promise
sendRequest().then((res) = > {
  console.log('res', res);
});
Copy the code

The return value is a promise, but the Generator is replaced with an async function, the yield is replaced with an await function, and the outer layer does not need to be wrapped with co. In fact, async function is a syntactic sugar of Generator plus automatic executor, which can be understood as supporting automatic execution of Generator from the language level. This code is equivalent to the CO version of Promise.

conclusion

Generator is a more modern asynchronous solution that supports coroutines at the JS language level
The return value of Generator is an iterator
This iterator needs to be tuned manuallynextCan be executed one by oneyield
nextReturns {value, done},valueIs the value of the expression following yield
yieldThe statement itself does not return a value, next callnextThe parameter will be used as the previous oneyieldStatement return value
The Generator cannot execute automatically on its own, and to do so requires the introduction of other schemes, as described earlierthunk“Offered a solution,coModules are also a popular auto-execution solution
The idea is similar in that you write a local method that will be calledgen.next, and the method itself is passed to the callback or promise’s success branch, and the local method is called again when the async endsgen.next, and so iterates until the iterator completes.
async/awaitIt is a syntactic sugar for Generator and autoexecutor, written and implemented in a manner similar to the PROMISE pattern of the CO module.

At the end of this article, thank you for your precious time to read this article. If this article gives you a little help or inspiration, please do not spare your thumbs up and GitHub stars. Your support is the motivation of the author’s continuous creation.

Welcome to follow my public numberThe big front end of the attackThe first time to obtain high quality original ~

“Front-end Advanced Knowledge” series:Juejin. Cn/post / 684490…

“Front-end advanced knowledge” series article source code GitHub address:Github.com/dennis-jian…