[1]- KoA and Koa-compose

On the second dig pit, koA2.x related source code analysis of the first. I have to say that KOA is a very lightweight and elegant HTTP framework, especially since the introduction of CO was removed after 2.x to make the code clearer.

Express and KOA were developed by the same people, and KOA was very small compared to Express. Because Express is a large and complete HTTP framework, middleware like router is built in to handle it. In KOA, middleware with similar functions is completely removed. Koa-compose was built into koA in the early days, and now it is also separated out. Koa retains a simple middleware integration, HTTP request processing, as a functional middleware framework with little logic of its own. Koa-compose is one of the most critical tools for integration middleware, an implementation of the Onion model, so look at the two together.

Koa basic structure

.├ ── Application.js ├─ Request.js ├─ response.js ├─ context.jsCopy the code

The implementation of the entire KOA framework is simply split into four files.

As simulated in the previous note, an object is created to register middleware and listen for HTTP services, which is what application.js is doing. The meaning of a framework is that within the framework, we should do things according to the rules of the framework, and also the framework will provide us with some easier ways to fulfill the requirements. An encapsulation of the request and response parameters of the HTTP.createserver callback to simplify common operations. For example, some of our operations on headers might look like this in a native HTTP module:

/ / get the content-type
request.getHeader('Content-Type')

/ / set the content-type
response.setHeader('Content-Type'.'application/json')
response.setHeader('Content-Length'.'18')
// Alternatively, ignore the preceding statusCode and set multiple headers
response.writeHead(200, {
  'Content-Type': 'application/json'.'Content-Length': '18'
})
Copy the code

In KOA, it can be handled like this:

/ / get the content-type
context.request.get('Content-Type')

/ / set the content-type
context.response.set({
  'Content-Type': 'application/json'.'Content-Length': '18'
})
Copy the code

Simplify some operations on request and response by encapsulating them in request.js and response.js files. But there’s also a problem with getting or setting the header. It’s a bit more hierarchical. You have to find the request and response in the context before you can do anything about it. So, KOA used node-delegates to further simplify these steps, delegates request. Get, Response. Set all to the context. In other words, the post-proxy operation looks like this:

context.get('Content-Type')

/ / set the content-type
context.set({
  'Content-Type': 'application/json'.'Content-Length': '18'
})
Copy the code

Get the Header, set the Header, don’t worry about writing request.setHeader anymore, all in one go, use context.js to integrate the behavior of request.js with response.js. Context.js also provides other utility functions, such as Cookie operations.

Application introduces the context, which integrates the request and response functions. The functions of the four files are already clear:

file	desc
applicaiton	Middleware management,http.createServerThe callback processing is generatedContextTake the parameters of this request and invoke the middleware
request	forhttp.createServer -> requestFunctional encapsulation
response	forhttp.createServer -> responseFunctional encapsulation
context	integrationrequestwithresponseSome of the features and provide some additional features

In the code structure, onlyapplicationThe foreignkoaIt is to use theClassThe way the other three files are thrown is a common oneObject.

Take a complete process to explain

Create a service

First, we need to create an HTTP service. Creating the service in KOA2.x is slightly different from koA1.x and requires instantiation:

const app = new Koa()
Copy the code

In the instantiation process, koA actually does only a limited thing, creating a few instance attributes. Copy the context, request, and Response into the instance through Object.create.

this.middleware = [] // The most critical instance attribute

// Used to create context after a request is received
this.context = Object.create(context)
this.request = Object.create(request)
this.response = Object.create(response)
Copy the code

After the instantiation is complete, we need to register the middleware to implement our business logic. As mentioned above, koA is only used as a middleware integration and request listener. So instead of providing router.get or router.post operations like express does, there is only a use() that is closer to http.createserver. The next step is to register the middleware and listen for a port number to start the service:

const port = 8000

app.use(async (ctx, next) => {
  console.time('request')
  await next()
  console.timeEnd('request')
})
app.use(async (ctx, next) => {
  await next()
  ctx.body = ctx.body.toUpperCase()
})

app.use(ctx= > {
  ctx.body = 'Hello World'
})

app.use(ctx= > {
  console.log('never output')
})

app.listen(port, () => console.log(` Server run as http://127.0.0.1:${port}`))
Copy the code

If you look at the source code of application.js, you can see that the most commonly used methods exposed to the outside world are basically use and listen. One is used to load middleware and the other is used to listen on ports and start services.

These two functions actually do not have much logic. In use, they only determine whether the passed parameter is a function, and in the 2.x version, some special processing is done for the Generator function to convert it into a function in the form of Promise. And push it into the Middleware array created in the constructor. This is a tool to transition from 1.x to 2.x, in version 3.x it will remove Generator support directly. Koa-convert also references co and KOa-compose inside koA-convert, so I won’t go into details here.

Listen is much simpler, simply calling http.createServer to create a service and listening for the corresponding port. One detail is that the callback passed in createServer is the return value of another method call from the KOA instance. This method is the actual callback processing, and Listen is just a shortcut to the HTTP module. This is intended for use with socket. IO, HTTPS, or some other HTTP module. This means that koA can be easily accessed as long as it provides consistent behavior with HTTP modules.

listen(... args) { debug('listen')
  const server = http.createServer(this.callback())
  returnserver.listen(... args) }Copy the code

Merge middleware with Koa-compose

So let’s look at the implementation of callback:

callback() {
  const fn = compose(this.middleware)

  if (!this.listenerCount('error')) this.on('error'.this.onerror)

  const handleRequest = (req, res) = > {
    const ctx = this.createContext(req, res)
    return this.handleRequest(ctx, fn)
  }

  return handleRequest
}
Copy the code

The first step inside the function is to deal with the middleware, converting an array of middleware to the onion model format we want. Koa-compose is the core component used here

In fact, it is functionally similar to CO, except that the logic of CO processing Generator functions is completely removed. The code of CO itself is only one or two hundred lines, so the reduced KOa-compose code is only 48 lines.

We know that async actually looks like this when stripped of its syntactic sugar:

async function func () {
  return 123
}

/ / = = >

function func () {
  return Promise.resolve(123)}// or
function func () {
  return new Promise(resolve= > resolve(123))}Copy the code

So take the use code above, for example, and koa-compose actually gets this argument:

[
  function (ctx, next) {
    return new Promise(resolve= > {
      console.time('request')
      next().then((a)= > {
        console.timeEnd('request')
        resolve()
      })
    })
  },
  function (ctx, next) {
    return new Promise(resolve= > {
      next().then((a)= > {
        ctx.body = ctx.body.toUpperCase()
        resolve()
      })
    })
  },
  function (ctx, next) {
    return new Promise(resolve= > {
      ctx.body = 'Hello World'
      resolve()
    })
  },
  function (ctx, next) {
    return new Promise(resolve= > {
      console.log('never output')
      resolve()
    })
  }
]
Copy the code

As indicated in the output of the fourth function, the fourth middleware will not be executed because the third middleware does not call next, so it would be interesting to implement an Onion model like this. First of all, leaving aside the constant CTX, the core of the Implementation of the Onion model lies in the next processing. Because next is your key to the next layer of middleware, you only get to the next layer of middleware if you manually trigger it. Then we also need to ensure that next will resolve after the middleware completes, returning to the previous layer of middleware:

return function (context, next) {
  // last called middleware #
  let index = - 1
  return dispatch(0)
  function dispatch (i) {
    if (i <= index) return Promise.reject(new Error('next() called multiple times'))
    index = i
    let fn = middleware[i]
    if (i === middleware.length) fn = next
    if(! fn)return Promise.resolve()
    try {
      return Promise.resolve(fn(context, dispatch.bind(null, i + 1)))}catch (err) {
      return Promise.reject(err)
    }
  }
}
Copy the code

So with these two points in mind, the code above should be clear:

1. Next is used to go to the next middleware
2. Next triggers a callback to notify the previous middleware when the current middleware execution is complete, but only if the internal middleware execution is complete (resolved)

You can see that koa-compose actually returns a self-executing function after being called. At the beginning of the execution function, determine the subscript of the current middleware to prevent multiple calls to Next in one middleware. Because multiple calls to Next would result in multiple executions of the next middleware, breaking the Onion model.

The second is that compose actually provides a callback after the Onion model has been fully executed, an optional parameter that is essentially the same as the then processing that follows the call to Compose.

As mentioned above, next is the key to the next piece of middleware, as can be seen in this application of the Currization function:

Promise.resolve(fn(context, dispatch.bind(null, i + 1)))
Copy the code

It will bind itself with index parameter and pass it into this middleware as the second parameter of calling function, that is, next. The effect is just like calling Dispatch (1), which is the implementation of onion model. If fn calls an async function, then the outer promise. resolve will not trigger resolve until the inner async executes resolve, for example:

Promise.resolve(new Promise(resolve= > setTimeout(resolve, 500))).then(console.log) // Console. log is triggered after 500ms
Copy the code

P.S. A pit that switches from koa1.x to koa2.x. Co does special processing to the array, wrapping it with promise.all, but koA2. x does not. So if you want to do asynchronous operations on an array in middleware, be sure to manually add promise. all, or await* in the draft.

// koa1.x
yield [Promise.resolve(1), Promise.resolve(2)]              / / [1, 2]

// koa2.x
await [Promise.resolve(1), Promise.resolve(2)]              // [<Promise>, <Promise>]

/ / = = >
await Promise.all([Promise.resolve(1), Promise.resolve(2)]) / / [1, 2]
await* [Promise.resolve(1), Promise.resolve(2)]             / / [1, 2]
Copy the code

Receives the request and processes the return value

With the above code, a KOA service is now up and running, and the next step is to access it to see the effect. Upon receiving a request, KOA takes the previously mentioned context and request and Response to create the context used for the request. In koA1.x, the context is bound to this, whereas in koA2.x it is passed in as the first argument. This is probably because Generator cannot use arrow functions and async functions can use arrow functions

In summary, we created a request context using the three modules mentioned above. Basically, we assigned values to each of them, and the code didn’t stick. There wasn’t much logic, except for one small detail that was interesting:

request.response = response
response.request = request
Copy the code

This creates a reference relationship between the two, whereby a response can be obtained from request and a request can be obtained from response. And this is a recursive reference, something like this:

let obj = {}

obj.obj = obj

obj.obj.obj.obj === obj // true
Copy the code

As mentioned above, during context creation, a large number of request and response properties and methods are delegated to itself. If you are interested, you can browse the source code for yourself. Koa. Js | context. Js the delegate implementation is relatively simple, and by taking the original attributes, and then put a reference, in its own property is triggered when the call the corresponding reference, similar to a private version of the Proxy, looking forward to the follow-up to be able to use the Proxy to replace it.

We then pass the generated context as a parameter to the koa-compose generated onion. Because the Onion will definitely return the result no matter what, we also need to have a FINISHED processing at the end, doing something like converting ctx.body to data for output.

Koa uses a number of get and set accessors to implement functions, such as the most common ctx.body = ‘XXX’, which is the set body from Response. This is probably the most logically complex method in request and Response. There are a lot of things to handle, such as changing the status code of the request to 204 when the body content is empty and removing the headers that are useless. And if the status code is not specified manually, it defaults to 200. It even determines whether the content-type should be HTML or plain text based on the currently passed argument:

// string
if ('string'= =typeof val) {
  if (setType) this.type = /^\s*</.test(val) ? 'html' : 'text'
  this.length = Buffer.byteLength(val)
  return
}
Copy the code

For example, if you want to use KOA to implement static resource download function, you can also call ctx.body directly to assign values. All things are handled in response.js for you:

// stream
if ('function'= =typeof val.pipe) {
  onFinish(this.res, destroy.bind(null, val))
  ensureErrorHandler(val, err => this.ctx.onerror(err))

  // overwriting
  if (null! = original && original ! = val)this.remove('Content-Length')

  if (setType) this.type = 'bin'
  return
}

// this is an example of this code
let stream = fs.createReadStream('package.json')
ctx.body = stream

// Processing in set body
onFinish(res, () => {
  destory(stream)
})

stream.pipe(res) // Make the response receive stream after the Onion model is fully executed
Copy the code

OnFinish is used to listen for the end of the stream and deStory is used to close the stream

The rest of the accessors are basically a wrapper around some common operations, such as those for QueryString. In the case of the native HTTP module, processing parameters in the URL requires importing additional packages of its own, the most common being QueryString. Koa also introduced this module internally. So the thrown query looks something like this:

get query() {
  let query = parse(this.req).query
  return qs.parse(query)
}

// use
let { id, name } = ctx.query // Because get Query is also proxied to the context, it can be referenced directly
Copy the code

Parse is the ParseURL library used to extract query parameters from request

Or the encapsulation of cookies, which are also built into the most popular cookies. The Cookie object is instantiated only when get cookies are triggered for the first time to hide these tedious operations from the user:

get cookies() {
  if (!this[COOKIES]) {
    this[COOKIES] = new Cookies(this.req, this.res, {
      keys: this.app.keys,
      secure: this.request.secure
    })
  }
  return this[COOKIES]
}

set cookies(_cookies) {
  this[COOKIES] = _cookies
}
Copy the code

So using cookies in KOA looks like this:

this.cookies.get('uid')

this.cookies.set('name'.'Niko')

// If you don't want to use the cookies module, you can assign the cookies you want to use
this.cookies = CustomeCookie

this.cookies.mget(['uid'.'name'])
Copy the code

This is because there is a judgment in get cookies that if there is no instance of Cookie available, it will be de-instantiated by default.

The Onion model performs some of the completed operations

A koA request flows like this: All middleware in the Onion is executed, and after execution, there is a callback function. This callback determines what data to return to the client based on what the middleware is doing during execution. The body and ctx.status parameters are processed. The processing of streams, including those mentioned earlier, is here:

if (body instanceof Stream) return body.pipe(res) // Wait until this is done before calling our 'onFinish' processing in 'set Body' above
Copy the code

If false, do nothing and return:

if(! ctx.writable)return
Copy the code

Response.finished is set to true after response.end() is triggered, that is, the request is finished. A bug was also handled in the accessor where the request was returned but the socket was still not closed:

get writable() {
  // can't write any more after response finished
  if (this.res.finished) return false

  const socket = this.res.socket
  // There are already pending outgoing res, but still writable
  / / https://github.com/nodejs/node/blob/v4.4.7/lib/_http_server.js#L486
  if(! socket)return true
  return socket.writable
}
Copy the code

This is a disadvantage for KOA vs. Express, because KOA uses an onion model. Assigning a return value using ctx.body = ‘XXX’ will result in the final call to Response. end after the onion has finished executing. In the callbacks described above, express gives middleware freedom to control when data is returned:

// express.js
router.get('/'.function (req, res) {
  res.send('hello world')

  // Do some other processing after sending the data
  appendLog()
})

// koa.js
app.use(ctx= > {
  ctx.body = 'hello world'

  // However, it still occurs before sending data
  appendLog()
})
Copy the code

The good news is that we can send data directly by calling the native Response object. When we manually call Response. end (Response. finished === true), it means that the final callback will be skipped.

app.use(ctx= > {
  ctx.res.end('hello world')

  // Do some other processing after sending the data
  appendLog()
})
Copy the code

Exception handling

The entire koA request is actually a Promise, so the listener behind the Onion model is handled not only resolve, but also Reject. Any bug in the loop will cause the subsequent middleware and the middleware waiting for the callback to terminate and jump directly to the nearest exception handling module. So, if you have middleware like interface time statistics, be sure to execute next in a try-catch:

app.use(async (ctx, next) => {
  try {
    await next()
  } catch (e) {
    console.error(e)
    ctx.body = 'error' // Because the internal middleware does not catch the exception, it is thrown here
  }
})

app.use(async (ctx, next) => {
  let startTime = new Date(a)try {
    await next()
  } finally {
    let endTime = new Date(a)// Throws an exception, but does not affect normal output here
  }
})

app.use(ctx= > Promise.reject(new Error('test')))
Copy the code

P.S. If the exception is caught, the subsequent response continues:

app.use(async (ctx, next) => {
  try {
    throw new Error('test')}catch (e) {
    await next()
  }
})

app.use(ctx= > {
  ctx.body = 'hello'
})

/ / curl 127.0.0.1
// > hello
Copy the code

If your middleware does not catch an exception, it will go to the default exception handling module. In the default exception module, there is basically some processing for statusCode, and some default error display:

const code = statuses[err.status]
const msg = err.expose ? err.message : code
this.status = err.status
this.length = Buffer.byteLength(msg)
this.res.end(msg)
Copy the code

Statuses is a third-party module that contains information about various HTTP codes: Statuses recommends that the outmost middleware do their own exception handling, because the default error message is a bit ugly (plain text), it is better to do their own handling to the exception handling page, and to avoid interface parsing failures due to default exception messages.

Redirect Considerations

To perform a 302 operation (commonly known as a redirect) in a native HTTP module, you need to do this:

response.writeHead(302, {
  'Location': 'redirect.html'
})
response.end()
// or
response.statusCode = 302
response.setHeader('Location'.'redirect.html')
response.end()
Copy the code

Koa also has redirect encapsulation. You can call the redirect function directly to complete the redirect, but note that response.end() is not directly triggered after the redirect is called. It just adds a statusCode and Location:

redirect(url, alt) {
  // location
  if ('back' == url) url = this.ctx.get('Referrer') || alt || '/'
  this.set('Location', url)

  // status
  if(! statuses.redirect[this.status]) this.status = 302

  // html
  if (this.ctx.accepts('html')) {
    url = escape(url)
    this.type = 'text/html charset=utf-8'
    this.body = `Redirecting to <a href="${url}">${url}</a>.`
    return
  }

  // text
  this.type = 'text/plain charset=utf-8'
  this.body = `Redirecting to ${url}. `
}
Copy the code

The rest of the code will continue to execute, so it is recommended that you end the request manually after the redirect, or return the request. Otherwise, status and body assignments will cause some weird problems.

app.use(ctx= > {
  ctx.redirect('https://baidu.com')

  // It is recommended to return directly

  // The following code is still executing
  ctx.body = 'hello world'
  ctx.status = 200 // The statusCode change causes the redirect to fail
})
Copy the code

reporter

Koa is a very interesting framework, in the process of reading the source code, actually found some minor problems:

1. Multiple people working together to maintain a piece of code can really show that people have different coding styles, for exampletypeof val ! == 'string'and'number' == typeof codeTwo styles, obviously. 2333
2. Delegate calls don’t look very good when you have a lot of properties, long chain calls, but they look better when you have loops

However, KOA is still a great framework for reading source code to learn from, and these are minor details that don’t matter.

Koa and koa-compose compose compose

• koaRegister middleware, registerhttpService, generate request context to invoke middleware, process middleware’s operations on context objects, and return data to terminate the request
• koa-composeConverts the middleware collection in the array into a serial call and provides the key (next) to jump to the next middleware, and to listennextGets notification of completion of internal middleware execution