Koa – BodyParser

series

• [Koa source code] Koa
• [Koa source code] Koa-router
• Koa – BodyParser
• Koa cookie
• [Koa source code learning] KOa-session

preface

In the native HTTP module, the request REq is an instance of HTTP.incomingMessage, which is a readable stream from which we can retrieve the request body. In Koa, the koA-BodyParser module is usually used, which parses the request data and adds it to ctx.request.body, so let’s take a look inside.

Registered middleware

The default export method for the KOA-BodyParser module is as follows:

/* koa-bodyparser/index.js */
module.exports = function (opts) {
  // ...

  // The koa-bodyParser module supports form, JSON, text, AND XML requests. Json and FORM parsing is enabled by default
  var enableTypes = opts.enableTypes || ['json'.'form'];
  var enableForm = checkEnable(enableTypes, 'form');
  var enableJson = checkEnable(enableTypes, 'json');
  var enableText = checkEnable(enableTypes, 'text');
  var enableXml = checkEnable(enableTypes, 'xml');

  // ...

  // Content-type for each Type

  // default json types
  var jsonTypes = [
    'application/json'.'application/json-patch+json'.'application/vnd.api+json'.'application/csp-report',];// default form types
  var formTypes = [
    'application/x-www-form-urlencoded',];// default text types
  var textTypes = [
    'text/plain',];// default xml types
  var xmlTypes = [
    'text/xml'.'application/xml',];// ...

  // Extend content-type
  var extendTypes = opts.extendTypes || {};

  extendType(jsonTypes, extendTypes.json);
  extendType(formTypes, extendTypes.form);
  extendType(textTypes, extendTypes.text);
  extendType(xmlTypes, extendTypes.xml);

  // Return koA-BodyParser middleware
  return async function bodyParser(ctx, next) {
    // ...
  };
};
Copy the code

As you can see, KoA-BodyParser handles form, JSON, Text, XML requests, and the Content-Type of the request can be extended with the extendTypes option to return a middleware BodyParser after the option configuration is processed.

Parse the message

When a request is received from the client, the bodyParser middleware function is executed with the following code:

/* koa-bodyparser/index.js */
module.exports = function (opts) {
  // ...

  return async function bodyParser(ctx, next) {
    if(ctx.request.body ! = =undefined) return await next();
    if (ctx.disableBodyParser) return await next();
    try {
      // Parse the message based on the requested Content-type
      const res = await parseBody(ctx);
      ctx.request.body = 'parsed' in res ? res.parsed : {};
      if (ctx.request.rawBody === undefined) ctx.request.rawBody = res.raw;
    } catch (err) {
      if (onerror) {
        onerror(err, ctx);
      } else {
        throwerr; }}await next();
  };

  async function parseBody(ctx) {
    if (enableJson && ((detectJSON && detectJSON(ctx)) || ctx.request.is(jsonTypes))) {
      return await parse.json(ctx, jsonOpts);
    }
    if (enableForm && ctx.request.is(formTypes)) {
      return await parse.form(ctx, formOpts);
    }
    if (enableText && ctx.request.is(textTypes)) {
      return await parse.text(ctx, textOpts) || ' ';
    }
    if (enableXml && ctx.request.is(xmlTypes)) {
      return await parse.text(ctx, xmlOpts) || ' ';
    }
    return{}; }};Copy the code

As you can see, the bodyParser method first uses the parseBody method. Depending on the content-type of the request, different parse methods are called to parse data from the context. If the parse succeeds, the result is saved to ctx.request.body. Then the next middleware is executed. Let’s take a look at how the parse method parses JSON and forms.

json

The method for handling JSON is defined in the Co-body module, which looks like this:

/* co-body/lib/json.js */
const raw = require('raw-body');
const inflate = require('inflation');

module.exports = async function(req, opts) {
  req = req.req || req;
  opts = utils.clone(opts);

  // defaults
  let len = req.headers['content-length'];
  const encoding = req.headers['content-encoding'] | |'identity';
  if (len && encoding === 'identity') opts.length = len = ~~len;
  opts.encoding = opts.encoding || 'utf8';
  opts.limit = opts.limit || '1mb';
  conststrict = opts.strict ! = =false;

  // The request body is extracted from the REq, the inflate is used to process compressed packets, and the RAW is used to extract content from the readable stream
  const str = await raw(inflate(req), opts);
  try {
    Parse the request message using json.parse
    const parsed = parse(str);
    return opts.returnRawBody ? { parsed, raw: str } : parsed;
  } catch (err) {
    err.status = 400;
    err.body = str;
    throw err;
  }

  function parse(str) {
    if(! strict)return str ? JSON.parse(str) : str;
    // strict mode always return object
    if(! str)return {};
    // strict JSON test
    if(! strictJSONReg.test(str)) {throw new Error('invalid JSON, only supports object and array');
    }
    return JSON.parse(str); }};Copy the code

As you can see, in this method, you first process the configuration option and then call the inflate method, which looks like this:

/* inflation/index.js */
function inflate(stream, options) {
  // ...

  options = options || {}

  // Extract content-encoding from the request header
  var encoding = options.encoding
    || (stream.headers && stream.headers['content-encoding') | |'identity'

  // Determine whether to decompress and use it according to content-encoding. Gzip and deflate are supported here
  switch (encoding) {
  case 'gzip':
  case 'deflate':
    break
  case 'identity':
    return stream
  default:
    var err = new Error('Unsupported Content-Encoding: ' + encoding)
    err.status = 415
    throw err
  }

  // no not pass-through encoding
  delete options.encoding

  // Unzip the data using zlib.Unzip
  return stream.pipe(zlib.Unzip(options))
}
Copy the code

As you can see, if the request body is compressed, the inflate method uncompresses it using zlib.unzip. Then call the RAW method, which is used to extract data from the request. The main code is as follows:

/* raw-body/index.js */
function getRawBody (stream, options, callback) {
  // ...
  // Return a Promise
  return new Promise(function executor (resolve, reject) {
    readStream(stream, encoding, length, limit, function onRead (err, buf) {
      if (err) return reject(err)
      resolve(buf)
    })
  })
}

function readStream (stream, encoding, length, limit, callback) {
  // ...
  // Register the corresponding event on the readable stream to extract data
  // attach listeners
  stream.on('aborted', onAborted)
  stream.on('close', cleanup)
  stream.on('data', onData)
  stream.on('end', onEnd)
  stream.on('error', onEnd)
  // ...
}
Copy the code

As you can see, the readStream method registers the data and end events on the REQ readable stream to extract the data as follows:

function readStream (stream, encoding, length, limit, callback) {
  // ...
  function onData (chunk) {
    if (complete) return

    received += chunk.length

    if(limit ! = =null && received > limit) {
      // ...
    } else if (decoder) {
      buffer += decoder.write(chunk)
    } else {
      buffer.push(chunk)
    }
  }

  function onEnd (err) {
    if (complete) return
    if (err) return done(err)

    if(length ! = =null&& received ! == length) {// ...
    } else {
      var string = decoder
        ? buffer + (decoder.end() || ' ')
        : Buffer.concat(buffer)
      done(null, string)
    }
  }

  function done () {
    // ...
    if (sync) {
      process.nextTick(invokeCallback)
    } else {
      invokeCallback()
    }

    function invokeCallback () {
      // ...
      callback.apply(null, args)
    }
  }
  // ...
}
Copy the code

Each time the data event is triggered, the data is stored in the buffer. When the data in the stream is read, the readable stream will trigger the End event. At this point, the done method will be called and the read data will be returned.

Going back to the JSON method above, once you have the data, the parse method is called, which internally converts the data to a JSON object using Json.parse and returns it. Finally, the requested data can be accessed via ctx.request.body.

form

The methods for handling forms are also defined in the Co-body module, which looks like this:

/* co-body/lib/form.js */
const qs = require('qs');
const raw = require('raw-body');
const inflate = require('inflation');

module.exports = async function(req, opts) {
  req = req.req || req;
  opts = utils.clone(opts);
  const queryString = opts.queryString || {};

  // keep compatibility with qs@4
  if (queryString.allowDots === undefined) queryString.allowDots = true;

  // defaults
  const len = req.headers['content-length'];
  const encoding = req.headers['content-encoding'] | |'identity';
  if (len && encoding === 'identity') opts.length = ~~len;
  opts.encoding = opts.encoding || 'utf8';
  opts.limit = opts.limit || '56kb';
  opts.qs = opts.qs || qs;

  const str = await raw(inflate(req), opts);
  try {
    // Use the QS module to parse the request data
    const parsed = opts.qs.parse(str, queryString);
    return opts.returnRawBody ? { parsed, raw: str } : parsed;
  } catch (err) {
    err.status = 400;
    err.body = str;
    throwerr; }};Copy the code

As you can see, the body logic is the same as json when processing the form. The only difference is that after retrieving the request data, the qs.parse method is used to parse the data. Finally, the data is also placed on ctx.request.body.

conclusion

Koa-bodyparser registers the events corresponding to the readable stream on reQ to get the requested data, parses it in different ways according to the content-type of the request, and then binds the data to ctx.request.body. In the following middleware, You can access the data directly through ctx.request.body.