Author: Huang Wei

background

Node as the big front edge tool of the team, now use it to do very much, now more is used as the interface layer, the middle layer we usually have a lot of ability, which we use as a server, link the back-end and front-end, forwarding, aggregation, API and so on related operations, there are also a lot of operational, but it’s not the point, The point is that as a server, we have to record the state and behavior of the server. That state is usually called monitoring, and the behavior is called logging

What about the journal section? Write your own or third-party library? Who is more efficient? What features are supported? And so on are the indicators that we need to measure

  • Use FS +stream for logging, and it must be synchronous (for log files)
  • If you are not sure or write by yourself, then use a third party, after all, the third party has been tested by many projects
  • As for who is more efficient (as will be proved below), I need to introduce several libraries log4js, Winston, Bunyan, and pino, which is very similar to Bunyan.
  • In terms of supporting features, both support files and HTTP, and both have varying degrees of customization

Framework of contrast

log4js

Log4js is a log library that I have written about in Java and I feel comfortable with ().

For an introduction to log4JS itself, check out the official website

Log4js supports multiple formats (custom), compression, encoding, reserved logs, and log levels.

This article is based on custom layout test

Import log4js from 'log4js' // other configurations omitted such as adding appender, filename, categorie, etc. // Here we apply the "static string" const resultLog = {a: `1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz`} log4js.addLayout('json', config => event => JSON.stringify(resultLog))Copy the code

To test the layout of the customized JSON name above, we need to write the JSON string to facilitate log parsing.

Const Koa = require(' Koa ') const port = 3000 const addr = '0.0.0.0' var log4js = require('log4js'); const resultLog = { a: `1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz` } // 62 char const app = new Koa() log4js.addLayout('json', config => event => JSON.stringify(event.data[0])) log4js.configure({ appenders: { 'testapd': {filename: './log/test.log', type: 'dateFile', layout: {type: 'json'}, // Use custom layout pattern: '-yyyy-MM-dd', alwaysIncludePattern: false, maxSize: 10485760, // 10M } }, categories: { default: { appenders: ['testapd'], level: 'info' } } }); const logger = log4js.getLogger() app.use(async (ctx, next) => { ctx.body = 'log4js test' logger.info(resultLog) next() }) app.listen(port, addr, () => { console.log(`To see your app, visit http://${addr}:${port}\n`) console.log(`To shut down, press <CTRL> + C at any time.\n`) })Copy the code

As can be seen from the above test, our AVERAGE QPS can be 14500.

winston

Winston, introduced as the second, is the most similar to Log4JS, which we’ll conclude below.

const winston_logger = createLogger({
  level: 'info',
  format: format.combine(format.json()),
  transports: [
    //
    // - Write to all logs with level `info` and below to `combined.log` 
    // - Write all logs error (and below) to `error.log`.
    //
    new transports.File({
      filename: 'error.log',
      level: 'error'
    }),
    new transports.File({
      filename: 'combined.log'
    })
  ]
});

app.use(async (ctx, next) => {
  ctx.body = 'log4js test'
  // logger.info(resultLog)
  winston_logger.log({
    level: 'info',
    message: JSON.stringify(resultLog)
  });
  next()
})
Copy the code

Here’s the rough test code, which simply creates a Logger object

From the above tests, QPS is basically on par with log4js, with an average of 22k less requests handled within 14300,120 seconds and log4js’ ‘120 seconds

bunyan

For this library, it is similar to Winston in terms of usage, and much worse in terms of functionality, mainly in terms of custom formatting. It is not even possible, but it can be very simple and crude.

The test results show that QPS can reach about 12500, and the request processing is about 1500K within 120s

pino

There’s one last pino, when you take a look at the website

Very low overhead Node.js logger, inspired by Bunyan.

In many cases, Pino is over 5x faster than alternatives.

First of all, this library is inspired by Bunyan and claims that Pino is 5 times faster than Bunyan in most cases, which I can live with and definitely have to give it a try.

The results are not far behind Bunyan at high concurrency

Note: The four libraries we tested above are just simple to write files, and there are some differences in the actual scene, for example, we may need to use HTTP, multiple Loggers, multi-core cluster mode, all of which have some influence. In the process of writing the middle layer of node, Use cluster for multi-core (similar to PM) rather than multi-instance. Log writing also needs to be written by the main process, through the process listening/receiving events for log data communication. So there’s a performance drain, there’s other logic, so there’s a bit of a performance drain on running middleware.

Pressure test: Both press and pressed are on the author’s MAC computer with 8GB memory, 2.9ghz I5, 128GB disk, and autocannon pressure test tool

The simple pressure test results are log4JS > Winston > Pino > Bunyan

After the final test in the actual framework, cluster multi-core mode is implemented in the framework, as well as other logical middleware (such as login), the performance order is log4JS > Winston > Bunyan > Pino

Finally, after considering the dimensions of star, recent release time, download times and so on of the four frameworks, log4JS is a safe choice.

As for the optimization scheme of the log, what can be optimized for the node middle layer? In talking about