Quick navigation
[Logger-Custom]Demand background[Logger-Custom]Custom logging plug-in development[Logger-Custom]Project extension[Logger-Custom]The project application[ContextFormatter]ContextFormatter Custom log format[Logrotator]Log cutting
Demand background
Achieve full link log tracking, facilitate log monitoring, problem, screening, interface response time data statistics, etc., the first service API interface receives the caller’s request, according to the caller and traceId, during the time of invocation chain processing business, if you need to print log, logging information in accordance with the contract specifications for printing, and record traceId, Implement log link tracing.
- Log Path Convention
/var/logs/${projectName}/bizLog/${projectName}-yyyyMMdd.log
Copy the code- Log format conventions
Log Time []traceId[] Server IP address [] Client IP address [] Log level [] Log contentCopy the codeUsing egg.js framework and Egg-Logger middleware, it was found in the implementation process that printing according to the above log format could not meet the requirements (at least I haven’t found the implementation method yet). If we want to implement it by ourselves, we may have to make wheels by ourselves. Fortunately, the official Egg-Logger middleware provides the custom log extension function. Refer to advanced custom log, it also provides log splitting, multi-process log processing and other functions.
Egg-logger provides a variety of transmission channels. Our requirement is to store the requested business logs in a customized format, mainly using fileTransport and consoleTransport channels to print logs to files and terminals respectively.
Custom logging plug-in development
Based on egg- Logger custom development of a plug-in project, refer to plug-in development, the following egg-Logger-custom as the project, show the core code writing
- Write the logger. Js
egg-logger-custom/lib/logger.js
const moment = require('moment');
const FileTransport = require('egg-logger').FileTransport;
const utils = require('./utils');
const util = require('util');
/** * inherits FileTransport */
class AppTransport extends FileTransport {
constructor(options, ctx) {
super(options);
this.ctx = ctx; // Get context for each request
}
log(level, args, meta) {
// Get a custom format message
const customMsg = this.messageFormat({
level,
});
// Prints an Error stack for Error messages
if (args[0] instanceof Error) {
const err = args[0) | | {}; args[0] = util.format('%s: %s\n%s\npid: %s\n', err.name, err.message, err.stack, process.pid);
} else {
args[0] = util.format(customMsg, args[0]);
}
// This is necessary, otherwise the log file will not be written
super.log(level, args, meta);
}
/** * Custom message format * can be customized according to your business needs * @param {String} level */
messageFormat({
level
}) {
const { ctx } = this;
const params = JSON.stringify(Object.assign({}, ctx.request.query, ctx.body));
return [
moment().format('YYYY/MM/DD HH:mm:ss'),
ctx.request.get('traceId'),
utils.serviceIPAddress,
utils.clientIPAddress(ctx.req),
level,
].join(utils.loggerDelimiter) + utils.loggerDelimiter;
}
}
module.exports = AppTransport;
Copy the code- tool
egg-logger-custom/lib/utils.js
const interfaces = require('os').networkInterfaces();
module.exports = {
/** * Log separator */
loggerDelimiter: '[]'./** * Obtain the current server IP address */
serviceIPAddress: (() = > {
for (const devName in interfaces) {
const iface = interfaces[devName];
for (let i = 0; i < iface.length; i++) {
const alias = iface[i];
if (alias.family === 'IPv4'&& alias.address ! = ='127.0.0.1' && !alias.internal) {
returnalias.address; }}}}) (),/** * get the current request client IP * insecure writing */
clientIPAddress: req= > {
const address = req.headers['x-forwarded-for'] | |// Check whether there is a reverse proxy IP address
req.connection.remoteAddress || // Determine the remote IP address of the connection
req.socket.remoteAddress || // Determine the IP address of the socket at the back end
req.connection.socket.remoteAddress;
return address.replace(/::ffff:/ig.' ');
},
clientIPAddress: ctx= > {
returnctx.ip; }},Copy the codeNote: Please do not use the above method to obtain the IP address of the current request client if you need to restrict the IP address of the user. Please refer to the popular science article: How to Forge and Obtain the Real IP address of the user. In egg.js you can also get it from ctx. IP, see pre-proxy mode.
- Initialize the Logger
egg-logger-custom/app.js
Copy the codeconst Logger = require('egg-logger').Logger;
const ConsoleTransport = require('egg-logger').ConsoleTransport;
const AppTransport = require('./app/logger');
module.exports = (ctx, options) = > {
const logger = new Logger();
logger.set('file'.new AppTransport({
level: options.fileLoggerLevel || 'INFO'.file: `/var/logs/${options.appName}/bizLog/${options.appName}.log`,
}, ctx));
logger.set('console'.new ConsoleTransport({
level: options.consoleLevel || 'INFO',}));return logger;
}
Copy the codeThe above is good for logging custom format development, but if you have actual business needs you can use it as an NPM middleware within your team.
Project extension
After the custom logging middleware is packaged, we need to take a step further in the actual project application. Egg provides framework extensions, including five: Application, Context, Request, Response, and Helper can be customized. For logging, we need to record the traceId carried by the current Request for a link trace. The Context (which is Koa’s request Context) extension is required.
New app/extend/context.js file
const AppLogger = require('egg-logger-custom'); // Middleware defined above
module.exports = {
get logger() { // The name can also be customLogger
return AppLogger(this, {
appName: 'test'.// Project name
consoleLevel: 'DEBUG'.// Level of terminal logs
fileLoggerLevel: 'DEBUG'.// File log level}); }}Copy the codeSuggestion: For the log level, use the configuration center such as Consul to configure the log level, and set the log level to INFO when online. When you need to troubleshoot production problems, you can dynamically enable the DEBUG mode. About Consul you can follow the service I wrote earlier to register and discover Consul series
The project application
The complete stack information of Error logs is directly recorded and output to the errorLog. To ensure that exceptions can be traced, it is necessary to ensure that all exceptions thrown are of the Error type, because only the Error type carries stack information to locate the problem.
const Controller = require('egg').Controller;
class ExampleController extends Controller {
async list() {
const { ctx } = this;
ctx.logger.error(new Error('Program exception! '));
ctx.logger.debug('test');
ctx.logger.info('test'); }}Copy the codeThe final log format is as follows:
2019/05/30 01:50:21 [] d373c38a b36 - b931-1-344 - b - 4 e8981aef14f [] 192.168.1.20 221.69.245.153 [] [] INFO [] testCopy the codeContextFormatter Custom log format
The latest version of egg-Logger supports custom log formats through the contextFormatter function, see previous PR: Support contextFormatter #51
The application is also simple by configuring the contextFormatter function. Here is the simple application
config.logger = {
contextFormatter: function(meta) {
console.log(meta);
return [
meta.date,
meta.message
].join('[]')},... };Copy the codeSame as before in your business where you need to print logs
ctx.logger.info('This is a test.');
Copy the codeThe following output is displayed:
[] This is a test dataCopy the codeLog cutting
The framework provides egg-Logrotator middleware, the default cutting is daily cutting, other ways can refer to the official website to configure.
- The framework default log path
An egg – logger module lib/egg/config/config. Default. Js
config.logger = {
dir: path.join(appInfo.root, 'logs', appInfo.name),
...
};
Copy the code- Custom log directory
It’s easy to redefine the logger dir path in the project configuration file as we want
config.logger = {
dir: /var/logs/test/bizLog/
}
Copy the codeIs that ok? ${projectName}- YYYYMMDd.log = ${projectName}- YYYYMMDd.log = ${projectName}- YYYYMMDD.log = ${projectName}- YYYYMMDD.log = ${projectName}- YYYYMMDD.log = ${projectName}- YYYYMMDD.log = ${projectName}- YYYYMMDD.log
Github.com/eggjs/egg-l…
_setFile(srcPath, files) {
// don't rotate logPath in filesRotateBySize
if (this.filesRotateBySize.indexOf(srcPath) > - 1) {
return;
}
// don't rotate logPath in filesRotateByHour
if (this.filesRotateByHour.indexOf(srcPath) > - 1) {
return;
}
if(! files.has(srcPath)) {// allow 2 minutes deviation
const targetPath = srcPath + moment()
.subtract(23.'hours')
.subtract(58.'minutes')
.format('.YYYY-MM-DD'); // Log format definition
debug('set file %s => %s', srcPath, targetPath); files.set(srcPath, { srcPath, targetPath }); }}Copy the code- Log splitting extension
The middleware egg-Logrotator reserved an extension interface. For custom log file names, you can use app. Logrotator provided by the framework to make a custom.
app/schedule/custom.js
const moment = require('moment');
module.exports = app= > {
const rotator = getRotator(app);
return {
schedule: {
type: 'worker'.// only one worker run this task
cron: '1 0 0 * * *'.// run every day at 00:00
},
async task() {
awaitrotator.rotate(); }}; };function getRotator(app) {
class CustomRotator extends app.LogRotator {
async getRotateFiles() {
const files = new Map(a);const srcPath = `/var/logs/test/bizLog/test.log`;
const targetPath = `/var/logs/test/bizLog/test-${moment().subtract(1.'days').format('YYYY-MM-DD')}.log`;
files.set(srcPath, { srcPath, targetPath });
returnfiles; }}return new CustomRotator({ app });
}
Copy the codeAfter segmentation, the file is displayed as follows:
$ ls -lh /var/logs/test/bizLog/
total 188K
-rw-r--r-- 1 root root 135K Jun 1 11:00 test-2019-06-01.log
-rw-r--r-- 1 root root 912 Jun 2 09:44 test-2019-06-02.log
-rw-r--r-- 1 root root 40K Jun 3 11:49 test.log
Copy the codeExtension: Based on the above log format, you can use ELK to collect, analyze and retrieve logs.
Author: you may link: www.imooc.com/article/287… Source: MOOC
Recommended reading
- Nodejs server stack: www.nodejs.red
- Public id: Nodejs Technology stack