This topic describes Java logs

First, initial log recording

Think back to when you first started logging in Java. We usually use system.out.println () to output debug logs, system.err.println () to output error logs, and e.prinintStackTrace () to output exception stacks.

In fact, this is how it was used before logging frameworks existed; Now I’m told not to use it for logging information. Has anyone given much thought to why the logging framework prevents the use of these primitive recording methods?

But there’s a rule in sonar that explains why in great detail:

Standard outputs should not be used directly to log anything

  • The smell
  • The main
  • The main code

When logging a message there are several important requirements which must be fulfilled:

  • The user must be able to easily retrieve the logs
  • The format of all logged message must be uniform to allow the user to easily read the log
  • Logged data must actually be recorded
  • Sensitive data must only be logged securely

If a program directly writes to the standard outputs, there is absolutely no way to comply with those requirements. That’s why defining and using a dedicated logger is highly recommended.

That is, the standard output stream was no longer sufficient for our logging needs, and a logging framework was born.

2. Overview of logging framework evolution

The framework we currently use is a combination of SLF4J and LogBack. Have you ever wondered why we use two logging frameworks and what their responsibilities are?

1.log4j

In the early years, the Log4J framework was used for logging and the following code was used for logging

importorg.apache.log4j.Logger; \\ omit Logger Logger = logger.getLogger (test.class); logger.trace("trace"); \ \ ellipsisCopy the code

2. J.U.L

Later, however, J.U.L(java.util.logging) was added to the JDK to introduce an official logging framework. This is someone who wants to change the logging framework in their project to J.U.L. What do they need to do? Find all the code that uses the Log4j API and manually change it line by line to the J.U.L API, i.e

importJava. Util. Logging. The Logger; \\ Omit Logger loggger = logger.getLogger (test.class.getName ()); logger.finest("finest"); \ \ ellipsisCopy the code

3. J.C.L

As you can see, switching logging frameworks is very inconvenient, and what if a better logging framework comes along in the future. This is when Apache’s JCL (Commons-Logging) was born. JCL is a Log Facade (logging Facade framework) that provides only the Log API, not the Implementation, and then Adapter to use Log4j or JUL as Log Implementation. It’s the logging frameworks behind the scenes (Log4j, J.U.L) that do the real work. The goal is that when a better logging framework comes along later, developers can switch to the new logging framework without changing the code (that is, without switching the log API) and simply change the configuration.

4. SLF4J & Logback

Things seem to have been settled here, the system of Log Facade + Log two-layer framework has been completed. The framework also has the four logging capabilities described above.

But the logging facade framework we use today is usually SLF4J, not the original JCL. So what are the superior qualities of SLF4J that made us choose it? An important one is the placeholder pattern in SLF4J, which reduces the overhead of string concatenation to some extent.

JCL

if (logger.isDebugEnabled()) {
    logger.debug("start process request, url:" + url);
}
Copy the code

SLF4J writing:

logger.debug("start process request, url:{}", url);
Copy the code

Logback is derived from Log4J, adding asynchronous Logger, filter and other features.

5. Log4j2

Including logging facade framework log4J-API and logging framework log4J-core, developed by the people who maintain Log4J and not compatible with log4j

Here’s a diagram of the relationship between the two frames to get a feel for it:

The following uses SLF4J and Logback as examples to describe the use of the logging framework:

Third, Logback

Logback is intended as a successor to the popular log4j project

The core configuration of logback is in the logback. XML file.


      
<configuration debug="false" scan="true" scanPeriod="30 seconds">
	<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
	<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
		<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
			<level>${logback.stdout.level}</level>
		</filter>
		<Target>System.out</Target>
		<encoder>
			<pattern>
			   %d{yyyy-MM-dd HH:mm:ss.SSS}[%-5level][%thread][%logger{20}:%L] %msg%n
			</pattern>
		</encoder>
	</appender>
	<appender name="R"
		class="ch.qos.logback.core.rolling.RollingFileAppender">
		<File>${LOG_HOME}/${SYSTEM_NAME}_stdout_${ip}_${port}.log</File>
		<encoder>
			<pattern>
			   %d{yyyy-MM-dd HH:mm:ss.SSS}[%-5level][%thread][%logger{20}:%L] %msg%n
			</pattern>
		</encoder>
		<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
			<fileNamePattern>${LOG_HOME}/${SYSTEM_NAME}_stdout_${ip}_${port}.%d.log
			</fileNamePattern>
		</rollingPolicy>
	</appender>
	<logger name="org.springframework" level="error" />
	<logger name="org.apache" level="error" />
	<root level="${logback.root.level}">
		<appender-ref ref="stdout" />
		<appender-ref ref="R" />
	</root>
</configuration>
Copy the code

  • Config file syntax:

    • Structure:<configuration>Element, zero or more<appender>Element, zero or more<logger>Element, at most one<root>The element
    • The label name is case-sensitive

  • < Configuration > : Logback delegates writing log events to a component called appender.

    • debug: Controls whether to output some status information about the Logback itself
    • scan="true" When the configuration file is changed, the system automatically loads the configuration. The default scanning interval is 1 minute
    • scanPeriod="30 seconds"The system automatically scans the configuration every 30 seconds to check whether there is any change


  • : status message listener

    • OnConsoleStatusListener: Same as debug=true, prints the status information to the console

    • protected PrintStream getPrintStream(a) {
    return System.out;
}
Copy the code

    • NopStatusListener: discards all status information

    • public void addStatusEvent(Status status) {
    // nothing to do
}
Copy the code

    • OnErrorConsoleStatusListener: in the console print error messages

    • protected PrintStream getPrintStream(a) {
    return System.err;
}
Copy the code

  • The < appender > :

    • name: The name of the appender used for the logger or root reference below
    • class: Concrete class used by the appender
      • ch.qos.logback.core.ConsoleAppender: Appends log events to the console, which is passedSystem.outorSystem.errTo do the output. The former is passed by default.
      • ch.qos.logback.core.rolling.RollingFileAppender: Outputs log events to a file. throughfileTo specify the target file. After certain criteria are met, the log is output to another file.
    • filter
      • ch.qos.logback.classic.filter.ThresholdFilter: Filters events based on a given threshold. If the level of the event is equal to or above a given threshold, when calleddecide()When,ThresholdFilterWill returnNEUTRAL. However, events below the threshold will be rejected.
      • ch.qos.logback.classic.filter.LevelFilter: The event whose level is equal to the configured level will be calledonMatchProperty configuration, unequal will be calledOnMismatchThe configuration properties
    • Target
      • ConsoleAppenderIf yes, set toSystem.out 或 System.err. The default isSystem.out
    • encoder:encoderConverts log events to a byte array while writing the byte array to aOutputStreamIn the.
      • pattern:PatternLayoutEncoder
    • rollingPolicy
      • TimeBasedRollingPolicy:
      • SizeAndTimeBasedRollingPolicy:

  • The < logger > :

    • name
    • level: TRACE, DEBUG, INFO, WARN, ERROR, ALL, OFF, INHERITED, NULL. whenlevelWhen INHERITED or NULL is used, the logger is forced to inherit the previous level
    • <appender-ref>: Uses an appender that inherits the appender of the previous level


  • : similar to

    , the name and additivity of root logger cannot be set, and the level cannot be set to INHERITED and NULL.

Fourth, the SLF4J

  • To create a Logger
    • Logger logger = LoggerFactory.getLogger(HelloWorld.class);
    • Integrate lombok plug-ins that can be used on classes@Slf4jcreate
  • use
    • logger.info("some message");
    • logger.error("some message", e)
    • Placeholder mode:logger.debug("some message: {}", message, e)
    • Fluent Logging API (SLF4J-API version 2.0.0 or higher) :
      • logger.atInfo().log("some message")
      • logger.atDebug().addArgument(newT).addArgument(oldT).log("Temperature set to {}. Old temperature was {}.");
      • logger.atDebug().addKeyValue("oldT", oldT).addKeyValue("newT", newT).log("Temperature changed.");= =logger.debug("oldT={} newT={} Temperature changed.", newT, oldT);

5. Log Protocol (Ali Code Protocol)

1. [Mandatory] Applications cannot use the API of logging system (Log4j and Logback) directly, but should rely on the API of logging framework SLF4J. Using the facade logging framework helps to maintain and unify the log processing mode of each class.

2. [Mandatory] In log output, the concatenation between string variables uses placeholder characters.

Note: Because String concatenation uses The StringBuilder append() method, there is a performance penalty. Using placeholders is just a substitution action, which can significantly improve performance.

Logger. Debug (“Processing trade with id: {} and symbol: {}”, id, symbol);

3. [Mandatory] For the output of trace, debug, or INFO logs, the log level must be enabled or disabled.

Note: Although the first line isDisabled(level.debug_int) in the debug method body is true (common Slf4j implementations Log4j and Logback), the argument may be concatenated with strings. In addition, if the debug(getName()) argument contains a getName() method call, there is no need to waste the overhead of the method call.

If the parameter contains a method call or string concatenation, log level on/off judgment must be performed, otherwise there is no need to enforce it

Is:

// If true, trace and DEBUG logs can be output
if (logger.isDebugEnabled()) {
    logger.debug("Current ID is: {} and name is: {}", id, getName()); 
}
Copy the code

4. [Mandatory] The exception information should include two types of information: the scene information and the exception stack information. If no, throw upward through the keyword throws.

Error (param.toString() + “_” + LLDB message (), e);

5. Log carefully. Debug logs cannot be generated in the production environment. Selectively output info logs; If you use WARN to record information about service behaviors when the logs are first launched, be careful about the output of logs to avoid overpowering server disks, and delete these logs in time.

Note: A large number of invalid logs are generated, which is not conducive to system performance improvement or fault location. When journaling, think: Is anyone really reading these journals? What can you do with this log? Does it benefit troubleshooting?

6. [Recommended] The WARN log level can be used to record user input parameter errors, preventing users from being confused when they complain. If not necessary, please do not type error level in this scene to avoid frequent alarm.

Note: Pay attention to the log output level. The error level records only logical errors, exceptions, and major errors.

Six, afterword.

After all, rules are dead. You need to think about logs based on specific scenarios. Where can I record logs to help troubleshoot problems? Where are the logs duplicated or even invalid? Combined with the exception processing, which exceptions should be thrown to the upper layer caller, according to the different scenarios of the upper layer application to choose different processing methods; When handling exceptions, make sure to record the exception information.

Attached: Reference links & documentation

  1. Standard outputs should not be used directly to log anything
  2. Java Logging Framework Parsing (Part 1) – Historical Evolution
  3. An architect must take you through the chaos of JAVA logging system!
  4. 13 | log: logging is not so easy as you think
  5. Logback Chinese manual
  6. SLF4J user manual
  7. Code of Ali