Review the log system

The Job system needs to log a lot of logs recently. (If you call someone else’s API, it is a pit. If you do not pay attention to someone else’s internal server, it is necessary to print more logs.) The author tried to optimize the logging system, and then successfully turned himself into a pit. But also because I do not fully understand the log system, take this opportunity to better understand the log system.

The means of optimization are relatively simple:

1. The printing console logs are deleted in the production environment
2. Change RollingFile to RollingRandomAccessFile
3. Add the Disruptor Jar to enable asynchronous Log4j2

Ps: Optimization 2 uses buffer in RollingRandomAccessFile to improve I/O efficiency. After the asynchronous Log4j2 is enabled, the author chooses to change RollingRandomAccessFile back to RollingFile. In addition, asynchrony is not perfect, and some logs will be lost when the service is stopped. It would be best if you could Sleep for a period of time when the service is stopped. This project does not care about some logs, so it does not add Sleep.

Step 1: Discard Log4j1

Since Log4J2 is needed, all Log4j1 jars need to be removed.

MVN dependency: tree -- > tree. TXT [INFO] | + - org.. Apache hadoop: hadoop - HDFS: jar: 2.5.1: compile [INFO] | | + - Log4j: log4j jar: 1.2.17: compileCopy the code

Print out the dependency tree, find HDFS Jar Log41, locate Log4j1, use:

<exclusion>
  <groupId>log4j</groupId>
  <artifactId>log4j</artifactId>
</exclusion>
Copy the code

And it didn’t work… Beep the dog. This Jar is included in the POM of Biz layer, so I directly removed the Biz Log4j1. The world was supposed to be quiet, but it was wrong.

Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 66 more
Copy the code

Should not ah online all logging system is to use common – log or slf4j, unless it is used directly Log41 code ~ see HDFS is in common – the log. Check for two days found that HDFS uses the HDFS INFO level constant ………… So, Log4j1 cannot be deleted.

Step 2: Isolate Log4j1

Commom – log principle

Commom – log:

1. Look for LogFactory from system.properties
2. To find from SPI
3. Use the default LogFactoryImpl

To make things easier for you to understand, I have scratched out the code and replaced the constants for you to see, which you can find by searching in common-log 1.2.

The first step is to look for the LogFactory from System.properties

String factoryClass = getSystemProperty("org.apache.commons.logging.LogFactory", null);
factory = newFactory(factoryClass, baseClassLoader, contextClassLoader);
Copy the code

Step two, look for it from SPI

final InputStream is = getResourceAsStream(contextClassLoader, "META-INF/services/org.apache.commons.logging.LogFactory"); . . factory = newFactory(factoryClassName, baseClassLoader, contextClassLoader );Copy the code

public InputStream getResourceAsStream(String name) {
    URL url = getResource(name);
    try {
        return url != null ? url.openStream() : null;
    } catch (IOException e) {
        return null;
    }
}
Copy the code

Ps: It’s also a good idea to use java.util.ServiceLoader provided with Java 1.6. Dubbo and many other frameworks are SPI’s best practitioners.

Third, use the default LogFactoryImpl

factory = newFactory("org.apache.commons.logging.impl.LogFactoryImpl", thisClassLoader, contextClassLoader);
Copy the code

Here’s what each Logger will look for:

private static final String[] classesToDiscover = {
        LOGGING_IMPL_LOG4J_LOGGER,
        "org.apache.commons.logging.impl.Jdk14Logger",
        "org.apache.commons.logging.impl.Jdk13LumberjackLogger",
        "org.apache.commons.logging.impl.SimpleLog"
};
Copy the code

Slf4j principle

ClassLoader loggerFactoryClassLoader = LoggerFactory.class.getClassLoader();
Enumeration<URL> paths;
if (loggerFactoryClassLoader == null) {
    paths = ClassLoader.getSystemResources("org/slf4j/impl/StaticLoggerBinder.class");
} else {
    paths = loggerFactoryClassLoader.getResources("org/slf4j/impl/StaticLoggerBinder.class");
}
while (paths.hasMoreElements()) {
    URL path = paths.nextElement();
    staticLoggerBinderPathSet.add(path);
}
Copy the code

conclusion

You can see that COMmom-log and Slf4j provide two great ideas for doing plug-in services. Either common-log using the interface (SPI’s pattern) or Slf4j using the facade pattern (StaticloggerBinder.class) are good choices.

Finally, a wave of asynchronous Log4j2 is recommended. Performance tests can be seen:

www.jianshu.com/p/570b406bd…