Writing in the front

The Logback configuration file can increase TPS by 10 times as much as the Logback configuration file.

To sum up:

  • Logback performance tests show little difference between synchronous and asynchronous TPS
  • It’s been 9102 years and they’re still using Logback

After seeing the comments from netizens, I went on to do a wave of tests and research

Server hardware

  • CPU six nuclear
  • 8 g memory

Testing tools

  • JMeter
  • JProfile
  • APM(New Relic)

Logback Logging framework synchronous and asynchronous testing

The previous test results had the following problems:

  • The number of test samples is too small (that is, the number of threads and the number of loops is too small, the number of threads was 100, the number of loops was 1, and the total number of samples was 100)
  • There were too few tests, only one test, and the results were haphazard
  • The results of the two tests were contaminated and the number of samples was different

In view of the above problems, the number of threads was changed to 200 in the re-test, with 100 cycles in each test. The total number of samples was 2W, which was 200 times larger than the number of samples in the original test, and the test was repeated for 5 times. The new results are as follows:

The new test results show that TPS with synchronous and asynchronous log output using the LogBack logging framework does not differ much. It’s more intuitive to make a histogram of the data

However, there should be a noticeable change in TPS since it seems that the thread accessing the API will respond faster to the client if logging is done asynchronously. After checking some data on the Internet, some people said that there would be a big difference between synchronous and asynchronous TPS through APM for performance monitoring, and TPS must be significantly changed (scream), so APM was used to monitor the requests sent by JMeter (JMeter parameter is set to 100 threads, Ramp-up Period = 0, cycle 100 times) : APM test results are as follows:

Under APM monitoring, the average TPS of synchronous log output within five minutes is 333rpm. It is found that TPS synchronization is not obvious compared with asynchronous synchronization

Once again failed to prove that although I do not understand but later discussed with netizens, inspired

The reasons for the insignificant change in TPS are as follows: TPS is the number of transactions processed per second, and each transaction includes the following three processes:

  • User request server
  • The server’s own internal processing
  • The server returns to the user

The server’s own internal requests include access to the database, processing logic, and printing logs. The only difference between synchronous and asynchronous is how the logs are printed. However, according to the test results, the log printing time only accounts for 5.3% of THE API access requests. Therefore, shortening the log printing time cannot significantly improve TPS, because the printing time, network request and service processing consumption time can be ignored. However, the test results show that although using asynchronous output mode cannot significantly improve TPS, However, it reduces the log printing time. Therefore, asynchronous output is recommended when using the LogBack logging framework

Log4j2 is recommended instead of logback

Log4j2 is an upgrade to log4j 1.x, which references some of the best design of Logback, fixes some problems, and brings some significant improvements in asynchronous performance. It provides Async Append and Async Log implementations. The Async Append asynchronous implementation is similar to the logback asynchronous implementation, while Async Log implements a high-performance asynchronous logger based on the LMAX Disruptor library. The log4j2 asynchronous implementation in this test is based on Async Log. The JMeter test parameters are the same as the previous Logback test, with 200 threads, 100 loops, and five repeats. The logJ2 log configuration file is basically the same as the logBack asynchronous configuration, meeting the following requirements:

  • Console Logs
  • Output Logs by Category
  • According to the days roll
  • The same log output format

The test results are as follows:

TPS increased by 6 times!! In the official test report, the performance of Log4j2 is better than that of LogBack. Asynchronous Loggers for low-latency Logging

conclusion

  • If the LogBack framework is used, asynchronous log output is recommended
  • Choose the logging framework, log4j2 is recommended

In addition, I have a small goal recently. I want to be promoted to LV3. If I think the article is good, PLEASE give it a thumbs up before I leave