preface

At present, the system of the company adopts the Spring Cloud architecture, in which Nacos is used for service registration and discovery components. Recently, the operation and maintenance complained that the disk is insufficient and the log growth is too fast. A quick check revealed Nacos as the culprit.

Nacos, as a service registry, shouldn’t be generating too many logs, and there aren’t many services involved, but to generate more than 1 gigabyte of logs in a few days is a bit crazy. This article is about Nacos’s logging system.

Background of the event

The most output logs are in {nacos.home}/logs/ access_log.YYYY-MM-dd. log format. The logs contain the logs of microservice system call Nacos and communication between clusters. Such as heart rate (/ nacos/v1 / ns/instance/beat), access service list (/ nacos/v1 / ns/instance/list), the state examination (/ nacos/v1 / ns/service/status), etc.

We know that Nacos is implemented based on Spring Boot, and the access_log log is the access log of Tomcat built in Spring Boot. For the configuration of this log, there is no maximum number of days to retain the log, and there is no control over the log size. Moreover, as Nacos Server directly generates access logs with each service, such as heartbeat, acquisition, registration, etc., the more microservices, the faster the log growth. These logs quickly occupy the disk space, resulting in resource waste and o&M costs.

The solution

Nacos provides a control switch for access_log output. By default, the application. Properties configuration file in the conf directory of Nacos has the following configuration:

#*************** Access Log Related Configurations ***************#
### If turn on the access log:
server.tomcat.accesslog.enabled=true

### The access log pattern:
server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D %{User-Agent}i %{Request-Source}i

### The directory of access log:
server.tomcat.basedir=
Copy the code

You can see the support for turning off the access log, the log output format, and the log output directory.

In the test environment, you can directly disable the log output by setting the enabled configuration item to False.

server.tomcat.accesslog.enabled=false
Copy the code

But in a production environment, this is risky. If production problems occur after the shutdown, you need to check the logs according to the logs.

In this case, you can only perform this operation in other ways. For example, in Linux, you can write crontab to delete logs periodically. The following is an example script:

#! / bin/bash logFile = "/ data/nacos/bin/logs/nacos_del_access log" # 14 days keep log date = ` date - d "$date - 14 day" + "% % Y - m - % d" ` # Location can be adjusted delFilePath = "/ data/nacos/bin/logs/access_log. ${date}. The log" if [! - f "${logFile}"]. Then the echo 'access print log. The log file/etc/cron. Daily/nacosDelAccessLogs sh will regularly delete access log file' > > ${logFile} fi # log file exists, If [-f "${delFilePath}"]; then rm -rf ${delFilePath} curDate=`date --date='0 days ago' "+%Y-%m-%d %H:%M:%S"` echo '['${curDate}'] Delete file '${delFilePath} >>${logFile} fiCopy the code

While the problem was resolved, it was obviously not elegant, and this was one of the problems with Nacos Server log output.

The log level is dynamically adjusted

As for the output level of Nacos Server logs, prior to version 1.1.3, a large number of logs were also printed and there was no way to adjust them dynamically. After this release, log output is optimized and log level adjustments are supported via apis as shown in the following example:

# adjust naming module naming - raft. The log level for error: curl -x PUT '$nacos_server: 8848 / nacos/v1 / ns/operator/log? LogName =naming-raft&logLevel=error' # change config-dump.log level to WARN: curl -X PUT '$nacos_server:8848/nacos/v1/cs/ops/log? logName=config-dump&logLevel=warn'Copy the code

Client Logs

After version 1.1.3, the service system integration client has been optimized to avoid the printing of a large number of logs (mainly heartbeat logs and polling logs).

In the application. Yml configuration file of the business system, this can be controlled by setting the log level:

Logging: level: com.baba.nacos: warnCopy the code

It can also be controlled by JVM parameters at startup, which default is info level:

-Dcom.alibaba.nacos.naming.log.level=warn -Dcom.alibaba.nacos.config.log.level=warn
Copy the code

The preceding example specifies the log level of the Naming client and Config client, respectively, and is applicable to version 1.0.0 and later.

Finer logging configuration

If you look at the nacos-logback.xml configuration in the conf directory, you will find that there are many nacOS-related log configuration items. If you need more detailed configuration for your project, you can directly configure them in this file.

Using append configuration corresponding to naming- Server as an example, the default configuration is as follows:

<appender name="naming-server"
          class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOG_HOME}/naming-server.log</file>
    <append>true</append>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
        <fileNamePattern>${LOG_HOME}/naming-server.log.%d{yyyy-MM-dd}.%i</fileNamePattern>
        <maxFileSize>1GB</maxFileSize>
        <maxHistory>7</maxHistory>
        <totalSizeCap>7GB</totalSizeCap>
        <cleanHistoryOnStart>true</cleanHistoryOnStart>
    </rollingPolicy>
    <encoder>
        <Pattern>%date %level %msg%n%n</Pattern>
        <charset>UTF-8</charset>
    </encoder>
</appender>
Copy the code

You can adjust the output log format, log file splitting, log retention date, and log compression as required.

summary

So much for the logging output of Nacos, overall there are too many related logging outputs, and the flexibility of configuration needs to be improved. Based on the current situation, you can customize or perform scheduled tasks to output and manage logs.

About the blogger: Author of the technology book SpringBoot Inside Technology, loves to delve into technology and writes technical articles.

Public account: “program new vision”, the blogger’s public account, welcome to follow ~

Technical exchange: Please contact the weibo user at Zhuan2quan