1. Error log alarm combat

1.1. Demand

In order to learn more about system errors in real time, I began to search for alarm solutions

1.2. Thinking

1.2.1. A scheme with good money

If there is no shortage of money and a more systematic and perfect solution, THE first thing THAT comes to my mind is CAT. It can not only realize error alarm, but also be more intelligent. The error interval of alarm, error alarm content, QPS alarm and other ways are more diversified, and it can also check the interface QPS traffic and so on

1.2.2. Consider implementing it yourself

  1. Their own implementation to consider whether rightlog.errorLogback provides an appender to mail, which is a great way to integrate directly

1.3. Configuration files

pom

< the dependency > < groupId > org. Codehaus. Janino < / groupId > < artifactId > janino < / artifactId > < version > 2.7.8 < / version > </dependency> <dependency> <groupId>javax.mail</groupId> <artifactId>mail</artifactId> <version>1.4.7</version> </dependency>Copy the code
<configuration> <contextName>logback</contextName> <! --> <springProperty scope="context" name="applicationName" source="spring.application.name"/>
    <springProperty scope="context" name="alertEmail" source="onegene.alert.email"/>
    <springProperty scope="context" name="profile" source="spring.profiles.active"/ > <! -- Mail --> <! -- SMTP server address, which must be specified. For example, the SMTP server address of netease is smtp.163.com --> <property name="smtpHost" value="hwhzsmtp.qiye.163.com"/ > <! -- Enter the address of the SMTP server from which you want to send mail (ask your DBA or manager)--> <! -- Port address of the SMTP server. Default value: 25 --> <property name="smtpPort" value="465"/ > <! -- Send mail account, default is null --> <property name="username" value="[email protected]"/ > <! -- Sender account --> <! -- Send mail password, default is null --> <property name="password" value="rVgkwPL4WsWmGV72"/ > <! -- Sender password --> <! -- If set totrue, the appender will use SSL to connect to the logging server. Default value:false -->
    <property name="SSL" value="true"/ > <! <to> <property name="email_to" value="${alertEmail}"/ > <! Multiple recipient accounts can be separated by commas --> <! -- Specifies the sender name. If set to "&lt; ADMIN&gt;" , the sender will be <ADMIN> --> <property name="email_from" value="[email protected]"/ > <! -- Specify the title of emial, which needs to meet the formatting requirements in PatternLayout. If the value is set to Log: %logger - % MSG, for example, the title of the email is [Error] : com.foo.Bar - Hello World. Default value:"%logger{20} - %m". -->
    <property name="email_subject" value=""${applicationName}:${profile}The Error 】 : % logger"/ > <! -- ERROR message sending --> <appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
        <smtpHost>${smtpHost}</smtpHost>
        <smtpPort>${smtpPort}</smtpPort>
        <username>${username}</username>
        <password>${password}</password>
        <asynchronousSending>true</asynchronousSending>
        <SSL>${SSL}</SSL>
        <to>${email_to}</to>
        <from>${email_from}</from>
        <subject>${email_subject}</subject> &emsp; &emsp; &emsp; &emsp; <! HTML format --> <layout class="ch.qos.logback.classic.html.HTMLLayout"> <Pattern>%date%level%thread%logger{0}%line%message</Pattern> </layout> &emsp; &emsp; &emsp; &emsp; <! --> <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>ACCEPT</onMatch>
            <onMismatch>DENY</onMismatch>
        </filter>
        <cyclicBufferTracker class="ch.qos.logback.core.spi.CyclicBufferTracker"> <! Send only one log entry per email --> <bufferSize>1</bufferSize> </cyclicBufferTracker> </appender> <! The value of name is the name of the variable, and the value of value is the value defined by the variable. The defined values are inserted into the Logger context. After defining a variable, you can make"The ${}"To use variables. --> <property name="log.path" value="log"/ > <! -- Color log --> <! -- Color log dependent render class --> <conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
    <conversionRule conversionWord="wex"
                    converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter"/>
    <conversionRule conversionWord="wEx"
                    converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter"/ > <! -- Color log format --> <property name="CONSOLE_LOG_PATTERN"
              value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }The CLR) {magenta} % (-) {abbreviation} % CLR ([% 15.15 t]) {abbreviation} % CLR (% 40.40 logger {39}) {cyan} % CLR (:) {abbreviation} % m % n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/ > <! --> <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender"> <! This log appender is configured for development purposes only at the lowest level. The console outputs logs with a log level greater than or equal to this level --> <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>debug</level>
        </filter>
        <encoder>
            <Pattern>${CONSOLE_LOG_PATTERN}</Pattern> <! - set the character set - > < charset > utf-8 < / charset > < / encoder > < appender > <! -- Output to file --> <! <appender name="DEBUG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <! -- The path and name of the log file being recorded --> <file>${log.path}/${applicationName}-log.log</file> <! <encoder> <pattern>%d{YYYY-MM-DD HH: MM :ss.SSS} [%thread] %-5level % Logger {50} - % MSG %n</pattern> <charset>UTF-8</charset> <! Set character set --> </encoder> <! -- logger rollingPolicy, record by date, record by size --> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <! -- Log Archive --> <fileNamePattern>${log.path}/${applicationName}-log-%d{yyyyMMdd}.log.%i</fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <maxFileSize>500MB</maxFileSize> </timeBasedFileNamingAndTriggeringPolicy> <! <maxHistory>15</maxHistory> </rollingPolicy> </appender> < Logger name="com.onegene.platform" level="debug"/>
    <logger name="com.onegene.platform.auth.authority" level="info"/>
    <logger name="org.springframework.security.oauth2.provider.endpoint" additivity="false"/>
    <springProfile name="local">
        <root level="info">
            <appender-ref ref="CONSOLE"/>
            <appender-ref ref="DEBUG_FILE"/>
        </root>
    </springProfile>
    <springProfile name="dev,pro">
        <root level="info">
            <appender-ref ref="CONSOLE"/>
            <appender-ref ref="DEBUG_FILE"/>
            <appender-ref ref="EMAIL"/>
        </root>
    </springProfile>
</configuration>
Copy the code

1.4. Configuration file Interpretation

  1. Focus on configuration files
    <springProperty scope="context" name="applicationName" source="spring.application.name"/>
    <springProperty scope="context" name="alertEmail" source="onegene.alert.email"/>
    <springProperty scope="context" name="profile" source="spring.profiles.active"/>
Copy the code
  1. I have pulled out most of the variables that can be pulled out. This configuration file can be put directly into any projectbootstrap.ymlIn thespring.application.nameParameters change
  2. The alarm sender can also be configured in the configuration file.Note here:onegene.alert.emailandspring.application.nameParameters are best inbootstrap.ymlIs configured instead ofapplication.ymlBecause thebootstrap.ymlThe read priority of theapplication.ymlOtherwise, you might not be able to read these two parameters

At this point, if we print the log.error log, we will send the error log to the specified email, but this is definitely not enough. We need to work with @ControllerAdvice. For example, if individual error logs are frequent and unavoidable, and do not need to be handled, we can extend them a little bit and define an interface injection to handle whether the error message does not need to be sent in business code

Code 1.5.

  1. Exception handling
@ControllerAdvice
@Slf4j
public class SystemExceptionHandler {

    @Autowired(required = false)
    private IExceptionFilter exceptionFilter;

    @ExceptionHandler(value = {DuplicateUniqueException.class, DuplicateKeyException.class})
    @ResponseBody
    public Result duplicateUniqueExceptionExceptionHandler(HttpServletRequest request, Exception e) {
        return getExceptionResult(e, StatusCode.FAILURE_SYSTEM_CODE, "Unique primary key duplicate (or joint unique key)".false);
    }

    @ExceptionHandler(value = {FeignException.class, RuntimeException.class})
    @ResponseBody
    public Result FeignExceptionHandler(HttpServletRequest request, Exception e) throws Exception {
        throw e;
    }

    @ExceptionHandler(value = Exception.class)
    @ResponseBody
    public Result commonExceptionHandler(HttpServletRequest request, Exception e) {
        return getExceptionResult(e, StatusCode.FAILURE_CODE, true);
    }


    private Result getExceptionResult(Exception e, int statusCode, boolean ignoreAlert) {
        returngetExceptionResult(e, statusCode, e.getMessage(), ignoreAlert); } private Result getExceptionResult(Exception e, int statusCode, String msg, boolean ignoreAlert) { e.printStackTrace();  String exceptionName = ClassUtils.getShortName(e.getClass()); StackTraceElement[] stackTrace = e.getStackTrace(); StringBuilder sb = new StringBuilder();for (StackTraceElement stackTraceElement : stackTrace) {
            sb.append(stackTraceElement.toString()).append("\n");
        }
        String message = e.getMessage();
        if (ignoreAlert && filter(e)) {
            log.error(ExceptionName ==> {}, message:{},detail:{}", exceptionName, message, sb.toString());
        }
        return Result.failure(statusCode, msg);
    }

    private boolean filter(Exception e) {
        if(exceptionFilter ! = null) {return exceptionFilter.filter(e);
        }
        return true; }}Copy the code

The interface is simple

public interface IExceptionFilter {
    boolean filter(Exception e);
}
Copy the code

Exceptions that do not need to be handled are handled here

/** * @author: laoliangliang * @description: Filter exceptions that do not need to be reported * @create: 2020/4/9 10:00 **/ @Component @Slf4j public class FilterAlert implements IExceptionFilter { @Override public boolean filter(Exception e) {if (e instanceof ConnectException) {
            return false;
        }
        return true; }}Copy the code

1.6. Summarize

  1. So far, the error alarm scheme has been completely realized, and the subsequent work is optimization. The results are as follows

Error mailing list

Incorrect email content