This article describes how SpringBoot uses Graylog log collection.

1. Graylog is introduced

Graylog is a production-level log collection system that integrates Mongo and Elasticsearch for log collection. Mongo is used to store Graylog metadata information and configuration information, ElasticSearch is used to store data.

The architecture diagram is as follows:

The production environment configuration diagram is as follows:

2. Install Graylog

Install Graylog, Mongo, elasticSearch, and docker-compose in docker-compose mode.

Docker-comemage. yml contains the following content:

version: '2'
services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: mongo:3
  # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.6/docker.htmlElasticsearch: image: docker. Elastic. Co/elasticsearch/elasticsearch - oss: 6.6.1 environment: - http.host=0.0.0.0 -transport. host= localhost-network. host=0.0.0.0 -"ES_JAVA_OPTS=-Xms256m -Xmx256m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    mem_limit: 512m
  # Graylog: https://hub.docker.com/r/graylog/graylog/Graylog: image: graylog/graylog: 3.0 the environment:# CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_PASSWORD_SECRET=somepasswordpepper
      # Password: admin
      - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4 bb8a81f6f2ab448a918 - GRAYLOG_HTTP_EXTERNAL_URI = http://106.13.35.42:9000/ links: - mongodb:mongo - elasticsearch depends_on: - mongodb - elasticsearch ports:# Graylog web interface and REST API
      - 9000:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
Copy the code

106.13.35.42 is my external IP address, and 127.0.0.1 is ok for the local service.

Other ways to view the official documentation, docs.graylog.org/en/3.0/page…

3. The configuration Graylog

Visit http://ip:9000 in your browser, as shown below:

The default user name and password are admin, as shown in the figure.

Select Input from the System button to enter an input source, as shown in the figure

Here, take GELF UDP as an example. Select GELF UDP in the position in the figure and click Launch New Input after selecting, as shown in the figure

Select your own installation in Node and fill in the rest as required, as shown in the figure below

After saving the image, the configuration is complete here.

4.SpringBoot logs are output to Graylog

The following examples are Logback logs and Log4j2 logs.

4.1 Logback logging

The logback-gelf used here outputs logs to Graylog. There is a detailed description of the logback-gelf usage on Github, but this is a simple example. Github address: github.com/osiegmar/lo… .

Create a new project, add logback-gelf dependency, poM file as follows:

<? xml version="1.0" encoding="UTF-8"? > <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> < modelVersion > 4.0.0 < / modelVersion > < the parent > < groupId > org. Springframework. Boot < / groupId > The < artifactId > spring - the boot - starter - parent < / artifactId > < version > 2.1.4. RELEASE < / version > < relativePath / > <! -- lookup parent from repository --> </parent> <groupId>com.dalaoyang</groupId> The < artifactId > springboot2_graylog < / artifactId > < version > 0.0.1 - the SNAPSHOT < / version > < name > springboot2_graylog < / name > < description > springboot2_graylog < description > < properties > < Java version > 1.8 < / Java version > < / properties > <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId>  </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>de.siegmar</groupId> <artifactId>logback-gelf</artifactId> <version>2.0.0</version> </dependency> </dependencies> <build> <plugins> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>Copy the code

Add logback log configuration and create logback-spring. XML as follows:

<configuration>

    <conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter" />
    <conversionRule conversionWord="wex" converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter" />
    <conversionRule conversionWord="wEx" converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter" />
    <property name="CONSOLE_LOG_PATTERN" value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }The CLR) {magenta} % (-) {abbreviation} % CLR ([% 15.15 t]) {abbreviation} % CLR (% 40.40 logger {39}) {cyan} % CLR (:) {abbreviation} % m % n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${CONSOLE_LOG_PATTERN}</pattern>
            <charset>UTF-8</charset>
        </encoder>
    </appender>

    <appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender"> <graylogHost>106.13.35.42</graylogHost> <graylogPort>12201</graylogPort> </appender> <! -- Console output log level --> <root level="info">
        <appender-ref ref="GELF" />
        <appender-ref ref="STDOUT" />
    </root>


</configuration>

Copy the code

Start the project, the current project port is 8081, view the Graylog console as shown:

4.2 Log4j2 log

Log4j2 log4j2-gELF log4j2-GELF log4j2 log4j2-GELF

<? xml version="1.0" encoding="UTF-8"? > <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> < modelVersion > 4.0.0 < / modelVersion > < the parent > < groupId > org. Springframework. Boot < / groupId > The < artifactId > spring - the boot - starter - parent < / artifactId > < version > 2.1.4. RELEASE < / version > < relativePath / > <! -- lookup parent from repository --> </parent> <groupId>com.dalaoyang</groupId> < artifactId > springboot2_graylog_log4j < / artifactId > < version > 0.0.1 - the SNAPSHOT < / version > <name>springboot2_graylog_log4j</name> <description>springboot2_graylog_log4j</description> <properties> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
            <exclusions>
                <exclusion>
                    <artifactId>spring-boot-starter-logging</artifactId>
                    <groupId>org.springframework.boot</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-log4j2</artifactId>
        </dependency>

        <dependency>
            <groupId>org.graylog2.log4j2</groupId>
            <artifactId>log4j2-gelf</artifactId>
            <version>1.3.1</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>
Copy the code

Create log4j2-spring. XML and output log information as follows:

<? xml version="1.0" encoding="UTF-8"? > <Configuration status="OFF" packages="org.graylog2.log4j2">
    <Properties>
        <Property name="LOG_PATTERN">%d{yyyy-MM-dd HH:mm:ss:SSS} - %-5level  - %pid - %t - %c{1.}:%L - %m%n</Property>
    </Properties>
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT" follow="true">
            <ThresholdFilter level="trace" onMatch="ACCEPT" onMismatch="DENY" />
            <PatternLayout pattern="${LOG_PATTERN}"/>
        </Console>
        <GELF name="gelfAppender" server="106.13.35.42" port="12201" hostName="appserver01.example.com">
            <PatternLayout pattern="%logger{36} - %msg%n"/>
            <Filters>
                <Filter type="MarkerFilter" marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                <Filter type="MarkerFilter" marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/> </Filters> <! -- Additional fields --> <KeyValuePair key="foo" value="bar"/>
            <KeyValuePair key="jvm" value="${java:vm}"/>
        </GELF>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="gelfAppender"/>
            <AppenderRef ref="Console"/>
        </Root>
    </Loggers>
</Configuration>
Copy the code

The port number used in this project is 8888, which can be clearly seen in the log.

5. ELK vs Graylog

Using log collection as an example, I will briefly describe the choice between the two. My personal advice is that Graylog can save a lot of money depending on the existing technology stack. For example, Mongodb is already available, and ELK is similar.

6. The source code

Springboot2_graylog: gitee.com/dalaoyang/s…

Springboot2_graylog_log4j: gitee.com/dalaoyang/s…