Prometheus custom buried point reporting for SpringBoot series

This is the 8th day of my participation in the Gwen Challenge in November. See details: The Last Gwen Challenge in 2021.

In a previous post on Integrating SpringBoot with Prometheus for data reporting, there was more on how a SpringBoot application could access Prometheus at minimal cost and configure a complete application monitor with Grafana

For those of you who have seen Prometheus, SpringBoot provides JVM, GC, and HTTP calls to Prometheus without any additional development. However, in the actual business development process, there are always some scenarios that need to be reported manually. So what can we do about it?

The core knowledge points of this paper:

• How to implement custom data reporting through an example of SpringBoot application

Previous post: SpringBoot integrates Prometheus for Application Monitoring

I. Project environment construction

The project demonstrated in this article is mainly SpringBoot2.2.1, the postures used in later versions are not much different, and 1.x is not guaranteed to work (since I didn’t test it).

1. Rely on

Pom dependencies, mainly the following packages

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>
</dependencies>
Copy the code

2. Configure information

Next is the configuration file, which registers information about Prometheus

spring:
  application:
    name: prometheus-example
management:
  endpoints:
    web:
      exposure:
        include: "*"
  metrics:
    tags:
      application: ${spring.application.name}
Copy the code

In the above configuration, there are two key pieces of information, described in previous blog posts, which are briefly explained here

• management.endpoints.web.exposure.includeThis specifies that all Web interfaces are reported
• metrics.tags.applicationAll metrics reported by the app will be labeled application

After the configuration is complete, a /actuator/ Prometheus endpoint is provided for Prometheus to pull Metrics

II. Custom report

Assuming that we now want to report information about HTTP requests ourselves, we currently plan to collect the following information

• Total requests: adoptedCounter
• Number of requests currently being processed: YesGauge
• Request Time histogram:Histogram

1. Prometheus Metric encapsulation

Based on the above analysis, we implemented three common Metric reports and provided a unified wrapper class to capture the corresponding Metric type

package com.git.hui.boot.prometheus.interceptor;

import io.prometheus.client.CollectorRegistry;
import io.prometheus.client.Counter;
import io.prometheus.client.Gauge;
import io.prometheus.client.Histogram;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.stereotype.Component;

/ * * *@author yihui
 * @date2021/11/09 * /
@Component
public class PrometheusComponent implements ApplicationContextAware {
    private static PrometheusComponent instance;


    /**
     * 请求总数
     */
    private Counter reqCounter;

    /** * The number of HTTP requests */
    private Gauge duringReqGauge;

    /** ** histogram, request distribution */
    private Histogram reqLatencyHistogram;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        instance = this;
        CollectorRegistry collectorRegistry = applicationContext.getBean(CollectorRegistry.class);
        // The SpringBoot container CollectorRegistry is specified here, which will not be collected if the default is used
        reqCounter = Counter.build().name("demo_rest_req_total").labelNames("path"."method"."code")
                .help("Total request count").register(collectorRegistry);
        duringReqGauge = Gauge.build()
                .name("demo_rest_inprogress_req").labelNames("path"."method")
                .help("Number of requests being processed").register(collectorRegistry);
        reqLatencyHistogram = Histogram.build().labelNames("path"."method"."code")
                .name("demo_rest_requests_latency_seconds_histogram").help("Request Time Distribution")
                .register(collectorRegistry);
    }

    public static PrometheusComponent getInstance(a) {
        return instance;
    }

    public Counter counter(a) {
        return reqCounter;
    }

    public Gauge gauge(a) {
        return duringReqGauge;
    }

    public Histogram histogram(a) {
        returnreqLatencyHistogram; }}Copy the code

Note the setApplicationContext() method implementation logic above, where Counter/Gauge/Histogram is created using the most basic usage provided in the Simpleclient package, rather than the Micrometer package. The differences between the two will be covered in a later post

The special feature of the above implementation is that when the Metric is created, the label label is already defined, as defined here

• Path: request URL path
• Method: HTTP method, get/ POST
• Code: indicates the status code, which indicates whether the request is successful or abnormal

2. The interceptor collects and reports customized information

Next we implement a custom interceptor that intercepts all HTTP requests and reports key information

public class PrometheusInterceptor extends HandlerInterceptorAdapter {

    private ThreadLocal<Histogram.Timer> timerThreadLocal = new ThreadLocal<>();

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        // The number of requests being processed
        PrometheusComponent.getInstance().gauge().labels(request.getRequestURI(), request.getMethod()).inc();

        timerThreadLocal.set(PrometheusComponent.getInstance().histogram()
                .labels(request.getRequestURI(), request.getMethod(), String.valueOf(response.getStatus()))
                .startTimer());
        return super.preHandle(request, response, handler);
    }

    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
        String uri = request.getRequestURI();
        String method = request.getMethod();
        int status = response.getStatus();
        // count Indicates the number of requests. The tags are request path, request method, and response HTTP code
        // Total number of applications requested: sum(demo_rest_req_total)
        // Number of HTTP requests per second: sum(rate(demo_rest_req_total[1m])
        Topk (10, sum(demo_rest_req_total) by (path))
        PrometheusComponent.getInstance().counter().labels(uri, method, String.valueOf(status)).inc();

        // Request complete, counter -1
        PrometheusComponent.getInstance().gauge().labels(uri, method).dec();

        // Histogram statistics
        Histogram.Timer timer = timerThreadLocal.get();
        if(timer ! =null) {
            timer.observeDuration();
            timerThreadLocal.remove();
        }
        super.afterCompletion(request, response, handler, ex); }}Copy the code

SpringBoot Interceptor Interceptor: SpringBoot Interceptor Interceptor: SpringBoot Interceptor Interceptor

There are two main concerns here

• Before execution (preHandle) : Gauge +1 starts the timer
• After execution (afterCompletion) : Guage count -1, counter count +1, timed collection

3. The test

Finally, we need to register the interceptor above and write a demo to test it out

@RestController
@SpringBootApplication
public class Application implements WebMvcConfigurer {
    private Random random = new Random();

    @Override
    public void addInterceptors(InterceptorRegistry registry) {
        registry.addInterceptor(new PrometheusInterceptor()).addPathPatterns("/ * *");
    }

    @GetMapping(path = "hello")
    public String hello(String name) {
        int sleep = random.nextInt(200);
        try {
            Thread.sleep(sleep);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return "hello sleep: " + sleep + " for " + name;
    }

    public static void main(String[] args) {
        SpringApplication.run(Application.class);
    }

    @Bean
    MeterRegistryCustomizer<MeterRegistry> configurer(@Value("${spring.application.name}") String applicationName) {
        return (registry) -> registry.config().commonTags("application", applicationName); }}Copy the code

After the application is started, visit hello’s HTTP interface a few times and then check the metric to see if it has the data we just reported

4. Summary

This blog post is the completion of the last one. If we want to customize and report some information, we can use the above method to support it

Of course, the report does not mean the end. It is also crucial to configure the information such as the market, especially how to configure Grafana in the histogram. How to check the time distribution of requests, as described below

III. Can’t miss the source code and related knowledge points

0. Project

• Project: github.com/liuyueyi/sp…
• Source: github.com/liuyueyi/sp…

1. Wechat official account: Yash Blog

As far as the letter is not as good, the above content is purely one’s opinion, due to the limited personal ability, it is inevitable that there are omissions and mistakes, if you find bugs or have better suggestions, welcome criticism and correction, don’t hesitate to appreciate

Below a gray personal blog, record all the study and work of the blog, welcome everyone to go to stroll

• A grey Blog Personal Blog blog.hhui.top
• A Grey Blog-Spring feature Blog Spring.hhui.top