The original address: https://xeblog.cn/articles/7

Prometheus profile

A brief history of

Prometheus was inspired by Google’s Brogmon monitoring system (similar to Kubernetes, which evolved from Google’s Brog system) and developed as open source software by former Google engineers at Soundcloud since 2012. An early version will be released in early 2015. In May 2016, Kubernetes became the second project to officially join the CNCF Foundation. In June of the same year, version 1.0 was officially released. At the end of 2017, version 2.0 was released based on the new storage layer, which can better cooperate with container platform and cloud platform.

architecture

The characteristics of

  • Multidimensional data model (k/ V key-value pairs based on time series).
  • Flexible query and aggregate statement (PromQL).
  • Nodes are independent of distributed storage.
  • The PULL mode based on HTTP collects time series data.
  • The push mode can be implemented using PushGateway (an optional middleware for Prometheus).
  • You can use dynamic service discovery or statically configure the target machine for collection.
  • Support a variety of graphics and dashboards.

Relevant concepts

The data model

Prometheus stores sequential data, i.e., a collection of consecutive data stored in a temporal dimension under the same name and label.

Monitoring samples

# HELP system_cpu_usage The "recent cpu usage" for the whole system
# TYPE system_cpu_usage gauge
system_cpu_usage 0.23587264544090683
# HELP logback_events_total Number of error level events that made it to the logs
# TYPE logback_events_total counter
logback_events_total{level="error",} 0.0Logback_events_total {level = "info",}557.0

Copy the code
  • HELP indicates the current indicator. TYPE indicates the data TYPE of the current indicator
  • System_cpu_usage and logback_events_total indicate the name of the current indicator
  • The labels in {} reflect some of the characteristics and dimensions of the current sample
  • 0.23587264544090683, 0.0, 557.0 are the specific values of the monitoring sample

The sequential type

Temporal sequence data of Prometheus are divided into four types: Counter, Gauge, Histogram and Summary.

The use of Prometheus

The installation

Official Download address

configuration

Pom depends on

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
	<groupId>io.micrometer</groupId>
	<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
Copy the code

Project Start monitoring

management.endpoint.metrics.enabled=true
management.endpoints.web.exposure.include=*
management.endpoint.prometheus.enabled=true
management.metrics.export.prometheus.enabled=true
Copy the code

Add the Job

Access the Prometheus installation root directory vim Promethe.yml

The new node

- job_name: xeblog-API metrics_PATH: /actuator/ Prometheus STATIC_configs: -targets: ['127.0.0.1:8080 ']Copy the code

Job_name: task name metrics_path: indicator path Targets: instance address or project address. Multiple targets can be configured

Run the Prometheus

/ Prometheus running success logs are displayed in the Prometheus installation root directory

Access address: localhost:9090

PromQL

Matching filter

Operator: =! = = ~! ~

Number of requests matching non-GET request methods whose monitoring task is Xeblog – API

http_server_requests_seconds_count{job="xeblog-api", method! ="GET"}
Copy the code

Matches the number of requests from non-GET request methods whose monitoring task is Xeblog-api, and the request path is not/API /message or/API /version

http_server_requests_seconds_count{job="xeblog-api", method! ="GET", uri ! ~"/api/message|/api/version"}
Copy the code

The main difference is, “=~! ~ “supports regular matching

Range queries

Http_request_total {}[5m] Optional unit:

  • S – seconds
  • M – minutes
  • H – hour
  • D – day
  • Week of w –
  • Y – years

Time shift operation (offset)

Query the sample data generated 5 minutes ago: http_request_total{} offset 5m

Query sample data within 1 day yesterday: http_request_total{}[1d] offset 1D

Aggregation operations

  • Sum (sum)
  • Min (minimum value)
  • Max (maximum)
  • Avg (average)
  • Stddev (standard deviation)
  • Stdvar (Standard Difference)
  • Count (= count)
  • Count_values (count values)
  • Bottomk (last N sequence)
  • Topk (the first N sequences)
  • Quantile…

Built-in function

  • Increase (v range-vector) : Retrieves the first and last samples of the interval vector and returns their increments
  • Rate (v range-vector) : calculates the average growth rate of interval vector V in the time window
  • Irate (v range-vector) : Calculates the average growth rate of interval vector V over the time window (instantaneous growth rate, based on the last two data)
  • Predict_linear (V range-vector, T scalar) : Predicts the value of time series V after t seconds
  • V instant-vector (ABS) : Returns the absolute value of the input vector
  • Sort (v instant-vector) : sort in ascending order
  • Sort_desc (v instant-vector) : sort in descending order…

Grafana visualization

The installation

Official Download address

Example of installation and startup on Mac

/ / installation
brew install grafana
/ / start
brew services start grafana
Copy the code

After startup, access address: localhost:3000

Login: The initial user name and password are admin

Add Prometheus data source

The new Dashboard

You can manually add or import a configured Json file to the Dashboard sharing community: grafana.com/dashboards You can download the Dashboard Json configuration file shared by others

Spring Boot Statistics

Monitoring alarm

Configure the sender email address

Vim grafana ini I file path is/usr/local/etc/grafana/grafana ini configuration is as follows:

[smtp]
enabled = true
host = smtp.qq.com:25User = your [email protected] # If the password contains #or ; you have to wrap it with trippel quotes. Ex """#password;"""Password = email password (not QQ password); cert_file = ; key_file = ; skip_verify =falseFrom_address = your [email protected] from_name = Grafana # EHLO identity in SMTP dialog (defaults to instance_name); ehlo_identity = dashboard.example.comCopy the code

QQ mailbox is configured here, and the same applies to other mailboxes

After configuration, restart Grafana

brew services restart grafana
Copy the code

Configuring recipients

Monitoring indicators

“Template variables are not supported in Alert queries” will be displayed for Prometheus, which does not support monitoring Settings with Template variables.

Monitoring example: Monitoring CPU usage PromQL:

system_cpu_usage{instance="Instance address", job="Task Name"}
Copy the code

Thank you for reading, if there are mistakes, please correct!

The resources

  • Songjiayang. Gitbooks. IO/Prometheus /…
  • Yunlzheng. Gitbook. IO/Prometheus -…
  • prometheus.io/
  • grafana.com/