Prometheus deployment:

Install the GO environment

As Prometheus was developed with Golang, a go environment was installed first. Go language is cross-platform, supporting Windows, Linux, Mac OS X and other systems, and also provides source code for compilation and installation

  • Binary package download address
Wget HTTP: / / https://dl.google.com/go/go1.13.5.linux-amd64.tar.gzCopy the code

To install a specific historical version, click here to go to the go language website

  • After downloading it, upload it to the server to be deployed. After specifying the decompression path, set the environment variables
[root@prometheus ~]# tar xf go1.13.4.linux-amd64.tar.gz -C /usr/local
Copy the code
  • Configuring environment Variables
[root@prometheus ~]# vim /etc/profile Export PATH=$PATH:/usr/local/go/bin # Effective environment variables [root@prometheus ~]# source /etc/profileCopy the code
  • Test whether the GO environment is installed successfully
[root@prometheus ~]# go version
go version go1.13.4 linux/amd64
Copy the code

Install the Prometheus

  • Binary package download address
Wget HTTP: / / https://github.com/prometheus/prometheus/releases/download/v2.14.0/prometheus-2.14.0.linux-amd64.tar.gzCopy the code

To install the specified historical version, go to Prometheus website (including all export components).

  • After the package is downloaded, upload it to the prepared server and decompress it to the specified directory
[root@prometheus ~]# tar xf Prometheus -2.14.0.linux-amd64.tar.gz -c /usr/localCopy the code
  • To facilitate configuration later, make a soft link
[root@prometheus ~]# ln -sv /usr/local/Prometheus -2.14.0. Linux -amd64/ /usr/local/PrometheusCopy the code
  • Then modify the Prometheus configuration file
[root@prometheus ~]# vim /usr/local/Prometheus/prometheus.yml 
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']
Copy the code
  • Start Prometheus in the background
[root@prometheus ~]# nohup /usr/local/Prometheus/prometheus --config.file=/usr/local/Prometheus/prometheus.yml &
Copy the code
  • Check the running status
[root@prometheus ~]# cat ./nohup.out
Copy the code

To open the monitoring page of Prometheus, use the following URL: localhost:9090, click targets to go to the monitoring target, and the configured monitoring object is displayed.

If the status of the monitored object is Down in the State column, the Node is not configured or incorrectly configured

Configuring hot Loading

As Promtheus stored a large amount of data in its timing database, each reboot of Prometheus was slower and slower. The configuration information of Prometheus is frequently adjusted in daily operation and maintenance. In fact, Prometheus provides the function of hot loading configuration information during running.

  • Hot loading: Send a POST request to /-/reload with the –web.enable-lifecycle option when Prometheus is started
[root@prometheus ~]# nohup /usr/local/Prometheus/prometheus --config.file=/usr/local/Prometheus/prometheus.yml --web.enable-lifecycle &
Copy the code
  • Hot load command
[root@prometheus ~]# curl -XPOST http://localhost:9090/-/reload
Copy the code

This eliminates the need to close and start Prometheus after changing the configuration file

Expand (Add nodes in batches)

When adding a large number of host clusters, it was obviously inconvenient to add them one by one in Prometheus. yml. We wrote discovery files to manage hosts in batches

  • Write automatic discovery files
[root@prometheus ~]# mkdir -p /usr/local/Prometheus/prometheus/targets/node [root@prometheus ~]# vim /usr/local/Prometheus/prometheus/targets/node/node.yml - targets: - '172.16.214.141:9100' - '172.16.214.1414:9100' - '172.16.214.139:9100' Labels: IDC: "BJ" # Note Cluster nameCopy the code
  • Modifying a Configuration File
[root@prometheus ~]# vim /usr/local/Prometheus/prometheus/prometheus.yml
  - job_name: 'nm_mch-app_node'
    file_sd_configs:
      - files: ['/usr/local/Prometheus/prometheus/targets/node/node.yml']
        refresh_interval: 5s
Copy the code

Configure Systemd to start Prometheus

[root@prometheus ~]# vim /lib/systemd/system/prometheus.service [Unit] Description=Prometheus After=network.target [Service] ExecStart=/usr/local/Prometheus/prometheus --config.file=/usr/local/Prometheus/prometheus.yml --web.enable-lifecycle [Install] WantedBy=multi-user.target # reload [root@prometheus ~]# systemctl daemon-reload [root@prometheus ~]# systemctl start prometheus [root@prometheus ~]# systemctl enable prometheusCopy the code