keyword Rancher
Prometheus
swoole
docker
docker-compose
Prometheus
features
Prometheus is an open source monitoring alarm system and time sequence database (TSDB) developed by SoundCloud. Prometheus, developed using the Go language, is an open source version of Google’s BorgMon monitoring system and capable of supporting tens of thousands of clusters.
- Multidimensional data model
- Flexible query language
- Individual server nodes are independent of distributed storage
- Time series data is collected in PULL mode based on HTTP
- Temporal sequence data can be pushed through an intermediate gateway
- Discover target service objects through service discovery or static configuration
- Supports a wide variety of charts and interfaces, such as Grafana
The basic principle of
Prometheus uses HTTP to periodically fetch Metrics, sort out and clean Metrics data according to certain rules, and store them in a new time series. In this actual practice, the CORE indicators of PHP Swoole used by the business will be monitored and output through grafana graphics tools will be displayed.
Set up steps
Native orchestration of Prometheus Docker
For direct deployment on Rancher, we can use Docker-compose locally to conduct remote deployment of K8S after local arrangement experiment, and use Docker-compose locally to simulate K8S container arrangement, which is convenient for development experiment.
Start by creating a Monitor folder as the path for this project
$ mkdir monitor && cd monitor
Copy the code
Create a Prometheus folder to store our monitoring configuration files
$ mkdir prometheus && cd prometheus && touch prometheus.yml
Copy the code
Add the following to the prometheus.yml configuration file
global:
scrape_interval: 30s # Global period
scrape_configs:
- job_name: 'foo' # job name
scrape_interval: 30s # call cycle
metrics_path: "/health/metrics" Call path
static_configs:
- targets: ['foo.com:80'] # remote service address
Copy the code
In the configuration file above we pointed out the foo the monitoring work, and specify the 30 s running period, each call to address to foo.com: 80 / health/metrics that interface address. How to output data for this interface will be discussed below.
Create the docker-comemage. yml file at the root of the monitor folder
$ touch docker-compose.yml
Copy the code
Add the following
version: '2'
services:
prometheus:
container_name: prometheus # container name
image: prom/prometheus # mirror name
networks: # Shared network
- foo
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml # to local configuration files, mounted to the target container/etc/Prometheus/Prometheus yml position
ports:
- '9090:9090'
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--web.enable-admin-api'
- '--web.enable-lifecycle' Support hot update of configuration files
grafana:
container_name: grafana
image: Grafana/grafana: 6.4.3
ports:
- '3000:3000'
depends_on:
- prometheus # start dependent on Prometheus service
volumes:
- ./data:/var/lib/grafana
- ./prometheus.yml:/etc/prometheus/prometheus.yml
networks:
- foo
environment:
GF_SECURITY_ADMIN_USER: ${GRAFANA_USERNAME} # grafana username (variable)
GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD}
networks: Define an external network for easy access between services
foo:
external: true
Copy the code
The configuration writing problem in the Command option of the Prometheus service is highlighted here, because the PROM/Prometheus image already has program entry and is configured with –config.file and –web.enable-admin-api. In this case, we only need to append the web.enable-lifecycle command to enable the monitoring configuration of service hot update. If only the last parameter is written, the preceding two parameters will not be configured and the service will not be started due to the lack of configuration.
Swoole interface
In the Swoole project, write the interface /health/metrics/ as follows:
$server = new Swoole\Http\Server("127.0.0.1".80); // The $server object type is only shown here. The server object needs to be retrieved from the current project framework. Do not instantiate it again.
$metrics = $server->stats();
foreach ($metrics as $k= >$v) {
echo sprintf("%s %s\n".$k.$v);
}
echo "# coroutine state \n";
$metrics = Swoole\Coroutine::stats();
foreach ($metrics as $k= >$v) {
echo sprintf("%s %s\n".$k.$v);
}
Copy the code
http://127.0.0.1:80/health/metrics/ will have the following output
start_time 1573612504 connection_num 1 accept_count 7 close_count 6 worker_num 2 idle_worker_num 1 tasking_num 0 Request_count 10 worker_request_count 0 worker_dispatch_count 11 coroutine_num 1 # Coroutine status event_num 2 signal_listener_num 0 aio_task_num 0 c_stack_size 2097152 coroutine_num 1 coroutine_peak_num 1 coroutine_last_cid 3Copy the code
You can refer to the Swoole documentation for a detailed explanation of the parameters.
Enabling the Monitoring Service
Enter the monitor directory and run the docker-compose up -d command. A message may be displayed indicating that the external network does not exist. Create the external network as prompted. After the service is started, visit http://127.0.0.1:9090 to view the Prometheus Web page.
Go to http://127.0.0.1:9090/targets
The Prometheus service is properly monitored
Visit http://127.0.0.1:3000/ by clicking Add Data Source and selecting Prometheus as the data source
Enter http://prometheus:9090 in the HTTP URL field, because the local container uses the same network and can use the service name directly for service access.
Click Grafana Home, New Dashboard, Choose Visualization, and select Graph
Click Queries and select Query as Prometheus
Add Metrics in A. Use Legend to Add Metrics. Click Add Query to draw multiple index lines in A chart. The local deployment is now complete.
Rancher K8S deployment
Rancher deployment is the same, the only difference being that Rancher is used as a container choreographer. Open Rancher and click Deploy Service
Set the image to PROM/Prometheus, configure the data volume, and configure the YML file for the monitor to mount the container. Refer to the Rancher documentation for mounting and use here.
. In order to join – config file = / etc/Prometheus/Prometheus yml — web. Enable the admin – API – web. Enable – lifecycle
Click the start button at the bottom and configure load balancing in your workload
In this case, you can access grafana.google.com locally. If no public NETWORK DNS exists, you need to modify /etc/hosts locally.
Grafana can be configured in the same way according to the local build mode, which will not be repeated in this article.
Prometheus configuration file hot update
$ curl -X POST 'http://grafana.google.com/-/reload'
Copy the code