Involving the file download address: link: pan.baidu.com/s/18XHK7ex_… Password: 0qN6 The image that needs to be downloaded in the file must be downloaded in advance. Eg: PROM/Node-exporter: V0.16.0

Prometheus official website, or Baidu themselves understand imagination:prometheus.io/



Official documentation links

Prometheus is an open source system monitoring tool. Specify metrics on the target using HTTP/S periodically scrape/pull based on the configured job. The target can be specified statically or automatically. Prometheus stores scraped metrics on local or remote storage. Prometheus collects metrics in pull mode. Prometheus Server: a core component responsible for collecting, scraping, and storing time series data, and providing a query interface; Jobs/Exporters: the client monitors and collects metrics to expose the HTTP service (/metrics). There are already a lot of software native support Prometjeus, provide /metrics, can be used directly; For applications that do not provide /metrics, you can use the existing exporters or develop your own exporters to provide /metrics services. Pushgateway: Short-lived Jobs push indicators to Pushgateway and Prometheus Server pull indicators from Pushgateway. Alertmanager: alarm component that responds to configured rules, such as sending emails. Web UI: Prometheus has a simple built-in Web console, which can query indicators, view configuration information or Service Discovery, etc. In actual work, Grafana is usually used to view indicators or create dashboards, Prometheus as the data source of Grafana. Data structure Prometheus stores indicators in terms of a time series, each consisting of Notation + Samples: Notation: Usually consisting of a Notation name with a set of labels:  <metric name>{<label name>=<label value>, ... } Samples: Samples, usually containing a 64-bit floating-point value and a millisecond timestampCopy the code

The following is a demonstration of monitoring Kubernetes on a Mac installation using Prometheus+Grafana

Link ==-==Kubernetes Dashboard install, fast and easy to run Dashboard

  • The environment is as follows: Docker for Mac or Edge version of the built-in Kubernetes cluster

    • Command to view environment information
shenhl:~ user$ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER RUNTIME Docker-for-Desktop Ready Master 20D v1.9.6 < None > Docker for Mac 4.9.87-linuxkit-aufs docker://18.5.0 shenhl:~ user$ kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE default Kubernetes-bootcamp-5dbf48f7d4 -dtn4f 1/1 Running 0 14d 10.1.0.92 Docker-for-desktop default mysql-756vx 1/1 Running 0 11D 10.1.0.96 Docker-for-desktop default myweb-494kV 1/1 Running 0 8D 10.1.0.102 Docker-for-desktop default myweb-77r2b 1/1 Running 0 8D 10.1.0.101 Docker-for-desktop default myWeb-7p7H8 1/1 Running 0 8D 10.1.0.98 docker-for-desktop default Myweb-jbfs9 1/1 Running 0 8d 10.1.0.100 Docker-for-desktop default myweb-v6mnf 1/1 Running 0 8d 10.1.0.99 Docker-for-desktop docker comement-5d4f4d67b6-ttmpk 1/1 Running 0 20D 10.1.0.94 Docker-for-desktop Docker Compose - api-7bb7b5968F-BGvZ8 1/1 Running 1 20d 192.168.65.3 docker-for-desktop kube-system etcd-docker-for-desktop 1/1 Running 7 20d 192.168.65.3 docker-for-desktop kube-system kube-apiserver-docker-for-desktop 1/1 Running 7 20d 192.168.65.3 docker-for-desktop kube-system kube-apiserver-docker-for-desktop 1/1 Docker-for-desktop kube-system kube-controller-manager-docker-for-desktop 1/1 Running 7 20d 192.168.65.3 Docker-for-desktop kube-system kube-dns-6f4fd4bdf-bxkgg 3/3 Running 0 20d 10.1.0.95 Docker-for-desktop kube-system Kube-proxy-znhpr 1/1 Running 0 20d 192.168.65.3 Docker-for-desktop kube-system kube-scheduler-docker-for-desktop 1/1 Running 7 20d 192.168.65.3 Docker-for-desktop kube-system kubernetes-dashboard-5bd6f767c7-z7zQt 1/1 Running 1 15d 10.1.0.93 docker-for-desktop NS-monitor grafana-7bcf9c9F9-2g5bf 1/1 Running 0 8D 10.1.0.103 Docker-for-desktop Ns-monitor node- exporters - VPMM8 1/1 Running 0 7h 192.168.65.3 Docker-for-desktop Ns-monitor Prometheus - 65565d74b8-CTx7b 1/1 Running 0 8D 10.1.0.97 Docker-for-desktop shenhl:~ user$Copy the code
  • The Dashboard Kubernetes

  • Create a namespace called nS-monitor in Kubernetest

Create the file name: namespace yaml, content as follows: 【 file in baidu cloud disk can download links: pan.baidu.com/s/18XHK7ex_… password: 0 qn6 】

apiVersion: v1
kind: Namespace
metadata: 
  name: ns-monitor
  labels:
    name: ns-monitor
Copy the code

At the file directory, run the following command:

kubectl apply -f namespace.yaml
Copy the code
  • Deploy node-exporter in Kubernetest, where node-exporter collects physical indicators such as Memory and CPU of each node in the Kubernetes cluster. It can be directly installed on each physical node. Here, we deploy on each node with DaemonSet and use hostNetwork: true and hostPID: True lets it get Node’s physical metrics, and tolerations is configured to also launch a POD at the master Node.

Create the file name: node – exporter. Yaml files, content is as follows: 【 file in baidu cloud disk can download links: pan.baidu.com/s/18XHK7ex_… password: 0 qn6 】

kind: DaemonSet
apiVersion: apps/v1beta2
metadata: 
  labels:
    app: node-exporter
  name: node-exporter
  namespace: ns-monitor
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      containers:
        - name: node-exporter
          image: prom/node-exporter:v0.16.0
          ports:
            - containerPort: 9100
              protocol: TCP
              name: http
      hostNetwork: true
      hostPID: true
      tolerations:
        - effect: NoSchedule
          operator: Exists

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: node-exporter
  name: node-exporter-service
  namespace: ns-monitor
spec:
  ports:
    - name: http
      port: 9100
      nodePort: 31672
      protocol: TCP
  type: NodePort
  selector:
    app: node-exporter
Copy the code

At the file directory, run the following command:

kubectl apply -f node-exporter.yaml 
Copy the code

Check whether the creation is successful:

kubectl get pods -n ns-monitor -o wide
Copy the code

Verify that Node-Exporter is running successfully

http://127.0.0.1:31672/metricsorhttp://127.0.0.1:9100/metrics

  • Deployment in kubernetest Prometheus official reference documentation: / etc/Prometheus/Prometheus yaml configuration to create the file name Prometheus. Yaml, content is as follows: 【 file in baidu cloud disk can download links: pan.baidu.com/s/18XHK7ex_… password: 0 qn6 】
--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: prometheus rules: - apiGroups: [""] # "" indicates the core API group resources: - nodes - nodes/proxy - services - endpoints - pods verbs: - get - watch - list ...... To save space, please download it from Baidu cloud diskCopy the code

Executing the create command

kubectl apply -f prometheus.yaml
kubectl get pods -n ns-monitor -o wide
Copy the code



Verify Prometheus’s correctness:http://127.0.0.1:31710/graphorhttp://127.0.0.1:31710/service-discoveryorhttp://127.0.0.1:31710/targets





  • Deployment in the kubernetest grafana create grafana. Yaml files, content is as follows: 【 file in baidu cloud disk can download links: pan.baidu.com/s/18XHK7ex_… password: 0 qn6 】
apiVersion: v1 kind: PersistentVolume metadata: name: "grafana-data-pv" labels: name: grafana-data-pv release: stable spec: capacity: storage: 5Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Recycle nfs: path: / NFS /grafana/data server: 192.168.65.3 -- apiVersion: v1 kind: PersistentVolumeClaim Metadata: name: grafana-data-pvc namespace: ns-monitor spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi selector: matchLabels: name: grafana-data-pv release: stable --- kind: Deployment apiVersion: apps/v1beta2 metadata: labels: app: grafana name: grafana namespace: ns-monitor spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: grafana template: metadata: labels: app: grafana spec: containers: - name: grafana image: grafana/grafana:latest env: - name: GF_AUTH_BASIC_ENABLED value: "true" - name: GF_AUTH_ANONYMOUS_ENABLED value: "false" readinessProbe: httpGet: path: /login port: 3000 volumeMounts: - mountPath: /var/lib/grafana name: grafana-data-volume ports: - containerPort: 3000 protocol: TCP volumes: - name: grafana-data-volume persistentVolumeClaim: claimName: grafana-data-pvc --- kind: Service apiVersion: v1 metadata: labels: app: grafana name: grafana-service namespace: ns-monitor spec: ports: - port: 3000 targetPort: 3000 selector: app: grafana type: NodePortCopy the code

Run the create command and view

kubectl apply -f grafana.yaml 
kubectl get pods -n ns-monitor -o wide
Copy the code

Verify that Grafana runs successfully:http://127.0.0.1:30591/loginThe default user name and password are admin and admin

  • Configure Grafana: Configure Prometheus as the data source



    prometheus-service.ns-monitor:9090Source of this link:



  • And then import the Dashboard

  • Put kubernetes Dashboard template imported shows: the JSON format content directly to the replication in line 【 file in baidu cloud disk can download links: pan.baidu.com/s/18XHK7ex_… password: 0 qn6 】
{ "__inputs": [ { "name": "DS_PROMETHEUS", "label": "prometheus", "description": "", "type": "datasource", "pluginId": "prometheus", "pluginName": "Prometheus" } ], "__requires": [ { "type": "grafana", "id": "grafana", "name": "Grafana", "version" : "5.0.4},"... to save space, the content is omittedCopy the code





Each Panel in the Dashboard can edit, save, and roll back itself! If the instance drop-down box displays a problem, click Settings ~ Variables in the upper right to modify the Regex value of the $instance variable. You can clear it directly. Operations such as configuring data sources, importing dashboards, and installing plug-ins can be configured in grafana.yaml file, but the configuration process is rather troublesome. Here are the instructions for operation on the interface, which will be dealt with later.Copy the code



Reference blog:blog.csdn.net/chenleiking… See prometheus. IO /