In this article, we will install an Etcd cluster and configure monitoring with Prometheus and Grafana, all through Rancher.

We’ll see how easy it is to make use of Rancher’s app store without having to rely on it. In this article, we will not need:

  • Configure the interactive box specifically for running Kubectl and point to the Kubernetes cluster
  • Kubectl, because we can do all this using the Rancher UI
  • Install/configure Helm Binary

Preparation for Demo

You will need:

  • A Google Cloud Platform account (free). Any other cloud will do.
  • Rancher V2.4.7 (current as of this writing)
  • Kubernetes cluster running on GKE (version 1.16.3-Gke.1) (running on EKS or AKS is also fine)

Start a Rancher instance

First, start your instance of Rancher. You can check out the quick Start guide at the following link:

www.rancher.cn/quick-start…

Deploy a GKE cluster using Rancher

To set up and configure a Kubernetes cluster using Rancher, you can check out the documentation:

Docs. The rancher. Cn/docs/ranche…

Deploy ETCD, Prometheus, and Grafana

We can use the Rancher app store to install all the software. The app store is a collection of Helm Charts that make it easy for users to deploy these applications repeatedly.

When our cluster is up and running, let’s select the Default project we created for it. In the Apps TAB, click Launch.

The first application to install is etcD-operator. Leave all the default values it prepopulated with, and make sure you also enable the creation of the ETCD cluster (we deselect the ETCD Backup Operator and the ETCD Restore Operator for demo simplicity).

The role of an Operator is to observe, analyze, and act. It uses the Kubernetes API to observe the state of the current cluster. If there are any differences between the running state and the required state, it finds them and fixes them.

For example, suppose we are running an ETCD cluster with three members. If something happens and one of the members falls down, the Operator will observe this. It makes a difference based on the required state and recovers the missing member based on the difference. As a result, we have a healthy cluster without human intervention.

To install Prometheus and Grafana, activate the cluster monitoring support integrated in Rancher. From the Global view, select the cluster you want to configure and select Tools → Monitor to enable it. To allow persistent changes to Grafana, be sure to enable persistent storage for Grafana and Prometheus. If you don’t have any persistent storage, check out Longhorn, Kubernetes’ cloud-based distributed block storage.

There are a few tabs you can explore while everything is installed. Check the progress of the workload (Pods, Deployments, DaemonSet) or the services created.

Let’s connect to an ETcd Pod to use some basic etcdctl commands (see the previous article for more details). Select a Pod, click its vertical ellipsis (3 vertical dots) menu button, and select Execute Shell.

Configure Prometheus and Grafana

One of the best and easiest ways to monitor an ETCD cluster is to use Prometheus and Grafana. Let’s log in to Grafana – click any Grafana icon in the cluster overview to log in.

Grafana has pre-configured Prometheus as the data source and contains several dashboards to visualize the cluster status.

Log in to Grafana to add a dashboard to your ETCD. The default user name and password are both “admin” (you will be prompted to change them on your first login). Then import the default ETCD dashboard template with ID3070. Click Load, and the remaining step is to select the Prometheus data source.

We’ve successfully imported dashboard, and we can see various charts, but no data. Why is that? We already have Prometheus running and Grafana integrated with it. The problem is that we didn’t tell Prometheus to collect targets related to our ETCD cluster.

Let’s go back to the Rancher to solve this problem. Go to the system project and click import YAML under the resources TAB. Then import the following resources into the cattle-prometheus namespace:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    source: rancher-monitoring
  name: etcd
  namespace: cattle-prometheus
spec:
  endpoints:
    - port: client
  namespaceSelector:
    matchNames:
      - etcd-operator
  selector:
    matchLabels:
      app: etcd
Copy the code

How do we verify that our new configuration is working and that Prometheus is doing its job? To check this out, click on the Apps TAB of the system project and click the second /index/.html link in the Cluster-Monitoring application.

This opens the Prometheus Web UI interface. On the interface, go to Graphs and perform some queries manually. If any data is displayed, then our Settings are complete.

The last thing we need to do is look at the Grafana and see the data graphs that we have.

Uninstall the application and cluster

To clean up the resources we use in this article, all we need to do is select our cluster at the global level and click [Delete].

By doing so, everything except the persistent storage created for Prometheus will be deleted. We need to handle this from our cloud provider console.

Of course, we could just do the cleanup from the Rancher, but the steps are slightly different.

  • Disable Monitoring: At the global level, navigate to the cluster, select Tools → Monitor, and click the disable button.
  • Remove persistent storage: go to “System Item “→” Resource “→” Workload “→” Volume”; Select your volume and click delete.
  • Delete a cluster: Select a cluster at the global level and delete it.

Total knot

In this demo, we saw how to install Etcd (using the ETCD-operator), Prometheus, and Grafana using the Rancher. All integration is out of the box: we just need to add a few things to complete all the configuration. The Rancher also provides all the visibility needed to easily troubleshoot if necessary.