An overview of the

The Banzai Logging Operator is available in V3. This project is based on Fluentd, using the implementation mode of Operator, in the form of CRD on Kubernetes, to customize the log collection behavior, and carry out filtering, routing and other operations. Finally, logs can be exported to Elasticsearch, Loki, S3, Kafka, and many other backends.

As you can see, Logging divides Logging into three phases: collection, routing, and output. These three stages correspond to three different TYPES of CRD:

  • Acquisition:

    Use Fluentbit to collect logs

  • Routing:

    Multiple criteria are used in FluentD to filter log entries and send the results to the destination

  • Output:

    Various backends can be defined to receive and store logs.

The installation

You can install using Helm:

$ kubectl create ns loggingnamespace/logging created$ helm repo add banzaicloud-stable \    https://kubernetes-charts.banzaicloud.com"banzaicloud-stable" has been added to your repositories$ helm install --namespace logging \    logging banzaicloud-stable/logging-operator \    --set createCustomResource=false...Copy the code

After installation, you should see pods in the Logging namespace:

$ kubectl get po -n loggingNAME READY STATUS RESTARTS AGE... logging-logging-operator-7b4f9987f9-86clp 1/1 Running 0 120mCopy the code

Logging

We can first define a new Logging object:

apiVersion: logging.banzaicloud.io/v1beta1kind: Loggingmetadata:  name: default-logging-simplespec:  fluentd: {}  fluentbit: {}  controlNamespace: loggingCopy the code

Commit to cluster:

$ kubectl apply -f empty-logging.yamllogging.logging.banzaicloud.io/empty-logging created$ kubectl get podsNAME                                         READY   STATUS              RESTARTS   AGEempty-logging-fluentbit-2pghs                0/1     ContainerCreating   0          0sempty-logging-fluentbit-cc4r4                0/1     ContainerCreating   0          0sempty-logging-fluentbit-jzkpr                0/1     ContainerCreating   0          0sempty-logging-fluentd-0                      0/2     Pending...Copy the code

Describe newly generated pods that you will find loaded with several Secret configuration files. For example, fluentbit:

$ kubectl view-secret empty-logging-fluentbitChoosing key: fluent-bit.conf... [OUTPUT] Name forward Match * Host empty-logging-fluentd.logging.svc Port 24240 Retry_Limit FalseCopy the code

The configuration here indicates that logs collected by the collector are output to the empty-logging-Fluentd service.

If you look at the fluentd configuration, you find that the output configuration is empty, that is, no output capability is provided.

Query operator logs:

$ kubectl logs -f -l app.kubernetes.io/name=logging-operator... {" level ", "info", "ts" : 1583835777.4591844, "logger" : "controllers. Logging," "MSG" : "resource created","name":"empty-logging-fluentbit","namespace":"logging","apiVersion":"apps/v1","kind":"DaemonSet"}... {"level":"info","ts":1583835834.1114376," Logger ":"controllers.Logging"," MSG ":"no flows found, generating empty model"}Copy the code

In the output log, the flow definition is missing.

Output

Install Loki using Helm and then define an output:

apiVersion: logging.banzaicloud.io/v1beta1kind: Outputmetadata: name: loki-outputspec: loki:   url: http://loki-1583844504.loki:3100   configure_kubernetes_labels: true   buffer:     timekey: 1m     timekey_wait: 30s     timekey_use_utc: trueCopy the code

After this resource is created, the Secret and the Pod are not changed, even the Operator Pod log is not output, it seems that we still need to create Flow to export the log.

Flow

Create the following Flow object:

apiVersion: logging.banzaicloud.io/v1beta1kind: Flowmetadata:  name: loki-flowspec:  filters:    - tag_normaliser: {}    - parser:        remove_key_name_field: true        reserve_data: true        parse:          type: nginx  match:    - select:        labels:          app.kubernetes.io/name: log-generator  outputRefs:    - loki-outputCopy the code

The filters member identifies what we want to do with the application and adds the Kubernetes tag using tag_normaliser.

IO /name: log-generator = app.kubernetes. IO /name: log-generator = app.kubernetes. IO /name: log-generator

OutputRefs specifies the output to the loki-output created earlier.

After the CRD is created

If you enter the Shell of FluentD Pod, you will find that the configuration has changed:

$ cat fluentd/app-config/fluentd.conf<match **>...    <match>      labels app.kubernetes.io/name:log-generator      namespaces logging...  <match kubernetes.**>    @type tag_normaliser    @id loki-flow_0_tag_normaliser    format ${namespace_name}.${pod_name}.${container_name}  </match>  <filter **>    @type parser    @id loki-flow_1_parser...    <parse>      @type nginx...  <match **>    @type lokiCopy the code

Deploying an application:

apiVersion: apps/v1kind: Deploymentmetadata: name: log-generatorspec: selector: matchLabels: app.kubernetes.io/name: log-generator replicas: 1 template: metadata: labels: app.kubernetes.io/name: log-generator spec: containers: - name: Nginx image: banzaicloud/log - generator: 0.3.2Copy the code

Grafana = Grafana; Grafana = Grafana; Grafana = Grafana;

conclusion

This product well demonstrates the characteristics of Operator solidification operation and maintenance skills. Simplify the complexity, select the complex knowledge required in the log collection process into a series of configuration combinations, and present it to non-expert customers in the form of CRD, out of the box.

Kubectl’s view-secret plugin has a serious Bug, don’t ask me how I know.