Description: iLogtail is the collection part of the simple log service, also known as SLS, in Ali Cloud. It is used to collect telemetry data, such as logs, traces, and metrics, and is now officially open source (github.com/alibaba/ilo…

Preparation before use

  • Opened ali Cloud log service and created a Project (refer to the previous section “Ilogtail Starting to Use – Host Environment Log Collection SLS” for specific steps)
  • Prepare an X86-64 K8S cluster with public network access permission.

Creating a Log Configuration

  1. Jump to the logging services console (sls.console.aliyun.com) and click on the Project you created in the previous section.

2. After entering the Project query page, click the Magnifying Glass icon in the left sidebar to expand the LogStore management page. Click + to display the right sidebar of Create LogStore. Enter the logStore name and click OK.

3. After logStore is created, cancel the data access wizard. Click the Cube button in the left sidebar and select Machine Group from the resource float that pops up. In the left bar of the expanded Machine Group, click the “four squares” icon in the upper right corner and select “Create machine Group” in the floating layer that pops up.

4. On the Create Machine Group sidebar, set Machine Group ID to User-defined ID as prompted. You are advised to keep Name, Machine Group Topic, and User-defined ID the same. “User defined identity” is one of the most important configurations, and “my-k8S-group” is used in this tutorial and will be used again when ilogTail is installed. Click to save the machine group.

5. Click the Magnifying Glass icon in the left pane to expand the LogStore management page. Click the Expand Down icon of the LogStore created in Step 2 to display the Configure LogStore menu. Click the “+” button of “Logtail Configuration”.

6. In the Quick Access Data dialog box that is displayed, search for kube and select Kubernertes- Files. In the Information dialog box that is displayed, click Continue.

7. On the Kubernertes File configuration page, select Use Existing machine Group.

8. Go to the Configure Machine Group page. Select the machine group created in Step 4, click > to add it to Application Machine Group, and then click Next.

9. In ilogtail configuration, modify only configuration Name and Log Path, and click Next.

10. Complete the index configuration. This step does not modify any options, directly click next to complete the configuration.

At this point, the log configuration is complete. Please keep the page open.

Install ilogtail

  1. Log in to the central computer that can control the K8S cluster. Edit Ilogtail’s ConfigMap YAML.

    $ vim alicloud-log-config.yaml

Paste the following in Vim and save it (note, modify the fields prompted in the comment, lines 7-11).

apiVersion: v1 kind: ConfigMap metadata: name: alibaba-log-configuration namespace: kube-system data: log-project: "My - project" # modified for practical project name log - the endpoint: "cn-wulanchabu.log.aliyuncs.com" # modified for the actual endpoint log - machine - group: "My-k8s-group" # alter machine group name log-config-path: "/etc/ilogtail/conf/cn-wulanchabu_internet/ilogtail_config.json" "* * * * * * * * *" # modified for ali cloud UID access - key - id: "" # this tutorial without access to key - secret:" "# this tutorial without CPU core - limit:" 2 "mem - limit: "1024" max-bytes-per-sec: "20971520" send-requests-concurrency: "20"Copy the code

2. Calculate the Sha256 hash of alicloud-log-config.yaml and edit ilogtail’s DaemonSet YAMl.

$ sha256sum alicloud-log-config.yaml
f370df37916797aa0b82d709ae6bfc5f46f709660e1fd28bb49c22da91da1214  alicloud-log-config.yaml
$ vim logtail-daemonset.yaml
Copy the code

Paste the following in Vim and save it (note, modify the fields prompted in the comment, lines 21, 25).

apiVersion: apps/v1 kind: DaemonSet metadata: name: logtail-ds namespace: kube-system labels: k8s-app: logtail-ds spec: selector: matchLabels: k8s-app: logtail-ds updateStrategy: type: RollingUpdate template: metadata: labels: k8s-app: IO /cluster-service: "true" version: v1.0 Annotations: checksum/config: F370df37916797aa0b82d709ae6bfc5f46f709660e1fd28bb49c22da91da1214 # must be modified for alicloud - log - config. Yaml hash spec: containers: - name: logtail image: registry.cn-wulanchabu.aliyuncs.com/log-service/logtail:latest # can be modified for recent distance k8s cluster regional resources: limits: cpu: 2 memory: 1024Mi requests: cpu: 100m memory: 256Mi livenessProbe: httpGet: path: /liveness port: 7953 scheme: HTTP initialDelaySeconds: 30 periodSeconds: 60 securityContext: privileged: true env: - name: HTTP_PROBE_PORT value: "7953" - name: "ALIYUN_LOGTAIL_CONFIG" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: log-config-path - name: "ALIYUN_LOGTAIL_USER_ID" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: log-ali-uid - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: log-machine-group - name: "ALICLOUD_LOG_ACCESS_KEY_ID" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: access-key-id - name: "ALICLOUD_LOG_ACCESS_KEY_SECRET" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: access-key-secret - name: "ALICLOUD_LOG_DOCKER_ENV_CONFIG" value: "true" - name: "ALICLOUD_LOG_ECS_FLAG" value: "false" - name: "ALICLOUD_LOG_DEFAULT_PROJECT" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: log-project - name: "ALICLOUD_LOG_ENDPOINT" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: log-endpoint - name: "ALICLOUD_LOG_DEFAULT_MACHINE_GROUP" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: log-machine-group - name: "ALIYUN_LOG_ENV_TAGS" value: "_node_name_|_node_ip_" - name: "_node_name_" valueFrom: fieldRef: fieldPath: spec.nodeName - name: "_node_ip_" valueFrom: fieldRef: fieldPath: status.hostIP # resource limit for logtail self process - name: "cpu_usage_limit" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: cpu-core-limit - name: "mem_usage_limit" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: mem-limit - name: "max_bytes_per_sec" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: max-bytes-per-sec - name: "send_request_concurrency" valueFrom: configMapKeyRef: name: alibaba-log-configuration key: send-requests-concurrency volumeMounts: - name: sock mountPath: /var/run/ - name: root mountPath: /logtail_host readOnly: true mountPropagation: HostToContainer terminationGracePeriodSeconds: 30 tolerations: - operator: "Exists" hostNetwork: true dnsPolicy: "Default" volumes: - name: sock hostPath: path: /var/run/ - name: root hostPath: path: /Copy the code

3. Apply YAML configuration to create ConfigMap and DaemonSet.

$ kubectl apply -f alicloud-log-config.yaml
configmap/alibaba-log-configuration created
$ kubectl apply -f logtail-daemonset.yaml
daemonset.apps/logtail-ds created
Copy the code

4. Wait 1 minute and check whether DeamonSet runs normally

$ kubectl get -f logtail-daemonset.yaml
Copy the code

At this point, the console should print a message similar to the following, indicating successful installation

NAME         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
logtail-ds   3         3         3       3            3           <none>          2m1s
Copy the code

Report and view logs

  1. Create a Pod for continuous logging.

    $ vim demo-pod.yaml

Paste the following in Vim and save it (note that you may need to modify the fields prompted in the comment, lines 8-9).

apiVersion: v1
kind: Pod
metadata:
  labels:
    name: demo-pod
  name: demo-pod
spec:
#  imagePullSecrets:          # Comment out to enable specific image pull secret
#    - name: myregistrykey    # repleace it to specific registry key containers
containers:
    - image: busybox
      imagePullPolicy: IfNotPresent
      name: demo-pod
      command: ["/bin/sh"]
      args: ["-c", "while true; do echo $(date) >>/tmp/demo.log; sleep 10; done"]
      resources: {}
      securityContext:
        capabilities: {}
        privileged: false
      terminationMessagePath: /dev/termination-log
  dnsPolicy: ClusterFirst
  restartPolicy: Always
Copy the code

Apply YAML configuration to create Pod

$ kubectl apply -f demo-pod.yaml
pod/demo-pod created
Copy the code

2. Return to the Web console, and click Query Log on the configuration page to switch to the log query page. Click the “Magnifying Glass” icon on the left of the page, select LogStore, click the “Eye” icon, try to adjust the Shard and time range in the “Consumption Preview” sidebar on the left, and click Preview to view the reported logs.

What’s Next

You can go to the next section to learn how to Use IlogTail – IlogTail Local Deployment Solution

Understand the principle of container collection: Collecting Container Standard Output through DaemonSet- Console

You can also learn more about ilogtail collection:

  • Collect the container text log in DaemonSet- console mode
  • Collecting Container Logs
  • How do I get the container’s Label and environment variables

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.