preface

After the previous series of articles, we should have a simple introduction to the K8S. At the beginning of this article, we will gradually explain some advanced knowledge points. In previous articles, we focused on how to fully publish an externally accessible service. The key point is springboot+ VUejS front and back end separation project. Future articles will also focus on SpringBoot +vuejs, but will be based on it to extend new services such as Redis, RabbitMQ, ELK, etc.

About Health Testing

A Health Check is a simple way to let the system know if your application instance is working properly. If your application instance is no longer working, other services should not access the application or send requests to it. Instead, send the request to the prepared application instance or try again later. The system should also be able to restore your application to health.

By default, when all containers in a Pod start, Kubernetes starts sending traffic to the Pod and restarts the container in the event of a crash. It is not very reasonable for Pod to start external services immediately. Because the Pod startup success does not mean that the container deployed services can be external services, such as the Springboot project, container startup success, Springboot startup needs some initialization work, really to be able to access the external, as soon as a few seconds, slow takes more than 10, 20 seconds, or even longer. Kubernetes is designed with this in mind and can make deployments more robust by creating custom health checks. There is a keyword involved here: container probe.

Container probe

Probes are periodic diagnostics performed by Kubelet on containers. To perform diagnostics, Kubelet calls the Handler implemented by the container. There are three types of handlers:

  • ExecAction: Executes the specified command in the container. If the return code is 0, the diagnosis is successful.
  • CPSocketAction: TCP checks the IP address of the container on the specified port. If the port is open, the diagnosis is considered successful.
  • HTTPGetAction: Performs an HTTP Get request for the container’s IP address on the specified port and path. If the status code of the response is greater than or equal to 200 and less than 400, the diagnosis is considered successful.

Each probe yields one of three results:

  • Success: The container passed the diagnosis.
  • Failed: The container failed the diagnosis.
  • Unknown: Diagnosis failed, so no action will be taken.

Kubelet can choose whether to execute and react to both probes running on the container:

  • livenessProbe(Survival probe) : Indicates whether the container is running. If the survival probe fails, Kubelet kills the container and the container is affected by its restart strategy. If the container does not provide a survival probe, the default state isSuccess.
  • readinessProbeReady probe: Indicates whether the container is ready for a service request. If the ready probe fails, the endpoint controller removes the Pod’s IP address from the endpoints of all services that match the Pod. The ready state before the initial delay defaults toFailure. If the container does not provide ready probes, the default state isSuccess.

High-level LIVENESS probe example

The definition of probes is also an important part of K8S choreography.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mldong-admin
  namespace: mldong-admin-test
spec:
  selector:
    matchLabels:
      app: mldong-admin
  replicas: 1
  template:
    metadata:
      labels:
        app: mldong-admin
    spec:
      containers:
        - name: mldong-admin
          env:
            - name: TZ
              value: Asia/Shanghai
          image: registry.cn-zhangjiakou.aliyuncs.com/mldong/java/mldong-admin:202007220017_c608761
          livenessProbe:
              httpGet:
                # PodIP = "PodIP"
                # host: my-host
                Scheme allows only "HTTP" and "HTTPS" when "scheme" is not defined
                # scheme: HTTPS
                path: /healthz This requires the service to have this interface address
                port: 8080     The port corresponding to the service
                httpHeaders:   Request headers can be carried
                - name: X-Custom-Header
                  value: Awesome
              initialDelaySeconds: 30   # The time of the first health check-up
              periodSeconds: 5    # Check cycle
              timeoutSeconds: 5   Check the timeout
              successThreshold: 1 The number of successes determines success
              failureThreshold: 1 Failed to count the number of failures
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
              name: port
              protocol: TCP
          command: ["/bin/sh"]
          args: ["-c"."set -e && java -jar app.jar --spring.profiles.active=test --server.port=8080"]
Copy the code

Probe types livenessProbe and readinessProbe are the same as the image defined by the container. Both can exist at the same time. The former defines the survival probe, checks the health status, and periodically checks whether the service is alive. The latter defines ready probes, availability checks, which periodically check whether a service is available and remove it from the service’s endpoints.

As described above, we define a livenessProbe that uses httpGet to check the request path /healthz, port 8080, and httpHeaders x-custom-header =Awesome. InitialDelaySeconds for the first health check is 30 seconds after POD is started. PeriodSeconds of the check period is 5s. Check the timeout interval timeoutSeconds is 5s. If the successThreshold is 1, success is determined. FailureThreshold: 1 Indicates a failure.

Description of inspection methods:

  • httpGetCheck the return status of an HTTP request 2xx,3xx normal, 4xx,5xx error

Parameters that

parameter The default value instructions
host When “host” is not defined, use “PodIP” The request of the domain name
scheme HTTP Only “HTTP” and “HTTPS” are allowed.
path / Request address, which needs to be present in the interface service
port 80 Port corresponding to the service
httpHeaders There is no Request header name/value

Sample:

httpGet:
    path: /healthz
    port: 8080
    httpHeaders:
    - name: X-Custom-Header
      value: Awesome
Copy the code
  • execExecuting a command returns 0, not 0

Parameter Description:

parameter The default value instructions
command Array of commands to execute

Sample:

exec: # Execution mode
    command:  # initial command
    - cat
    - /tmp/healthy
Copy the code
  • tcpSocket Tests whether a port can be connected

Parameter Description:

parameter The default value instructions
port 80 port

Sample:

tcpSocket:
	port: 80
Copy the code

Check Rule Description

parameter The default value instructions
initialDelaySeconds Too little time for the first health check may cause pod to restart indefinitely.
periodSeconds Check the cycle
timeoutSeconds Check timeout
successThreshold 1 Success count Indicates the success
failureThreshold 1 Failure count Indicates the failure

Practical demonstration

Before going into combat, learn the following commands:

for a in {1.. 10}; do curl http://vueadmin.mldong.com/api/login; date +"%Y%m%d%H%M%S"; sleep 2; done;

The interface is accessed every 2 seconds for 10 cycles.

The renderings are as follows:

Live probe

  1. newv1.yaml
cat << EOF > /mldong/k8s/mldong-probe/v1.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: mldong-probe
---
apiVersion: v1
kind: Service
metadata:
  name: mldong-admin-nodeport
  namespace: mldong-probe
spec:
  type: NodePort
  ports:
  - port: 8080
    targetPort: 8080
  selector:
    app: mldong-admin
---
apiVersion: v1
kind: Service
metadata:
  name: mldong-admin
  namespace: mldong-probe
spec:
  type: ClusterIP
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: mldong-admin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mldong-admin
  namespace: mldong-probe
spec:
  selector:
    matchLabels:
      app: mldong-admin
  replicas: 1
  template:
    metadata:
      labels:
        app: mldong-admin
    spec:
      containers:
        - name: mldong-admin
          env:
            - name: TZ
              value: Asia/Shanghai
          image: registry-vpc.cn-zhangjiakou.aliyuncs.com/mldong/java/mldong-admin:202007220017_c608761
          livenessProbe:
              tcpSocket:
                port: 8080     The port corresponding to the service
              initialDelaySeconds: 15  # The time of the first health check-up
              periodSeconds: 5    # Check cycle
              timeoutSeconds: 5   Check the timeout
              successThreshold: 1 The number of successes determines success
              failureThreshold: 1 Failed to count the number of failures
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
              name: port
              protocol: TCP
          command: ["/bin/sh"]
          args: ["-c"."set -e && java -jar app.jar --spring.profiles.active=test --server.port=8080"]
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
  name: mldong-admin-ingress
  namespace: mldong-probe
spec:
  rules:
    - host: a.test.com
      http:
        paths:
          - backend:
              serviceName: mldong-admin
              servicePort: 8080
            path: /
EOF
Copy the code

  1. Start another terminal and run the following command:
kubectl get pods -A -w | grep mldong-probe
Copy the code
  1. On the old terminal, run the following command:
kubectl apply -f v1.yaml
Copy the code

Normal startup effect is as follows:

4. Delete redo

kubectl delete -f v1.yaml
Copy the code

Change the port to another non-service port

tcpSocket:
  port: 8081
Copy the code
  1. Rerun the
kubectl apply -f v1.yaml
Copy the code

The effect is as follows:

If the listening service does not exist, restart the container.

And finally, clear the space

kubectl delete -f v1.yaml
Copy the code

Ready probe

  1. The new v2. Yaml

    cat << EOF > /mldong/k8s/mldong-probe/v2.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: mldong-probe
---
apiVersion: v1
kind: Service
metadata:
  name: mldong-admin-nodeport
  namespace: mldong-probe
spec:
  type: NodePort
  ports:
  - port: 8080
    targetPort: 8080
  selector:
    app: mldong-admin
---
apiVersion: v1
kind: Service
metadata:
  name: mldong-admin
  namespace: mldong-probe
spec:
  type: ClusterIP
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: mldong-admin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mldong-admin
  namespace: mldong-probe
spec:
  selector:
    matchLabels:
      app: mldong-admin
  replicas: 1
  template:
    metadata:
      labels:
        app: mldong-admin
    spec:
      containers:
        - name: mldong-admin
          env:
            - name: TZ
              value: Asia/Shanghai
          image: registry-vpc.cn-zhangjiakou.aliyuncs.com/mldong/java/mldong-admin:202007220017_c608761
          readinessProbe:
              tcpSocket:
                port: 8080     The port corresponding to the service
              initialDelaySeconds: 15  # The time of the first health check-up
              periodSeconds: 5    # Check cycle
              timeoutSeconds: 5   Check the timeout
              successThreshold: 1 The number of successes determines success
              failureThreshold: 1 Failed to count the number of failures
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
              name: port
              protocol: TCP
          command: ["/bin/sh"]
          args: ["-c"."set -e && java -jar app.jar --spring.profiles.active=test --server.port=8080"]
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
  name: mldong-admin-ingress
  namespace: mldong-probe
spec:
  rules:
    - host: a.test.com
      http:
        paths:
          - backend:
              serviceName: mldong-admin
              servicePort: 8080
            path: /
EOF
Copy the code

  2. Add the host configured in 1 to /etc/hosts

  3. Start another terminal and run the following command

    for a in {1.. 30}; do curl http://a.test.com/api/login; date +"%Y%m%d%H%M%S"; sleep 2; done;Copy the code

  4. Run the following command on the old terminal

    kubectl apply -f v2.yaml
Copy the code

    The effect is as follows:

    The start service does not exist and accesses the default service of nginx-ingress. After the service is started, the interface can be accessed normally. This is just an example of a single service startup. In fact, this is also the case when we do the system version upgrade. For example, to upgrade from service version 1 to service version 2, we just need to change the version of the image and then execute kubectl apply -f k8s.yaml. In this way, new versions of the service can be constantly updated.

And finally, clear the space

kubectl delete -f v2.yaml
Copy the code

summary

This paper briefly introduces k8S health check – survival probe and ready probe, and makes a simple case demonstration. The purpose of using both live and ready probes is to make our service more robust. You can choose which type of probe and inspection method to use according to your service situation. Spring Boot 2.3 is said to offer K8s activity and readiness probes for those interested.

Project source code address

  • The back-end

Gitee.com/mldong/mldo…

  • The front end

Gitee.com/mldong/mldo…

Related articles

Walk you through K8S – cluster creation and Hello World

Take you through K8S-ConfigMap and persistent storage

Take you hand in hand to play K8S – complete release of an externally accessible service

Docker-compose k8S-docker advanced Dockerfile and docker-compose

K8s – One click deployment of springboot project

Take you through k8S – One-click deployment of VUE projects

Take you hand in hand to play K8S – common object details

Walk you through k8S-Jenkins installation and assembly line

Walk you through k8S-Jenkins assembly line grammar

Take you hand in hand through the K8S-Jenkins assembly line to launch springboot project

Take you through the K8S-Jenkins assembly line to launch vUE projects