Make writing a habit together! This is the fourth day of my participation in the “Gold Digging Day New Plan · April More text Challenge”. Click here for more details.

Pod is the basic unit in Kubernetes, and the container itself is not allocated directly to the host, but encapsulated in a Pod object. A Pod typically represents a single application, consisting of one or more related containers that have the same lifecycle and are scheduled on the same Node as a whole, sharing the environment, storage volumes, and IP controls. Although there may be multiple containers in a Pod, in Kubernetes the Pod is the smallest unit for scheduling, scaling, and sharing resources, and managing the lifecycle.

Basic operation of Pod

Let’s start by looking at Pod creation, query, modification, and deletion.

Create a Pod

# expod.yml

apiVersion: v1
kind: Pod
metadata:
  name: expod
spec:
  containers:
  - name: expod-container
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ['sh', '-c']
    args: ['echo "Hello Kubernetes!"; sleep 3600']
Copy the code

Simple template meaning:

  • ApiVersion indicates the API version. V1 indicates the stable version using kubernetes API.
  • Kind represents the resource object to be created.
  • Metadata represents the metadata of the resource object. You can have multiple metadata. Name indicates the name of the current resource.
  • Spec represents the specific Settings for the resource object. Containers is a collection of containers, and we have a simple container here. Name: Name of the container to be created. Image: indicates the image address of the container. ImagePullPolicy: indicates the download policy of an image. Always, Never, and IfNotPresent are supported. Command: a list of commands used to start a container. If this command is not configured, internal commands are used. Args: indicates the list of startup parameters

Run the command to create the Pod.

kubectl apply -f expod.yml
Copy the code

Once created, query all the pods that are currently running

kubectl get pod
Copy the code

Query the Pod

There are multiple commands to query Pod information, and the query details are different:

  • Query all pods in the default namespace
kubectl get pod
Copy the code
  • Query information about a Pod
kubectl get pod expod
Copy the code
  • Continuous monitoring of Pod status
kubectl get pod -w
Copy the code
  • More summary information is displayed
kubectl get pod -o wide
Copy the code

IP addresses and owning nodes in the cluster are displayed more than the brief information

  • Output Pod information in the specified format
kubectl get pod expod -o yaml
Copy the code
  • Displays the most detailed information, including events
kubectl describe pod expod
Copy the code

This command is the most commonly used resource information query command. The command displays comprehensive information, including basic resource information, container information, preparation, storage volume information, and related event list. If a problem occurs during resource deployment, you can use this command to query details and analyze the fault cause.

  • Example Query Pod logs
Kubectl logs Pod nameCopy the code

Modify the Pod

You can modify existing Pod properties using the replace command

Kubectl replace -f pod yaml fileCopy the code

“Hello Kubernetes!” “Hello Kubernetes!” Change to “Hello Kubernetes press!” .

# expod.yml

apiVersion: v1
kind: Pod
metadata:
  name: expod
spec:
  containers:
  - name: expod-container
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ['sh', '-c']
    args: ['echo "Hello Kubernetes replaced!"; sleep 3600']
Copy the code

There are many properties of Pod that cannot be modified. If you must modify them, you need to add the –force parameter, which is equivalent to rebuilding Pod.

kubectl replace -f expod.yml --force
Copy the code

Take a look at the command output

Delete the Pod and create a replacement Pod for expod.

kubectl logs expod
Copy the code

You can see that the Pod information has been modified.

Delete the Pod

Removing a Pod is as simple as executing the following command:

kubectl delete pod expod
Copy the code

If we are creating from a Pod template file, it is recommended to use a template-file-based delete command.

kubectl delete -f expod.yml
Copy the code

Pod and container

As we may have seen, Pod templates are very similar to Docker-compose, but Pod templates can be configured with more and more complex parameters.

How Pod creates containers

In the Containers part of the Pod template, specify the deployment mode of the container, which will be converted into the corresponding container to run commands during the deployment process. Take our initial Pod template as an example:

apiVersion: v1
kind: Pod
metadata:
  name: expod
spec:
  containers:
  - name: expod-container
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ['sh', '-c']
    args: ['echo "Hello Kubernetes!"; sleep 3600']
Copy the code

When kubernetes performs scheduling, the following command is executed:

docker run --name expod-container busybox sh -c 'echo "Hello Kubernetes!" ; sleep 3600'Copy the code

The command and args Settings override EntryPoint and CMD defined in the Docker image, respectively.

The way Pod organizes containers

Pod is a whole composed of containers and is scheduled to a Node at the same time. Containers can share resources, network environment and dependencies, and have the same life cycle.

How does a container form a Pod

Each Pod contains one or a set of closely related business containers, but there is also a special Pause container that becomes the “root container.” Pause containers are part of Kubernetes. If a group of Pause containers is mounted as a whole, it is difficult to determine whether the whole Pod is mounted. This problem can be solved by introducing a business-neutral Pause container, using it as the root container for the Pod, and its state as a proxy for the state of the entire group of containers. In addition, all containers in a Pod share the IP address of the Pause Container and its attached storage volume, which simplifies communication and data sharing between containers, similar to the Container mode of the Docker Container network.

You can log in to the corresponding Node machine to view the container information for the Pod you started creating.

kubectl get pod -o wide
Copy the code

After logging in to k8S-node2, run the docker ps command to view the details:

You can see that there are three pause containers, two of which are Flannel and Proxy containers, and one of which is our ExPOD container, which together with another ExPOD container forms a Pod.

Containers in PODS share two resources – storage and networking.

  • storage

In Pod, we can specify one or more storage volumes that are accessible to all containers in Pod. Storage volumes can be temporary or persistent.

  • network

In a Kubernetes cluster, each Pod is assigned a unique IP address, and containers in pods share a network namespace, including IP addresses and network ports. Containers within a Pod can communicate with each other via localhost. We can compare the difference between Docker and Kubernetes in cyberspace.

Docker’s cyberspace

As can be seen from the figure, containers are connected through docker0 network card, and each container has an independent internal network address

Kubernetes’ cyberspace

As you can see from the figure, all containers in the Pod share a network address

How do PODS communicate with each other

The communication between PODS is mainly divided into two cases:

  • Communication between pods on the same Node

All pods on the same Node use the same bridge (Docker0) to connect to each other, which is equivalent to the Docker network space, but based on pods. Each Pod has a global IP address. Different pods in the same Node are connected to the same Docker0 bridge through veth. Their IP addresses are dynamically obtained from the Docker0 bridge and associated with the same Docker0 bridge. Address segments are the same, so they can communicate directly with each other.

  • Pod communication between different nodes

To communicate between pods of different nodes, first ensure that pods have globally unique IP addresses in a Kubernetes cluster. And because the Pod on a Node communicates with the outside through the Docker bridge, this function can be realized by configuring the Docker bridge on different nodes into different IP address segments. Flannel can configure the Docker bridge by modifying the startup parameter BIP of Docker. In this way, the Docker bridge of the machines in the cluster can get a globally unique IP address segment, and the containers created on the machines can also have globally unique IP addresses.

Communication between pods across nodes

When we set up the Kubernetes cluster, we created a Kube-Flannel container on each Node machine. This uses the Flannel virtual network card on each Node machine to take over the container and communicate with each host. When the container of one Node visits the container of another Node, Data on the source node is routed from the Docker0 network bridge to the Flannel0 network card, and data on the destination node is routed from the Flannel0 network card to the Docker0 network card, and then forwarded to the target container. Flannel replanned the container cluster network to ensure global uniqueness of container IP addresses and enable containers on different machines to communicate with each other using Intranet IP addresses. Flannel assigns only subnet addresses. Therefore, IP addresses for containers are dynamically allocated within the subnet.

Because the Pod IP address itself is a virtual IP address, only machines inside the Kubernetes cluster (Master and Node) and other pods can directly access this IP address. Machines outside the cluster cannot directly access the Pod IP address. We created an Nginx Pod:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: exnginx
    image: nginx
    imagePullPolicy: IfNotPresent
    ports:
    - name: nginxport
      containerPort: 80
Copy the code

Run the following command to query the following IP addresses:

[root@k8s-master]# kubectl apply -f exnginx.yml
[root@k8s-master]# kubectl get pod -o wide
Copy the code

All machines and pods in the cluster have access to this virtual address and the port exposed by the containerPort

[root @ k8s - master] # curl http://10.244.1.6Copy the code

At the same time, we see that we have two pods, one on Node1 and one on Node2, and the Pod IP address on Node2 is 10.244.2.6, and the Pod IP address on Node1 is 10.244.1.6. Run the ifconfig flannel.1 command to check the cluster subnet segment.

k8s-node1

K8s-node2

To enable machines outside the cluster to access services provided by pods, we’ll cover Service and Ingress later to publish services.

The Pod life cycle

The state of the Pod

Once a Pod is created on a cluster node, it will first enter the Pending state. As long as all containers in the Pod are started and Running properly, the Pod will then enter the Running state. If the Pod is terminated and all containers terminate with a status code of 0, The Pod will enter the Succeeded state.

There are three reasons for the Failed state.

  • When a container fails to run, the Pod will change from Pending to Failed.
  • The Pod is in the Running state. If a container in the Pod is suddenly damaged or the status code is not 0 upon exit, the Pod will enter the Failed state from Running.
  • When the Pod is required to shut down normally, the Pod will enter the Failed state as long as a container exit status code is not 0.

The Pod restart policy

There is a spec module in the Pod configuration template file that has a field called restartPolicy with values of Always, OnFailure, and Never. Kubelet restarts the Node using restartPolicy. Kubelet restarts the exited container with an incremental delay (10s, 20s, 40s…). Try a reboot with an upper limit of 5 minutes. The cumulative value of the delay is reset after a successful run of 10 minutes. Once a Pod is bound to one node, it will never be rebound to another node.

The impact of the restart policy on the Pod status is as follows:

  • Suppose there is a running Pod containing a container that exits successfully.

Always: Restarts the container. The Pod status is still Running.

OnFailure: The Pod status changes to Completed.

Never: The Pod status changes to Completed.

  • Suppose there is a running Pod containing a container that exits after a failure.

Always: Restarts the container. The Pod status is still Running.

OnFailure: Restart the container with Pod status still Running.

Never: The Pod status changes to Failed.

  • Suppose you have a running Pod containing two containers after the first container exits.

Always: Restarts the container. The Pod status is still Running.

OnFailure: Restart the container with Pod status still Running.

Never: The container will not be restarted and the Pod status is still Completed.

  • Suppose the first container doesn’t run and the second container exits.

Always: Restarts the container. The Pod status is still Running.

OnFailure: Restart the container with Pod status still Running.

Never: The Pod status changes to Failed.

  • Suppose you have a running Pod containing a container that runs out of memory.

Always: Restarts the container. The Pod status is still Running.

OnFailure: Restart the container with Pod status still Running.

Never: Records Failed events, and the Pod status changes to Failed.

The Pod is created during the destruction process

The Pod creation process:

  • The kubectl command translates into a call to the API Server.
  • The API Server validates the request and saves it to etCD.
  • Etcd notifies API Server.
  • The API Server calls the scheduler.
  • The scheduler decides which node to run the Pod on and returns it to the API Server.
  • API Server saves its corresponding node in etCD.
  • Etcd notifies API Server.
  • API Server invokes Kubelet in the corresponding node.
  • Kubelet interacts with the container runtime API, communicating with the container daemon to create the container.
  • Kubelet updates the Pod status to the API Server.
  • API Server saves the latest state to etCD.

Pod destruction process:

  • The user sends the command to delete the Pod.
  • The Pod object in the API Server will be updated to set the approximate time for the Pod to be “destroyed” (default: 30s), after which the Pod will be forcibly terminated.
  • The Pod is marked as Terminating. When Kubelet finds that the Pod has been marked Terminating, the Pod shutdown process will be performed. The Endpoint controller detects that the Pod is about to be deleted and overflows all the endpoints related to the Pod in the Service object.
  • If Pod defines a preStop callback, this will be executed in Pod, and if preStop is still running after the grace period, API Server will be notified to add a small grace period (2s).
  • The process in the Pod receives the TERM signal.
  • If the grace period expires and the process in Pod is still running, it is terminated by SIGKILL.
  • Kubelet through the API Server set the grace period to 0 (delete immediately), complete the Pod removal operation, Pod removed from the API.

The default deletion delay is 30 seconds. The kubectl delete command supports –grace-period= seconds. You can customize the delay time. If this value is set to 0, Pod is forcibly removed, but with –grace-period=0 the option –force is added.

The Pod life cycle time

There are two major phases of Pod operation, the first phase is the initial container operation phase, and the second phase is the formal container operation phase. Each phase has different events

Initialize the container run phase

Pod in the initialization can contain one or more container, the container is running before application container formal operation, mainly responsible for some of the initialization, all initialization after the container to perform to perform application container, therefore initialization container can’t be a long running container, after but perform a certain operation must be ended. If multiple initialization containers are used, they can only be executed sequentially, not simultaneously. All initialization containers must end normally before the application container can be started.

When the container fails to initialize, if restartPolicy is OnFailure or Always, the failed container will be initialized repeatedly until it succeeds. If restartPolicy is Never, the failed initialization container will not be restarted.

As an example, check that the DB is ready and execute the following initialization script before deploying the application.

apiVersion: v1
kind: Pod
metadata:
  name: expodinitcontainer
spec:
  containers:
  - name: maincontainer
    image: busybox
    command: ['sh', '-c']
    args: ['echo "maincontainer is running!"; sleep 3600']
  initContainers:
  - name: initdbcheck
    image: busybox
    command: ['sh', '-c']
    args: ['echo "checking db!"; sleep 30; echo "checking done!"']
  - name: initscript
    image: busybox
    command: ['sh', '-c']
    args: ['echo "init script exec!"; sleep 30; echo "init script exec done!"']
Copy the code

Here we have a primary container and two initializers. Each initializer performs 30 seconds. Next we create pods and check their status:

kubectl apply -f expodinitcontainers.yml
Copy the code

Initialize the first container within 30 seconds, so the Pod state is Init:0/2. Initialize the second container between 30 and 60 seconds, so the Pod state is Init:1/2. When all initialization containers are completed, the state will first change to PodInitializing. Then it becomes Running. Using the kubectl describe command to view Pod details, you can see that initdbCheck is executed, initscript is executed, and mainContainer is run last.

Formal container run phase

The PostStart event is triggered when the formal container is successfully created, and while the container is running, you can set up survival and readiness probes to continuously monitor the health of the container, and a PreStop event is triggered before the container terminates.

  • PostStart: This event is triggered just after the container is successfully created. If the callback fails, the container is terminated and whether to restart the container is determined according to the restart policy.
  • PreStop: Before the container starts and ends, this event is emitted and terminates the container regardless of the result of execution.

Callbacks can be done in two ways: Exec executes a script and HttpGet executes a specific request.

  • The Exec callback executes a specific command. If the command executed by Exec exits normally, the execution succeeds. Otherwise, the execution is considered abnormal. The configuration method is as follows:
PostStart /preStop: exec: command: XXXXX #Copy the code
  • The HttpGet callback executes a specific Http request and determines whether the request was successfully executed using the returned status code as follows:
PostStart /preStop: httpGet: host: XXXX # port: xx # path: XXXXXX # request path, such as github.com/xingxingzaixian, "/ xingxingzaixian" path scheme is: HTTP/HTTPS # request protocol, default to HTTPCopy the code

Let’s use the PostStart and PreStop events as examples

apiVersion: v1
kind: Pod
metadata:
  name: expodpostpre
spec:
  containers:
  - name: postprecontainer
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ['sh', '-c']
    args: ['echo "Hello Kubernetes!"; sleep 3600']
    lifecycle:
      postStart:
        httpGet:
          host: www.baidu.com
          path: /
          port: 80
          scheme: HTTP
      preStop:
        exec:
          command: ['sh', '-c', 'echo "preStop callback done!"; sleep 60']
Copy the code

In postStart we execute the HttpGet callback to visit the baidu home page, and preStop executes the command to output a piece of text, and then stays for 60 seconds. Let’s execute the create command and observe the status of the Pod.

kubectl apply -f expodpostpre.yml
Copy the code

In normal cases, it is the same as before. We do some test operations, such as writing the url of postStart to an inaccessible url, for example:

host: www.fackbook.com
Copy the code

View the log information

kubectl logs expodpostpre
Copy the code

Details can also be viewed through Kubectl Describe

Pod health check

While the container is running, we can continuously check the condition of the container through probes. Kubernetes provides us with two kinds of probes: live probes and ready probes.

  • LivenessProbe: Checks if the container is running. If Failure is returned, Kubelet terminates the container and the container executes according to the restart policy. If no survival probe is provided, the default status is Success.
  • ReadlinessProbe: Detects whether the container is ready to start the application Service. If Failure is returned, the Endpoint controller will remove the IP address of the Pod from the Endpoint of all services. The default ready state from container startup until the first probe is Failure. If no ready probe is provided, the default status is Success.

There are three methods in the container configuration to perform probe detection: exec, tcpSocket, and httpGet.

  • Exec: Executes the specified command inside the container. If the command exits with status code 0, the diagnosis is successful. The configuration is as follows:
LivenessProbe/readlinessProbe: exec: command: [XXXX] # command listCopy the code
  • TcpSocket: Performs TCP detection on the specified port of the container. If the port is open, the diagnosis is successful. The configuration is as follows:
LivenessProbe/readlinessProbe: tcpSocket: port: Number # designated port NumberCopy the code
  • HttpGet: checks HTTP services in the container. If the status code of the response ranges from 200 to 400, the diagnosis is successful. The configuration is as follows:
LivenessProbe/readlinessProbe: httpGet: port: Number # designated port path: String # specified pathCopy the code

Here are a few examples to illustrate the use of the probe.

Use of survival probes

apiVersion: v1
kind: Pod
metadata:
  name: expodlive
spec:
  containers:
  - name: livecontainer
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ['sh', '-c']
    args: ['mkdir /files_dir; echo "Hello Kubernetes!" > /files_dir/newfile; sleep 3600']
    livenessProbe:
      exec:
        command: ['cat', '/files_dir/newfile']
Copy the code

Files_dir, newfile, Hello Kubernetes! Content, and then use CAT in the survival probe to view the file contents. Execute the create Pod command:

kubectl apply -f expodlive.yml
Copy the code

So far Pod is working fine. Now we can do something destructive, go inside the Pod and delete the newFile file.

[root@k8s-master]# kubectl exec -it expodlive -- /bin/sh
/# rm -f /files_dir/newfile
/# exit
Copy the code

Since the probe periodically checks for the existence of the /files_dir/newfile file, and our Pod defaults to an exception reboot, you can view Pod details with the following command:

kubectl describe pods expodlive
Copy the code

You can see that the Event prints the status of the survival probe: The survival probe failed and is ready to restart.

Ready to use the probe

apiVersion: v1
kind: Pod
metadata:
  name: expodread
spec:
  containers:
  - name: readcontainer
    image: nginx
    imagePullPolicy: IfNotPresent
    ports:
    - name: portnginx
      containerPort: 80
    readinessProbe:
      httpGet:
        port: 80
        path: /
Copy the code

We create an Nginx container that exposes port 80 via the containerPort property, and then set up a ready probe to periodically send HttpGet requests to port 80 to check if the response range is 200-400. Once the apply command has been successfully created, we also need to do some testing.

[root@k8s-master]# kubectl apply -f expodread.yml
[root@k8s-master]# kubectl exec -it expodread -- /bin/sh
/# nginx -s stop
Copy the code

After executing nginx -s stop, the container will exit, because the nginx container is continuously serving, when the service stops, the container will be abnormal, and then kubernetes will pull up a new container. In this process, Using the kubectl get pod command, the status of the container changed to Completed, CrashLoopBackOff, and Running.

For starters, the difference between a live probe and a ready probe is often confusing. When should a live probe be used? When should a ready probe be used? My advice is as follows:

  • If the process in the container can crash on its own in the event of a problem or exception, as in the Nginx container, then the survival probe is not necessary and Kubelet will automatically perform the correct operation according to the Pod restart policy.
  • If you want to terminate and restart the container when a probe fails, you can specify a living probe and set the restart policy to Always or OnFailure.
  • If you want to send a network request to Pod only when the probe succeeds, you can specify a ready probe, such as HttpGet.
  • If the container needs to process large data, configuration files, or migrations during startup, the ready probe is used.

There are also five parameters you can set for each probe:

  • InitialDelaySeconds: Wait time for the first monitoring detection after the container is started, in seconds.
  • TimeoutSeconds: specifies the timeout period for waiting for a response after sending a monitoring request. If the timeout period expires, the probe is considered to have failed. The default value is 10s.
  • PeriodSeconds: Indicates the execution period of the probe, which is executed every 10 seconds by default.
  • SuccessThreshold: If a failure occurs, the diagnosis can be confirmed only after multiple successful probes. The default value is 1.
  • FailureThreshold: If a failure occurs, the Pod must be restarted (for live probes) or marked Unready (for ready probes) for multiple consecutive failures. The default value is 3.

The specific setting method is as follows:

livenessProbe/readinessProbe:
  exec/tcpSocket/httpGet:
    initialDelaySeconds: Number
    timeoutSeconds: Number
    periodSeconds: Number
    successThreshold: Number
    failureThreshold: Number
Copy the code

conclusion

In this article, we mainly talked about the addition, deletion, modification and check of Pod, the relationship between Pod and container, how to organize container, the life cycle and corresponding events of Pod, and the health check mechanism of Pod.