In order to facilitate everyone to learn Kubernetes system, I organized a Kubernetes learning series of articles, covering the basic knowledge of Kubernetes, installation steps and the related content of the whole Kubernetes system, I believe we read this series, To have a deeper understanding of Kubernetes.
Example Set the number of resource requests
When creating a Pod, you can specify resource consumption limits for each container. The Pod resource request limit is the sum of all container requests in the Pod.
apiVersion: v1
kind: Pod
metadata:
name: requests-pod
spec:
containers:
- image: busybox
command: ["dd"."if=/dev/zero"."of=/dev/null"]
name: main
resources:
requests:
cpu: 200m
memory: 10Mi
Copy the codeIf CPU request resources are not specified, it indicates that the container does not care about the CPU resources allocated to the container. Specifying resource requests indicates the minimum resource requirements of Pod. Therefore, if the remaining Node resources cannot meet Pod’s requirements, it will not be scheduled to the corresponding Node. Scheduler does not pay attention to the specific resource usage at the time of scheduling, but schedules according to the resource requests of existing PODS. This can be problematic, especially if the Pod is allowed to use more resources than requested. The picture below makes sense.
The scheduling judgment first excludes the nodes that do not meet the request, and then sorts the nodes that do. Node ordering is divided into two policies based on the number of resource requests, LeastRequestPolicy and MostRequestPolicy. Literally, we can see that one is assigned to nodes with low resource requests and one is assigned to nodes with high resource requests. In a production environment, it is recommended to use LeastRequestPolicy to distribute the load evenly across machines. You are advised to use MostRequestPolicy in public cloud environments to improve resource utilization and reduce costs.
Without setting resource usage limits, A Pod may use more resources than requested. For CPU resources, if two PODS request the remaining resources at the same time, when allocating the remaining resources, the scheduler will allocate resources among the different pods according to the ratio of the number of requests. For example, Pod A requests A CPU of 100 MB and Pod B requests A CPU of 20 MB. If the CPU usage of the two pods exceeds the request, the CPU usage ratio is 5:1.
usekubectl describe nodesThe Node resource usage command is used to view Node resource usage.
If Kubernetes cannot find a Node that meets the resource request, the Pod creation will remain Pending.
Set the upper limit of resource usage
When a Pod is created, you can set the upper limit of resources that can be used by each container, such as CPU and memory. If no upper limit is set, then theoretically all Node resources can be used. If you want to prevent containers on Node from interacting with each other, it is best to specify an upper limit for pods.
The CPU is a resource that can be squeezed, filled up, and pods do not interact with each other. Memory is different, and the memory allocated between pods cannot be used by each other. The number of requests resources must be equal to or smaller than the Node capacity. The total number of LIMITS resources can exceed the Node capacity. Some containers may be killed when the node’s resources are used up. In particular, Kubernetes will perform OOMKilled after using memory limit. If the Pod restart policy is Always, you may not even notice that the Pod is restarted, but the delay for each restart increases as the number of times it occurs.
apiVersion: v1
kind: Pod
metadata:
name: limit-pod
spec:
containers:
- image: busybox
command: ["dd"."if=/dev/zero"."of=/dev/null"]
name: main
resources:
requests:
cpu: 200m
memory: 10Mi
limits:
cpu: 1
memory: 20Mi
Copy the codeRun the top command in the container and you will see that the total amount of CPU and memory in the container is the total amount of Node. This can cause some applications to have different capacity detections than Limits Limits, resulting in usage exceeding the request.
For CPU and memory, you can use Metadata to obtain the size of the limit through the API. Can also be in the/sys/fs/cgroup / / CPU. The CPU cfs_quota_us, / sys/fs/cgroup / / CPU. The CPU cfs_period_us to view.
QoS: indicates a Pod Kill policy
When the resource usage of Pod exceeds Node capacity, Kubernetes will select some of them and kill them in order to ensure Node running. Therefore, how to determine which Pod to kill? Here we need to introduce a concept of QoS.
QoS is the Quality of Service. There are three Quality of Service policies. Kubernetes kills the Pod of the three policies in turn. If the QoS of the two devices is the same, select Kill with high resource utilization.
- BestEffort applies to pods with no resource limits, can use as many resources as possible, and may be killed first
- Burstable limits The Pod type that exceeded requests
- A Guaranteed warranty applies to pods with consistent requests and limits (limits are the same as Requests by default), which cannot use excess resources but are Guaranteed to survive
For single-container PODS, follow these guidelines
For pods with multiple containers, if the policies of the two containers are inconsistent, the Burstable policy is used, and if the policies of the container are consistent, the Burstable policy is used.
Set the default request for Pod/Container and limit LimitRange
By creating the LimitRange object, you can set a punishing requests and limits limit for all created pods in a namespace.
apiVersion: v1
kind: LimitRange
metadata:
name: limitrange-demo
spec:
limits:
- type: Pod
min:
cpu: 50m
meomery: 5Mi
max:
cpu: 1
meomery: 1Gi
- type: Container
defaultRequest:
cpu: 100m
memory: 10Mi
default:
cpu: 200m
memory: 100Mi
min:
cpu: 50m
memory: 5Mi
max:
cpu: 1
memory: 1Gi
maxLimitRequestRatio:
cpu: 4
memory: 10
- type: PersistentVolumeClaim
min:
storage: 1Gi
max:
storage: 10Gi
Copy the codeIf you create a Pod that does not meet LimitRange requirements, the following error occurs.
Set resource quotas for the cluster
In addition to setting the default upper limit for each Pod, you can also set the upper limit of available resources for a cluster using ResourceQuota. ResourceQuota sets the maximum number of computing resources available to a cluster and the number of objects that users can create.
apiVersion: v1
kind: ResourceQuota
metadata:
name: cpu-and-mem
spec:
hard:
requests.cpu: 400m
requests.memory: 200Mi
limits.cpu: 600m
limits.memory: 500Mi
Copy the codeView current resource limits
You can also limit the number of stores and various objects, as shown in YAML below.
apiVersion: v1
kind: ResourceQuota
metadata:
name: storage-object
spec:
hard:
pods: 10
replicationcontrollers: 5
secrets: 10
configmaps: 10
persistentvolumeclaims: 4
services: 5
services.loadbalancers: 1
services.nodeports: 2
ssd.storageclass.storage.k8s.io/persistentvolumeclaims: 2
Copy the codeQuota values are set in the namespace by default, and can be set to different limits based on QoS.
apiVersion: v1
kind: ResourceQuota
metadata:
name: quota-qos
spec:
scopes:
- BestEffort
- NotTerminating
hard:
pods: 4
Copy the codeThere are four strategies: BestEffort, NotBestEffort, Terminating, NotTerminating. The first two select the Pod according to QoS, and the last two according to whether the Pod is setactiveDeadlineSecondsProperty to select. Take a look at the picture below.
monitoring
Kubernetes includes cAdvisor to monitor container and node health. If you want to take a holistic view of resource usage, you need to install the Heapster component. But these two
kubectl top node
kubectl top pod
kubectl top pod --container
Copy the codeYou can use these two commands to view the usage of Pod and Node resources in a short period of time, as well as the usage of each container resource. If you want to save performance data, install heapster\ Influxdb \grafana. The details are not explained in this article.
References:
- cAdvisor
- InfluxDB
- Grafana
- influxdb yaml