K8s Layer 4 network

Abstract into a four-layer networkCopy the code

Node Network

The underlying infrastructure supports network communication between hosts and nodesCopy the code

Pod network

IP address and communicate with each otherCopy the code

A Pod network on the same node

Most scenarios have one container under a POD and some scenarios have multiple containers under a POD that share the network stack within the PODCopy the code
  • eth0
It is the network card on the host, the device for traffic in and out, and the device for network addressing and communication between nodes in the K8S clusterCopy the code
  • docker0
A virtual bridge is a virtual switch that supports IP addressing and communication between nodesCopy the code
  • veth0
It is a virtual nic on POD1. Containers in POD can communicate with each other. Containers in devices can access each other through localhostCopy the code
  • pause
Pause is a special container that sets up the veTH0 network shared interface for pods and is visible through Docker PSCopy the code
  • pod
The IP of the pod is assigned by the Docker0 bridgeCopy the code

Pod networks on different nodes

Pod1 and Pod2 are in the same bridge dokcer0, so they can communicate with each other. The pod network is 172.17.0.0/24. The host network is 10.100.0.0/24, which means that the pod network and the node network are not in the same spaceCopy the code

How can PODS on different nodes communicate with each other?

Routing scheme

The pod in 172.17.1.0/24 network segment is forwarded to 10.100.0.3. The POD in 172.17.0.0/24 network segment is forwarded to 10.100.0.2. When host eth0 receives packets from the POD network, the packets are forwarded to the internal POD bridge This allows a POD network of different nodes to IP address and communicate. This scheme introduces the underlying network equipment but has less extra network overheadCopy the code
Overlay Network Scheme
If the public cloud network does not support routing scheme, it can use overlay network scheme. The so-called overlay network scheme is to build a virtual network on the basis of the existing network, which supports many technologies. For example, Flannel/Weavenet mostly adopts tunnel encapsulation technologyCopy the code

The packet of POD network is encapsulated into the packet of node network before leaving the node. After reaching the target node through the underlying network, the packet will be unsealed and then forwarded to the internal POD network. This scheme does not rely too much on the underlying network, but the packet unpacking has an impact on the performance CNI technology was introducedCopy the code

CNI is a Pod network integration standard that simplifies the integration of K8s and different Pod network implementation technologies. Kubelet uses CNI plug-ins to operate Pod networks without paying attention to the underlying implementationCopy the code

Service

A service often consists of clusters of multiple pods, which introduces problems such as service discovery and load balancingCopy the code

Service discovery and load balancing

How to find and locate pod IP?

The POD IP in the account-app Pod collection may change to republish or hang itself and the k8S rescheduling deployment will also change

Solving service discovery problems requires an understanding of technology evolution

DNS

When the DNS component is running, K8s will register the Pod information (IP and port) in the Pod cluster to DNS. Client applications can find the target Pod by querying DNS, so they can initiate calls. No intrusion disadvantages for client: Some clients cache DNS information for a long time by default. When the target Pod IP address changes, the cache is not refreshed in time, which will lead to the failure of access to Pod. 3 Some do not support load balancing, so K8s does not use this method of service discovery, but in fact, K8s does introduce the DNS component to access the service by domain nameCopy the code

The Service Registry + Client programs

Typical examples of such schemes are: The Eureka+Ribbon/Consul/Nacos K8s distributed storage etCD can be used to implement Service Registry. 1. K8s automatically registers pod information in pod cluster to Service Registry 2. The client application can find the target POD through servcie Registry query and then initiate and call this scheme to achieve no complexity. The client can also achieve flexible load balancing strategy, but it is invasive to the client application, so K8S does not directly adopt this schemeCopy the code

K8s Service network implementation principle

Kube-proxy, Kubelet, K8s Master, K8s implement service discovery and registration in the key "service registration stage" 1. Kubelet is responsible for starting the Pod instances when they are published. 2 3. When Servcie is published, K8s will assign ClusterIP to the Service and the ClusterIP will be recorded on the Master node. There is a mapping between ClusterIP and PodIP. Kube-proxy will listen to the Master and discover the list of ClusterIp and Pod instances of the Service and modify the Linux Iptables forwarding mechanism to load balance when receiving a ClusterIp and forward it to the corresponding Pod "Actual service invocation phase" When a consumer Pod needs to access a service, it invokes DNS to obtain the service name and makes the call through ClusterIP. The call is intercepted by the local Iptables mechanism and forwarded to the Pod instance through load balancing The DNS component also listens on the Master node and discovers the mapping between the service name and the ClusterIPCopy the code

K8s service discovery compared with eureka+ Ribbon solution

The ribbon is embedded in client applications in the form of libary, which is intrusive to client applications. Kube-proxy for K8S is an independent component. There is a Kube-Proxy on every Docker node This is similar to how the SideCar Ribbon in ServiceMesh forwards through iptables. In K8s forwards through iptables. Kube-proxy is only responsible for service discovery and requests to modify iptables rules are not penetrated through Kube-Proxy The new version does not wear Kube-proxy. The ribbon maps service names to service instance IP addresses. It has only one layer of mapping Kube-dns implements the mapping of service name to ClusterIP. Clusterip shields service discovery and load balancing problems through clusterIPCopy the code