Wechat official account: Operation and maintenance development story, author: Hua Zai

The introduction

Recently, I have heard many friends say that Cilium is very strong and is bound to become mainstream. Because it uses EBPF, it has good performance and supports network policies. So I decided to spend some time studying. In the process of learning through the official website documents, I found that using Cilium as CNI, there is no need to install Kube-Proxy. This reminds me of a question I was asked in an interview before. The interviewer asked me whether KUbe-Proxy could be installed without any other alternatives. Well, now we have the answer. By the way, it is really a bit difficult to study from official documents (after all, it is all In English); However, it is still recommended that you read the official documents to learn, do not translate into Chinese oh. So now I’m going to do it.

The environment that

The serial number The matters instructions
1 kubernetes version v1.21.3
2 cilium version v1.10.3
3 Kubernetes installation mode kubeadm
4 Cilium networking mode vxlan
5 os Ubuntu 18.04
6 Kubernetes Cluster size 1 master, 2 node

The body of the

  1. Initialize the cluster on master and ignore the installation of kube-proxy by adding — Skip-Phases =addon/kube-proxy
Kubeadm init - apiserver - advertise - address = 10.211.55.50 - image - repository registry.aliyuncs.com/google_containers --kubernetes-version v1.21.3 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=all --skip-phases=addon/kube-proxyCopy the code
  1. Execute kubeadm join on both nodes to join the cluster
Kubeadm join 10.211.55.50:6443 --token ouez6j.02MS269v8i4psl7p --discovery-token-ca-cert-hash sha256:5fdafe0fe1adb3b60cd7bc33f033f028279a94a3944816424cc7f5bb498f6868Copy the code
  1. Use HELM (V3) to install cilium. Add the Cilium library first
helm repo add cilium https://helm.cilium.io/
Copy the code
  1. Install cilium using the following command, adding the kubeProxyReplacement=strict parameter
Helm install cilium cilium/cilium --version 1.10.3 --namespace kube-system --set kubeProxyReplacement=strict --set K8sServiceHost = 10.211.55.50 - set k8sServicePort = 6443Copy the code
  1. Check cilium installation
root@cilium1:/# kubectl -n kube-system get Pods -l k8s-app= Cilium NAME READY STATUS RESTARTS AGE cilium-8gwg2 1/1 Running 0 8m4s cilium-t9ffc 1/1 Running 0 8m39s cilium-x42r6 1/1 Running 0 8m16s # See cilum operator root @ cilium1: ~ # kubectl get Po - A - o wide | grep cilium - operator kube - system Cilium-operator-5df88875-867hd 1/1 Running 5 41h 172.16.88.47 cilium3 <none> <none> kube-system Cilium-operator-5df88875-9kx8c 1/1 Running 5 41h 172.16.88.253 cilium2 <none> <none>Copy the code
  1. Check whether there is a Kube-Proxy component. You can see that this component does not exist
root@cilium1:/# kubectl get po -n kube-system
NAME                              READY   STATUS    RESTARTS   AGE
cilium-8gwg2                      1/1     Running   0          10m
cilium-operator-5df88875-867hd    1/1     Running   5          27h
cilium-operator-5df88875-9kx8c    1/1     Running   5          27h
cilium-t9ffc                      1/1     Running   0          11m
cilium-x42r6                      1/1     Running   0          10m
coredns-59d64cd4d4-hbwg4          1/1     Running   1          27h
coredns-59d64cd4d4-l2pmt          1/1     Running   1          27h
etcd-cilium1                      1/1     Running   2          27h
kube-apiserver-cilium1            1/1     Running   2          27h
kube-controller-manager-cilium1   1/1     Running   2          27h
kube-scheduler-cilium1            1/1     Running   2          27h
Copy the code
  1. Check cilium status to ensure proper installation
root@cilium1:/# kubectl exec -n kube-system cilium-t9ffc -- cilium status Defaulted container "cilium-agent" out of: cilium-agent, mount-cgroup (init), clean-cilium-state (init) KVStore: Ok Disabled Kubernetes: Ok 1.21 (v1.21.3) [Linux/AMD64] Kubernetes APIs: ["cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "core/v1::Namespace", "core/v1::Node", "core/v1::Pods", "core/v1::Service", "discovery/v1::EndpointSlice", "networking.k8s.io/v1::NetworkPolicy"] KubeProxyReplacement: Strict [eth0 10.211.55.50 (Direct Routing)] Cilium: Ok 1.10.3 (v1.10.3-4145278) NodeMonitor: Listening for events on 8 CPUs with 64x4096 of shared memory Cilium health daemon: Ok IPAM: IPv4: 2/254 Allocated from 10.0.0.0/24, BandwidthManager: Disabled Host Routing: Legacy Masquerading: BPF [eth0] 10.0.0.0/24 [IPv4: Enabled, IPv6: Disabled] Controller Status: 20/20 Healthy Proxy Status: Hubble: OK Current/Max Flows: 817/4095, Flows/s: Metrics: Disabled Encryption: Disabled Cluster health: 3/3 reachable (2021-08-07T15:29:05z)Copy the code
  1. Deploy Nginx to test network connectivity
# nginx deployment yaml file cat deployments-nginx. yaml apiVersion: apps/v1 kind: deployment metadata: name: nginx spec: selector: matchLabels: run: nginx replicas: 4 template: metadata: labels: run: nginx spec: containers: - name: nginx image: nginx ports: - containerPort: 80 # create nginx deployment kubectl create -f deployments-nginx. yaml root@cilium1:/# kubectl get Po -o wide NAME READY STATUS RESTARTS AGE IP NODE NODE READINESS GATES NginX-649c4b9857-8f2V5 1/1 Running 1 26h 10.0.2.212 Cilium2 <none> <none> nginx-649c4b9857-mHSXS 1/1 Running 1 26h 10.0.1.23cilium3 <none> <none> nginx-649c4b9857-qw2jj 1/1 Running 1 26h 10.0.1.126cilium2 <none> <none> nginx-649c4b9857-vj9w2 1/1 Running 1 26h 10.0.1.126cilium3 <none> <none>Copy the code
  1. Create a nodePort Service to verify that the service is accessible
# kubectl expose deployment nginx --type=NodePort --port=80 # kubectl get SVC nginx NAME TYPE cluster-ip external-ip PORT(S) AGE nginx NodePort 10.97.209.103 < None > 80:31126/TCP 26hCopy the code
  1. Verify that nodeport and Cluster are accessible
# curl 127.0.0.1:31126 <! DOCTYPE html> <html> <head> <title>Welcome to nginx! </title> # curl 10.97.209.103 <! DOCTYPE html> <html> <head> <title>Welcome to nginx! < / title > root @ # check iptables found empty cilium1: / # iptables - save | grep KUBE - SVC root @ cilium1: / # # check ciliun service root@cilium1:/# kubectl exec -n kube-system cilium-t9ffc -- cilium service list Defaulted container "cilium-agent" out of: cilium-agent, mount-cgroup (init), Clean-cilium-state (init) ID Frontend Service Type Backend 1 10.96.0.1:443 ClusterIP 1 => 172.16.88.57:6443 2 10.96.0.10:9153 ClusterIP 1 => 10.0.2.229:9153 2 => 10.0.2.80:9153 3 10.96.0.10:53 ClusterIP 1 => 10.0.2.229:53 2 => 10.0.2.80:53 4 10.97.209.103:80 ClusterIP 1 => 10.0.2.69:80 2 => 10.0.1.23:80 3 => 10.0.1.126:80 4 => 10.0.2.212:80 5 172.16.88.57:31126 NodePort 1 => 10.0.2.69:80 2 => 10.0.1.23:80 3 => 10.0.1.126:80 4 => 10.0.2.212:80 6 0.0.0.0:31126 NodePort 1 => 10.0.2.69:80 2 => 10.0.1.23:80 3 => 10.0.1.126:80 4 => 10.0.2.212:80Copy the code
  1. From the above installation and test results, although we did not install the KUbe-Proxy component of K8S, the cluster is still normal. The kube-proxy component is indeed replaceable.

conclusion

Although the above has completed the construction and testing of Kubernetes without Kube-Proxy, there are still many things left unexplained. For example, the system requirements for using Cilium, what cilium is, several networking modes, and network policies. But don’t worry. Look forward to my next posts.

reference

  • Docs. Cilium. IO/en/v1.10 / ge…
  • Kubernetes. IO/docs/concep…
  • Helm. Sh/docs/type /…