The original article

Configuration requirements

  • At least two 2-core 4G servers
  • In this document, the CPU must be x86 architecture
  • CentOS 7.8 or CentOS Stream 8

The software version after installation is

  • Kubernetes v1.21. X
    • The calico 3.17.1
    • Nginx – ingress 1.9.1
  • Containerd. IO 1.4.3

Operating System Compatibility

CentOS version Whether this document is compatible note
CentOS Stream 8 😄 The authenticated
CentOS 7.8 😄 The authenticated
CentOS 7.7 😞 No validation
CentOS 7.6 😞 No validation

{{< notice success “Container Runtime” >}}

  • Kubernetes v1.21 removes docker dependencies by default. If the host has Docker and Containerd installed, it will use Docker as a container engine first. If docker is not installed on the host and only containerd is installed, containerd will be used as a container to run the engine.
  • This article uses Containerd as the container running engine;

{{< /notice >}}

{{< notice success >}}

  • Kubeadm is the official Kubernetes supported installation method, “binary” is not. This document uses the kubeadm tool recommended by kubernetes. IO to install the Kubernetes cluster.

{{< /notice >}}

Check centos/hostname

#Execute on both master and worker nodes
cat /etc/redhat-release

#Here the hostname output will be the node name of the machine in the Kubernetes cluster
#You cannot use localhost as the node name
hostname

#Run the lscpu command to check CPU information
#Architecture: x86_64 This document does not support the ARM Architecture
#CPU(s): 2 The number of CPU cores cannot be less than two
lscpu
Copy the code

Change the hostname

#Change the hostname
hostnamectl set-hostname your-new-host-name
#Viewing the Modification Result
hostnamectl status
#Setting hostname resolutionEcho "127.0.0.1 $(hostname)" >> /etc/hostsCopy the code

Check the network

Run the command on all nodes

[root@demo-master-a-1 ~]$IP route show default via 172.21.0.1 dev eth0 169.254.0.0/16 dev eth0 scope link metric 1002 172.21.0.0/20 dev eth0 proto kernel scope link SRC 172.21.0.12 [root@demo-master-a-1 ~]$IP address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host LO valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:16:3E :12: A4 :1b BRD FF: FF :ff: FF: FF inet 172.17.216.80/20 BRD 172.17.223.255 scope Global Dynamic eth0 VALID_lft 305741654sec preferred_lft 305741654secCopy the code

{{< notice info “kubelet IP address “>}}

  • ip route showCommand, you can know the machine’s default network card, usually iseth0For example, default via 172.21.0.23 dev eth0
  • ip addressCommand to display the IP address of the default network card, which Kubernetes will use to communicate with other nodes in the cluster, such as172.17.216.80
  • IP addresses used by Kubernetes on all nodes must be interoperable (no NAT mapping, no security group or firewall isolation)

{{< /notice >}}

The installation

{{< notice WARNING “” >}} Execute the following code on all nodes as root to install the software:

  • containerd
  • nfs-utils
  • kubectl / kubeadm / kubelet

{{< /notice >}} {{< tabs Quick install Manual install >}} {{< TAB >}} Please replace the 1.21.0 at the end of the script with the version you want (it must be a smaller version of 1.21, Do not replace v1.21.x in the middle of the script

Select a Docker Hub image based on your network

  • The fourth is Tencent Cloud Docker Hub image
  • DaoCloud Docker Hub image file 6
  • Huawei Cloud Docker Hub image
  • The tenth is aliyun Docker Hub image
#Execute on both master and worker nodes
#The final parameter, 1.21.0, specifies the Kubenetes version, which supports all 1.21.x installs
#Tencent Cloud Docker Hub image
# export REGISTRY_MIRROR="https://mirror.ccs.tencentyun.com"
#DaoCloud mirror
# export REGISTRY_MIRROR="http://f1361db2.m.daocloud.io"
#Huawei Cloud Image
# export REGISTRY_MIRROR="https://05f073ad3c0010ea0f4bc00b7105ec20.mirror.swr.myhuaweicloud.com"
#Aliyun Docker Hub imageexport REGISTRY_MIRROR=https://registry.cn-hangzhou.aliyuncs.com curl -sSL https://kuboard.cn/install-script/v1.21.x/install_kubelet.sh | 1.21.0 sh - sCopy the code

{{< / TAB >}} {{< TAB >}} Manually execute the following code, the result is the same as a quick installation. Replace ${1} at line 79 of the script (highlighted) with the version number you want, for example 1.21.0

Select a Docker Hub image based on your network

  • The fourth is Tencent Cloud Docker Hub image
  • DaoCloud Docker Hub image file 6
  • 8. Alibaba Docker Hub image
#Execute on both master and worker nodes
#The final parameter, 1.21.0, specifies the Kubenetes version, which supports all 1.21.x installs
#Tencent Cloud Docker Hub image
# export REGISTRY_MIRROR="https://mirror.ccs.tencentyun.com"
#DaoCloud mirror
# export REGISTRY_MIRROR="http://f1361db2.m.daocloud.io"
#Aliyun Docker Hub image
export REGISTRY_MIRROR=https://registry.cn-hangzhou.aliyuncs.com
Copy the code
#! /bin/bash

#Execute on both master and worker nodes

#Install containerd
#Reference documents are as follows
# https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1  net.bridge.bridge-nf-call-ip6tables = 1 EOF
# Apply sysctl params without reboot
sysctl --system

#Uninstall the previous version
yum remove -y containerd.io

#Set the yum repository
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

#Install containerdYum install - y containerd. IO - 1.4.3 mkdir -p/etc/containerd containerd config default > / etc/containerd/config toml sed -i "s#k8s.gcr.io#registry.aliyuncs.com/k8sxio#g" /etc/containerd/config.toml sed -i '/containerd.runtimes.runc.options/a\ \ \ \ \ \ \ \ \ \ \ \ SystemdCgroup = true' /etc/containerd/config.toml sed -i "s#https://registry-1.docker.io#${REGISTRY_MIRROR}#g" /etc/containerd/config.toml systemctl daemon-reload systemctl enable containerd systemctl restart containerd

#NFS - utils
#Nfs-utils must be installed before mounting NFS network storage
yum install -y nfs-utils
yum install -y wget

#Disabling the Firewall
systemctl stop firewalld
systemctl disable firewalld

#Close the SeLinux
setenforce 0
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config

#Close the swap
swapoff -a
yes | cp /etc/fstab /etc/fstab_bak
cat /etc/fstab_bak |grep -v swap > /etc/fstab

#Configure the K8S yum source
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
       http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

#Uninstall the previous version
yum remove -y kubelet kubeadm kubectl

#Install kubelet, kubeadm, kubectl
# å°† The ${1}Replace it with the Kubernetes version number, for example 1.20.1
yum install -y kubelet-${1} kubeadm-${1} kubectl-${1}

crictl config runtime-endpoint /run/containerd/containerd.sock

#Restart Docker and start Kubelet
systemctl daemon-reload
systemctl enable kubelet && systemctl start kubelet

containerd --version
kubelet --version
Copy the code

{{< notice warning “notice” >}} If the systemctl status kubelet command is executed, the kubelet startup failure message is displayed, please ignore this error. {{< /notice >}} {{< / TAB >}} {{< /tabs >}} {{< /tabs >}}

Initialize the master node

{{< notice warning “About environment variables used during initialization” >}}

  • APISERVER_NAME cannot be master hostname
  • APISERVER_NAME must contain all lowercase letters, digits, and decimal points, and cannot contain minus signs
  • The network segment used by POD_SUBNET cannot overlap the network segment of the master /worker node. The value of this field is a CIDR value. If you are not familiar with CIDR, run export POD_SUBNET=10.100.0.1/16 and do not change the value

{{< /notice >}}

{{< tabs Quick initialization Manual initialization >}} {{< TAB >}} Do not replace v1.21.x in the middle of the script with 1.21.0 at the end of the script (it must be a smaller version of 1.21, not 1.19.1)

#Execute only on the master node
#Replace X.X.X.X with the actual IP address of the master node (use the Intranet IP address).
# exportThe command is valid only in the current shell session. If you want to continue the installation process after opening a new shell window, re-execute the following commandexportThe command
export MASTER_IP=x.x.x.x
#Replace apiserver.demo with the dnsName you want
export APISERVER_NAME=apiserver.demo
#The network segment where the Kubernetes container group resides. This network segment is created by Kubernetes after installation and does not exist in your physical network beforehandExport POD_SUBNET=10.100.0.1/16 echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts curl-ssl https://kuboard.cn/install-script/v1.21.x/init_master.sh | 1.21.0 sh - sCopy the code

{{< / TAB >}} {{< TAB >}} Manually execute the following code, the result is the same as a quick installation. Replace ${1} at line 79 of the script (highlighted) with the version number you want, for example 1.21.0

#Execute only on the master node
#Replace X.X.X.X with the internal IP address of the master node
# exportThe command is valid only in the current shell session. If you want to continue the installation process after opening a new shell window, re-execute the following commandexportThe command
export MASTER_IP=x.x.x.x
#Replace apiserver.demo with the dnsName you want
export APISERVER_NAME=apiserver.demo
#The network segment where the Kubernetes container group resides. This network segment is created by Kubernetes after installation and does not exist in your physical network beforehandExport POD_SUBNET=10.100.0.1/16 echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hostsCopy the code
#! /bin/bash

#Execute only on the master node

#Abort script execution if an error occursset -e if [ ${#POD_SUBNET} -eq 0 ] || [ ${#APISERVER_NAME} -eq 0 ]; Then echo -e "\033[31;1m Make sure you have set the environment variables POD_SUBNET and APISERVER_NAME \033[0m" echo current POD_SUBNET=$POD_SUBNET echo Current APISERVER_NAME=$APISERVER_NAME exit 1 fi

#See the full configuration options at https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2rm -f ./kubeadm-config.yaml cat <<EOF > ./kubeadm-config.yaml --- apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v${1} imageRepository: registry.aliyuncs.com/k8sxio controlPlaneEndpoint: "${APISERVER_NAME}:6443" Networking: serviceSubnet: "10.96.0.0/16" podSubnet: "${POD_SUBNET}" dnsDomain: "Cluster.local" DNS: type: CoreDNS imageRepository: swr.cn-east-2.myhuaweicloud.com${2} imageTag: 1.8.0 -- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd EOF
# kubeadm init
#Depending on the speed of your server, you will need to wait 3 to 10 minutesEcho "echo" grab image, please wait... Kubeadm config images pull --config=kubeadm-config.yaml echo "" echo "initializing Master node kubeadm init --config=kubeadm-config.yaml --upload-certs
#Configuration kubectl
rm -rf /root/.kube/
mkdir /root/.kube/
cp -i /etc/kubernetes/admin.conf /root/.kube/config

#Install the Calico Network plug-in
#Reference documentation, https://docs.projectcalico.org/v3.13/getting-started/kubernetes/self-managed-onprem/onpremisesRm -f calico-3.17.1. Yaml kubectl create -f https://kuboard.cn/install-script/v1.21.x/calico-operator.yaml wget https://kuboard.cn/install-script/v1.21.x/calico-custom-resources.yaml sed -i "s # 192.168.0.0/16 # # ${POD_SUBNET}" calico-custom-resources.yaml kubectl create -f calico-custom-resources.yamlCopy the code

{{< /tab >}} {{< /tabs >}}

Check the master initialization result

#Execute only on the master node

#Run the following command and wait 3 to 10 minutes until all container groups are in the Running state
watch kubectl get pod -n kube-system -o wide

#View the initialization result of the master node
kubectl get nodes -o wide
Copy the code

Initialize the worker node

Get the join command parameters

Execute on the master node

#Execute only on the master node
kubeadm token create --print-join-command
#Returns the following
kubeadm join apiserver.demo:6443 --token 9ukzcs.qp4ozxbn1knc13sx --discovery-token-ca-cert-hash sha256:0cb945ea0c6329b4df58cf358e2be697fd31a85f55c23d47356f06f69a27283d 
Copy the code

{{< notice Success “Validity period” >}} This token is valid for 2 hours, within 2 hours, you can use this token to initialize any number of worker nodes. {{< /notice >}}

Initialize the worker

Execute for all worker nodes

#Only executed on worker nodes
#Replace X.X.X.X with the internal IP address of the master node
export MASTER_IP=x.x.x.x
#Replace apiserver.demo with the APISERVER_NAME used when initializing the master node
export APISERVER_NAME=apiserver.demo
echo "${MASTER_IP}    ${APISERVER_NAME}" >> /etc/hosts

#Replace it with the kubeadm token create command output on the master node
kubeadm join apiserver.demo:6443 --token 9ukzcs.qp4ozxbn1knc13sx --discovery-token-ca-cert-hash sha256:0cb945ea0c6329b4df58cf358e2be697fd31a85f55c23d47356f06f69a27283d 
Copy the code

Check the initialization result

Execute on the master node

#Execute only on the master node
kubectl get nodes -o wide
#The following output is displayed[root@master ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane,master 54m v1.21.0 node1 35 m v1.21.0 Ready < none >Copy the code

Install the Ingress Controller

{{< tabs quick install uninstall IngressController >}} {{< TAB >}} In the partial case on the master node, you need to execute the following command twice to succeed.

#Execute only on the master nodeKubectl apply -f https://kuboard.cn/install-script/v1.21.x/nginx-ingress.yamlCopy the code

{{< / TAB >}} {{< TAB >}} execute on the master node

Uninstall only if you want to select another Ingress Controller

#Execute only on the master nodeKubectl delete -f https://kuboard.cn/install-script/v1.21.x/nginx-ingress.yamlCopy the code

{{< /tab >}} {{< /tabs >}}

Configuring Domain Name Resolution

Resolve domain name cnsre.cn to DEMO-worker-a-2’s IP address Z.Z.Z (or demo-worker-a-1’s IP address Y.Y.Y)

Verify the configuration

Visit cnsre.cn in your browser and you will get the 404 NotFound error page