K8s offline installation and deployment tutorial

The file name The version number The Linux kernel
The docker version 20.10.9 x86
K8s version v1.22.4 x86
kuboard v3 x86

6. Set the IPVS mode

K8s whole cluster for access through; The default is iptables, which degrades performance (Kube-proxy synchronizes the contents of iptables between clusters)

Each pod needs to be assigned an IP address. Each node, Kube-proxy, synchronizes the IP address of the pod on other nodes to ensure the same iptables. In order for each node to access iptables, it constantly synchronizes iptables, which affects performance.

Check the default kube-proxy schema
kubectl get pod -A|grep kube-proxy
kubectl logs -n kube-system kube-proxy-xxxx

Ipvs = kube-proxy; ipvs = kube-proxy; The default is iptables, but it is slow when the cluster is largeIpvs excludeCIDRs: NULL minSyncPeriod: 0s Scheduler: excludeCIDRs""
      strictARP: falseSyncPeriod: 30s kind: KubeProxyConfiguration metricsBindAddress: 127.0.0.1:10249 Mode:"ipvs"
    
The kube-proxy configuration was changed, and in order for it to take effect again, it needed to kill the previous kube-proxy
kubectl get pod -A|grep kube-proxy
kubectl delete pod kube-proxy-xxxx -n kube-system
You can restart kube-proxy to make the changes take effect
Copy the code

7. Install kuboard

Kuboard installation: version: v3

Online Installation:

kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3.yaml
Copy the code

Offline installation:

Download the kuboard-v3.yaml file first

Kuboard-v3. yaml requires an image. If you cannot access the Internet, download the image from a server with an Internet connection.

cat kuboard-v3.yaml | grep image: | awk '{print $2}'
Copy the code

Eipwork/etcd – host: eipwork 3.4.16-1 / kuboard: v3

The following two images, which cannot be obtained by using this command, are described in the official website. Pull is required

Eipwork/kuboard – agent: v3 questdb/questdb: 6.0.4

# pull all mirrors
cat kuboard-v3.yaml \
    | grep image: \
    | awk '{print "docker pull " $2}' \
    | sh

# pull the other two imagesEipwork /kuboard-agent:v3 docker pull questDB/questDB :6.0.4Export image as compressed package in current directoryEipwork /kuboard:v3 docker save-o etcd-host-3.4.16-1. Tar eipwork/etcd-host:3.4.16-1 docker Save -o kuboard-agent-v3.tar eipwork/kuboard-agent:v3 docker save -o questdb-6.0.4.tar questdb/questdb:6.0.4Load into the Docker environmentDocker load -i kuboard-v3.tar docker load -i etcd-host-3.4.16-1. Tar docker load -i kuboard-agent-v3.tar docker load -i kuboard-agent-v3.tar docker load -i etcd-host-3.4.16-1 Questdb - 6.0.4. Tar# installation kuboard
kubectl apply -f kuboard-v3.yaml

# remove kuboard
kubectl delete -f kuboard-v3.yaml
Copy the code

Note that kuboard-v3.yaml, imagePullPolicy: you need to change Always to IfNotPresent

    
          image: 'eipwork/etcd - host: 3.4.16-1'
          # IfNotPresent = IfNotPresent
          imagePullPolicy: IfNotPresent
         


---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations: {}
  labels:
    k8s.kuboard.cn/name: kuboard-v3
  name: kuboard-v3
  namespace: kuboard
    
          image: 'eipwork/kuboard:v3'
          # IfNotPresent = IfNotPresent
          imagePullPolicy: IfNotPresent
Copy the code

The startup effect is as follows:

# start kuboard – v3

kubectl apply -f kuboard-v3.yaml

Check whether kuborad is successfully started:

kubectl get pods -n kuboard

If there are only three, the kuboard-agent-xxx container is not started. Please continue down the line:

8. Visit Kuboard

  • Open the link in your browserhttp://your-node-ip-address:30080
  • Enter your initial user name and password and log in
    • User name:admin
    • Password:Kuboard123

The home page, by default, will show the import (when the agent is not started), click to enter default

Export: kuboard-agent.yaml file

Note: this kuboard-agent.yaml file, image pull mode, need to change imagePullPolicy: need to change Always to IfNotPresent

kubectl apply -f ./kuboard-agent.yaml
Copy the code

End result:

9. The metrics – server installation

In order to be able to see the server resources in kuboard, you can monitor

Metrics -server.yaml, version 0.5.0

Step by step, preview the resulting yaml file and save it as the metrics-server.yaml file

Offline installation:

The metrics – server. Yaml files

Yaml requires an image. If you do not have access to the Internet, download the image from a server that has access to the Internet.

swr.cn-east-2.myhuaweicloud.com/kuboard-dependency/metrics-server:v0.5.0

# pull all mirrorsDocker pull swr.cn-east-2.myhuaweicloud.com/kuboard-dependency/metrics-server:v0.5.0Export image as compressed package in current directoryDocker save -o metrics - server - v0.5.0. Tar swr.cn-east-2.myhuaweicloud.com/kuboard-dependency/metrics-server:v0.5.0Load into the Docker environmentDocker load -i metrics - server - v0.5.0. Tar# installation kuboard
kubectl apply -f metrics-server.yaml

# remove kuboard
kubectl delete -f metrics-server.yaml
Copy the code

The metrics-server.yaml file is as follows:

 ---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
    - name: https
      port: 443
      protocol: TCP
      targetPort: 443
  selector:
    k8s-app: metrics-server

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: 'true'
    rbac.authorization.k8s.io/aggregate-to-edit: 'true'
    rbac.authorization.k8s.io/aggregate-to-view: 'true'
  name: 'system:aggregated-metrics-reader'
  namespace: kube-system
rules:
  - apiGroups:
      - metrics.k8s.io
    resources:
      - pods
      - nodes
    verbs:
      - get
      - list
      - watch

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: 'system:metrics-server'
  namespace: kube-system
rules:
  - apiGroups:
      - ' '
    resources:
      - pods
      - nodes
      - nodes/stats
      - namespaces
      - configmaps
    verbs:
      - get
      - list
      - watch

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: 'metrics-server:system:auth-delegator'
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: 'system:auth-delegator'
subjects:
  - kind: ServiceAccount
    name: metrics-server
    namespace: kube-system

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: 'system:metrics-server'
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: 'system:metrics-server'
subjects:
  - kind: ServiceAccount
    name: metrics-server
    namespace: kube-system

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
  - kind: ServiceAccount
    name: metrics-server
    namespace: kube-system

---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system

---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
  namespace: kube-system
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 1
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - preference:
                matchExpressions:
                  - key: node-role.kubernetes.io/master
                    operator: Exists
              weight: 100
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  k8s-app: metrics-server
              namespaces:
                - kube-system
              topologyKey: kubernetes.io/hostname
      containers:
        - args:
            - '--cert-dir=/tmp'
            - '--secure-port=443'
            - '--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname'
            - '--kubelet-use-node-status-port'
            - '--kubelet-insecure-tls=true'
            - '--authorization-always-allow-paths=/livez,/readyz'
            - '--metric-resolution=15s'
          image: > - swr.cn-east-2.myhuaweicloud.com/kuboard-dependency/metrics-server:v0.5.0          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /livez
              port: https
              scheme: HTTPS
            periodSeconds: 10
          name: metrics-server
          ports:
            - containerPort: 443
              name: https
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /readyz
              port: https
              scheme: HTTPS
            initialDelaySeconds: 20
            periodSeconds: 10
          resources:
            requests:
              cpu: 100m
              memory: 200Mi
          securityContext:
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 1000
          volumeMounts:
            - mountPath: /tmp
              name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      tolerations:
        - effect: ' '
          key: node-role.kubernetes.io/master
          operator: Exists
      volumes:
        - emptyDir: {}
          name: tmp-dir

---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: metrics-server
  namespace: kube-system
spec:
  minAvailable: 1
  selector:
    matchLabels:
      k8s-app: metrics-server
Copy the code

The startup effect is as follows:

# start metrics – server

kubectl apply -f metrics-server.yaml

You can view information about server resources, such as memory and CPU