Background:

Tencent cloud has been using personal warehouse to do mirror warehouse. Earlier, Tencent Cloud had TCR container image service:I took a peek and it felt a little expensive. A person has less than 50 images. I just want to use mirror security, vulnerability scanning. There are not as tough demands. More than 600 yuan a month still feels a little expensive! Build a Harbor!

Kubernetes1.21 build harbor

Note: We started with Kubernets 1.20.6 on TKE. Finally, it is on my Tencent Cloud self-built Kubernetes1.21 cluster. Refer to the earlier personal blogs: duiniwukenaihe. Making. IO / 2019/10/29 /…

1. Download the Harbor-Helm repository

Git clone way

git clone https://github.com/goharbor/harbor-helm
Copy the code

Helm necessary

Of course, helM3 is already installed here, and the HELM environment is a must

Wget https://get.helm.sh/helm-v3.6.3-linux-amd64.tar.gz tar ZXVF helm-v3.6.3-linux-amd64.tar.g CD Linux-amd64 cp helm /usr/local/bin/Copy the code

helm fetch

The helm command can also be used to add the warehouse directly. I directly used git Clone

[root@k8s-master-01 harbor-helm]# helm repo add harbor https://helm.goharbor.io "harbor" has been added to your repositories [root@k8s-master-01 harbor-helm]# cd /data/ [root@k8s-master-01 data]# helm search repo harbor NAME CHART VERSION APP VERSION DESCRIPTION Harbor/Harbor 1.7.2 2.3.2 An Open Source Trusted Cloud Native Registry TH... [root@k8s-master-01 data]# helm Fetch Harbor/Harbor -- Version 1.7.2Copy the code

Modifying a Configuration File

Modify the value.yaml configuration file: The cluster uses traefik proxy for external access. Expose Type is set to clusterIP. ExternalURL is set to storageclass. As follows:

type:

externalURL:

storageclass:

Note: Since the minimum unit of CBS is allowed to be 10g and the cut step size is 10g, 10G is used for all other stores except Registry. Of course the use of other storage can be personally reasonable Settings!

Helm install installation

helm install harbor -f values.yaml . --namespace kube-ops
kubectl get pods -n kube-ops -w  
Copy the code

Note: this figure is added later

helm upgrade

If values. Yaml updates the application later, you can use the following command to upgrade the application:

helm upgrade harbor -f values.yaml . --namespace kube-ops
Copy the code

If the Harbor application is deleted, then:

helm uninstall harbor -n kube-ops
Copy the code

Traefik agent Harbor external exposure applications:

ingressroute:

cat ingress-harbor.yml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-http
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/`)
      kind: Rule
      services:
        - name: harbor-portal
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-api
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/api`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-service
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/service`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-v2
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/v2`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-chartrepo
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/chartrepo`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-c
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/c`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
Copy the code

kubectl apply -f ingress-harbor.yaml
Copy the code

Default login password Harbor12345. You can also do this in value.yaml!

traefik ingress

Try it the Ingress way

helm upgrade harbor -f values.yaml . --namespace kube-ops
Copy the code

Note: this is bound to another domain name! Web access is fine too! — — — — — — — — — — — — — — — — — — — — — — — — — — — — — the delimiter — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Other issues that arise:

Docker push: Unkonwn blob?

The reason for this is probably that MY SLB did an HTTP jump over HTTPS automatically. An exception occurred when docker pushed. I read a lot of solutions online and I don’t know where to start. It basically says something like this, right?Finally lazy used a simple method: create a new SLB. Remove a server from the main SLB and place it on the new SLB. Direct TCP proxy. Do not do HTTP stronghop HTTPS.

 kubectl create secret tls all-xxxx-com --key=2_xxxx.com.key --cert=1_xxxx.com_bundle.crt -n kube-ops
Copy the code

ingress.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-http
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/`)
      kind: Rule
      services:
        - name: harbor-portal
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-api
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/api/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-service
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/service/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-v2
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/v2`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-chartrepo
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/chartrepo/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-c
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/c/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80

Copy the code

kubectl apply -f ingress.yaml
Copy the code

Also found a fun; I ended up building harbor in my own cluster. And then the storage is CBS! See also: Kuberentes cluster adds Tencent Cloud CBS as default storage. But my work nodes include AP-Shanghai2 and hosts in AP-Shanghai-3. Although I set the nodes in sector 3 to be unschedulable. But there are also fast storage built in zone 3 and the result is that the POD can’t run properly. After all, cloud hard disks cannot be mounted across regions. Create a storageclass ap-shanghai-2 and change the storageclass in harbor!

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: cbs-shanghai-2
provisioner: com.tencent.cloud.csi.cbs
parameters:
  diskZone: ap-shanghai-2
Copy the code

Eventually, of course, I switched to NFS…… Because I don’t want to waste resources by allocating 10GB hard drives to Redis Database. In terms of NFS storage, more attention should be paid to selfLink. For example, Kubernetes 1.19.12 has been promoted to 1.20.9 in selfLink.

Check out the censorship service

Well, to update it depends on….

Postscript:

I just wanted to check out Harbor’s censorship service. But the page still feels immature. Can you give me a vulnerability map when the scan is done? Proportion of high-risk vulnerabilities? What percentage of bugs per image? Vulnerability trends of different tags in the same image? ​

​