Haobor2.2.1 configuration (trivy scanner, mirror signature)

Docker – compose the download

https://github.com/docker/compose/releases
Copy the code

The installation

cp docker-compose /usr/local/bin
chmod +x /usr/local/bin/docker-compose
Copy the code

Harbor download

https://github.com/goharbor/harbor/releases
Copy the code

Unpack the

tar xf xxx.tgx
Copy the code

The configuration of harbor

Create a root :mkdir /data CD harbor/ mkdir certs CD certs/Copy the code

Generate a certificate and private key

openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout registry.key -out registry.crt
cd ..
cp -r certs/ /
Copy the code

Editing a Configuration File

mv harbor.yml.temp harbor.yml
hostname:example.com
 certificate: ./certs/registry.crt
 private_key: ./certs/registry.key
 harbor_admin_password:Abcd12345
Copy the code

Configuration testing

./prepare
Copy the code

The installation

The helm Charts module was added to the “with-Chartmuseum” module. The “with-trivy” module was also used to simulate the tary of the company. The Helm Charts module was added — with-ChartMuseum

./install.sh --with-notary --with-trivy --with-chartmuseum
Copy the code

To configure the hostname

Vim /etc/hosts Finally adds: 127.0.0.1 example.comCopy the code

Browser access

https://example.com or: IP address :80 Username: admin Password: Abcd12345Copy the code

Command line login

docker login example.com
Username:admin
Password:Abcd12345
Copy the code

Operation command

Shut down

docker-compose down
Copy the code

The configuration is deleted after the redeployment execution, and the project data is not deleted

./prepare
Copy the code

Start the

docker-compose up -d
Copy the code

Trivy is a simple and comprehensive container vulnerability scanner for CI. A software vulnerability is a fault, defect or weakness in software or an operating system. Trivy detects vulnerabilities in operating system packages (Alpine, RHEL, CentOS, etc.) and application dependencies (Bundler, Composer, NPM, YARN, etc.).

Trivy is easy to use, just install the binaries, and you’re ready to scan. The scan only needs to specify the image name of the container. Compared to other image scanning tools such as Clair, Anchore Engine, and Quay, Trivy has significant advantages in accuracy, convenience, and SUPPORT for CI.

It is recommended to use in CI where you can easily scan local Container images before pushing to Container Registry. Trivy has the following characteristics:

1. The detection surface is full, it can detect a full range of vulnerabilities, Operating system software packages (Alpine, Red Hat Universal Base Image, Red Hat Enterprise Linux, CentOS, Oracle Linux, Debian, Ubuntu, and Amazon Linux, openSUSE Leap, SUSE Enterprise Linux, Photon OS and Distrioless), application dependencies (Bundler, Composer, Pipenv, Poetry, NPM, YARN, and Cargo);
2. Easy to use, only need to specify the image name;
3. The scan is fast and stateless, and the first scan will be completed within 10 seconds (depending on your network). Subsequent scans will be completed within a second. Unlike other scanners, which take a long time (about 10 minutes) to retrieve vulnerability information on the first run and encourage you to maintain a persistent vulnerability database, Trivy is stateless and requires no maintenance or preparation;
4. Easy to install, installation method:

$ apt-get install trivy
$ yum install trivy
$ brew install trivy 
Copy the code

Problems encountered and solutions

Symptom The following error occurs when harbor image scanning tool trivy is configured:

2021-04-19T07:19:51.564Z [0m Downloading DB 2021-04-19T07:19:51.564Z [0m Downloading DB... 2021-04-19T07:20:01.566z [31mFATAL[0m failed to download vulnerability DB: Failed to download vulnerability DB: failed to list releases: Get "https://api.github.com/repos/aquasecurity/trivy-db/releases": dial tcp: Lookup api.github.com on 127.0.0.11:53: read udp 127.0.0.1:48822->127.0.0.11:53: I/O timeoutCopy the code

Is the cause of the problems in the download scanning loopholes when the database is timeout, lead to the failure to download, so the solution is to manually download the database and mount trivy container/home/scanner/cache/trivy/db/directory, download path: github.com/aquasecurit… /data/trivy-adapter/trivy/ and set skip_update to true.

2021-04-19T07:37:36.182Z	[31mERROR[0m	The first run cannot skip downloading DB
2021-04-19T07:37:36.182Z	[31mFATAL[0m	database error: --skip-update cannot be specified on the first run
: general response handler: unexpected status code: 500, expected: 200
Copy the code

The reason for this problem is that trivy cannot skip the database download step when scanning the image for the first time. Therefore, the solution is to install Trivy on the local host and scan any image locally to download the vulnerability database. Cache /trivy/ to /data/trivy-adapter/trivy/

2021-04-19T08:53:26Z [ERROR] [/pkg/scan/job.go:284]: check scan report with mime type application/vnd.scanner.adapter.vuln.report.harbor+json; Version =1.0: Running trivy wrapper: Running trivy: exit status 1: 2021-04-19T08:53:22.626z [31mFATAL[0m Unable to initialize the cache: Unable to initialize FS cache: failed to create cache dir: mkdir /home/scanner/.cache/trivy/fanal: no such file or directory : general response handler: unexpected status code: 500, expected: 200Copy the code

The reason for this error is that files under FANal should not be copied when copying the database. Initially, it is suspected that the file is automatically generated when Harbor is started, so the solution is to restore the folder. Copy the files in /root/.cache/trivy/db/ to /data/trivy-adapter/trivy/db and scan the image in harbor again.

The mirror signature function is enabled

Enable the content trust function in Harbor, check the selection box, after checked, unsigned images cannot be pulled, the reality is as follows:

# docker pull example.com/library/foo-apiserver@sha256:0b8cad3c45c2e0db91b070a94c7dc72487d5c1a357168267437518e455f0621f
Error response from daemon: unknown: The image is not signed in Notary.
Copy the code

Sh –with-notary tary –with-notary tary –with-notary tary

# cd ~/.docker/
# ls
config.json
# pwd
/root/.docker
# mkdir tls
# cd tls/
# mkdir example.com:4443
# cd example.com\:4443/
#Cp/home/work/harbor/certs/registry. The CRT ca. CRT / / registry. The CRT for certificate, here the certificate file when using the startup harbor
# ls
ca.crt
# export DOCKER_CONTENT_TRUST=1
# export DOCKER_DONTENT_TRUST_SERVER=https://example.com:4443
Copy the code

The root password and warehouse password must be set for the first upload

# docker push example.com/library/nginx:latestd37eecb5b769: Layer already exists 99134ec7f247: Layer already exists c3a984abe8a8: Layer already exists latest: digest: sha256:7ac7819e1523911399b798309025935a9968b277d86d50e5255465d6592c0266 size: 948 Signing and pushing trust metadata You are about to create a new root signing key passphrase. This passphrase will be used to protect the most sensitive key in your signing system. Please choose a long, complex passphrase and be careful to keep the password and the key file itself secure and backed up. It is highly recommended that you use a password manager to generate the passphrase and keep it safe. There will be no way to recover  this key. You can find the key in your config directory. Enter passphrase for new root key with ID a7d2071: Repeat passphrase for new root key with ID a7d2071: Enter passphrase for new repository key with ID fe4da48: Repeat passphrase for new repository key with ID fe4da48: Finished initializing "reg.westos.org/library/nginx" Successfully signed reg.westos.org/library/nginx:latestCopy the code

Upload image will be automatically signed, then pull image:

# docker pull example.com/library/nginx:latest
Pull (1 of 1): example.com/library/nginx:latest@sha256:c137f6c852bfdf74694fe20693bb11e61b51e0b8c50d17dff881f2db05e65de9
example.com/library/nginx@sha256:c137f6c852bfdf74694fe20693bb11e61b51e0b8c50d17dff881f2db05e65de9: Pulling from library/nginx
Digest: sha256:c137f6c852bfdf74694fe20693bb11e61b51e0b8c50d17dff881f2db05e65de9
Status: Image is up to date for example.com/library/nginx@sha256:c137f6c852bfdf74694fe20693bb11e61b51e0b8c50d17dff881f2db05e65de9
Tagging example.com/library/nginx@sha256:c137f6c852bfdf74694fe20693bb11e61b51e0b8c50d17dff881f2db05e65de9 as example.com/library/nginx:latest
example.com/library/nginx:latest
Copy the code

You only need to enter the warehouse password when you upload another image. You only need to enter the warehouse password when you upload another image with a different version

View authentication information:

# docker trust inspect example.com/library/nginx:latest 
[
    {
        "Name": "example.com/library/nginx:latest",
        "SignedTags": [
            {
                "SignedTag": "latest",
                "Digest": "c137f6c852bfdf74694fe20693bb11e61b51e0b8c50d17dff881f2db05e65de9",
                "Signers": [
                    "Repo Admin"
                ]
            }
        ],
        "Signers": [],
        "AdministrativeKeys": [
            {
                "Name": "Root",
                "Keys": [
                    {
                        "ID": "ef1860607d28455992ad93e71e0e830911e59a43d548c44a41794d490fb63d5b"
                    }
                ]
            },
            {
                "Name": "Repository",
                "Keys": [
                    {
                        "ID": "ebca503ac3b8be80c585a0ba5c5de386f17a978187dd8da75634ad0bb0a7bd4e"
                    }
                ]
            }
        ]
    }
]
Copy the code

Deleting a signature:

# docker trust revoke reg.westos.org/library/nginx:latest 
Enter passphrase for repository key with ID fe4da48: 
Successfully deleted signature for reg.westos.org/library/nginx:latest
Copy the code

Canceling the signature mechanism:

Export DOCKER_CONTENT_TRUST=0 In this case, you need to deselect the content trust option box. Otherwise, upload and deployment cannot be performedCopy the code

Harbor Data Migration

• First shut down harbor repository: docker-compose Down-V
• Copy harbor data /date/ and files to another directory:

Persistent data, such as images and databases, are stored in the /data/ directory of the host computer, while logs are stored in the /data/database/ directory of the host computer, such as authentication data/data /registry/ image file content......Copy the code

Harbor component performance consumption statistics

NAME	CPU %	MEM USAGE / LIMIT
harbor-jobservice	0.39%	20.38 the MiB / 15.42 GiB
nginx	0.00%	13.34 the MiB / 15.42 GiB
notary-server	0.09%	11.52 the MiB / 15.42 GiB
notary-signer	0.00%	8.781 the MiB / 15.42 GiB
harbor-core	0.00%	34.72 the MiB / 15.42 GiB
trivy-adapter	0.32%	32.22 the MiB / 15.42 GiB
registry	0.00%	14 mib / 15.42 GiB
harbor-db	0.00%	98.83 the MiB / 15.42 GiB
redis	0.45%	10.46 the MiB / 15.42 GiB
chartmuseum	0.08%	16.53 the MiB / 15.42 GiB
harbor-portal	0.05%	15.22 the MiB / 15.42 GiB
registryctl	0.11%	14.71 the MiB / 15.42 GiB
harbor-log	0.05%	14.52 the MiB / 15.42 GiB
total	1.54%	272.66 the MiB / 15.42 GiB