How to set up a private Docker registry on Ubuntu 20.04

As aWrite for DOnationsPart of the plan, the author choosesFree and Open Source foundationsTo receive donations.

Introduction to the

Docker Registry is an application that manages the storage and delivery of Docker container images. The registry centralizes container images, reducing developer build time. Docker images are virtualized to ensure the same operating environment, but building an image may take a lot of time. For example, instead of installing dependencies and packages separately to use Docker, developers can download a compressed image that contains all the necessary components from the registry. In addition, developers can use continuous integration tools such as TravisCI to automatically push images to the registry so they can be updated seamlessly during production and development.

Docker also has a free public registry, the Docker Hub, which can host your custom Docker images, but in some cases you don’t want your images to be publicly available. Images usually contain all the code needed to run an application, so it is best to use a private registry when using proprietary software.

In this tutorial, you will create and protect your own private Docker registry. You will use Docker Compose definition configuration to run your Docker container and use Nginx to forward server traffic from the Internet to the running Docker container. After completing this tutorial, you will be able to push custom Docker images to your private registry and securely extract images from remote servers.

The premise condition

• Two, according to theUbuntu 20.04 Initial Server Setup GuideSetup Ubuntu 20.04 server, including onesudoNon-root user and a firewall. A server willhostingYour private Docker registry, the other one will be yoursClient server.
• Follow steps 1 and 2 in How to Install and Use Docker on Ubuntu 20.04 to install Docker on both servers.
• Install Docker Compose on the primary server as described in Step 1 of How to Install and Use Docker Compose on Ubuntu 20.04.
• Follow the steps in How to Install Nginx on Ubuntu 20.04 to install Nginx on the host.
• Follow the tutorial on “How to Protect Nginx with Let’s Encrypt on Ubuntu 20.04” to secure Nginx for the private Docker registry on the server. Be sure to redirect all traffic from HTTP to HTTPS in step 4.
• A domain name that resolves to the server you use for your private Docker registry. You will set this up as part of the Let’s Encrypt prerequisite. In this tutorial, we will call ityour_domain.

Step 1 – Install and configure the Docker registry

Docker on the command line is useful when starting and testing containers, but proves inconvenient for large deployments that involve multiple containers running in parallel.

With Docker Compose, you can write a.yml file to set up the configuration of each container and the information the containers need to communicate with each other. You can use the Docker-compose command line tool to issue commands to all the components that make up your application and control them as a group.

Docker Registry is itself an application with multiple components, so you will use Docker Compose to manage it. To start an instance of the registry, you’ll set up a docker-comemage. yml file to define it and the location on disk where your registry will store data.

You will store the configuration in a directory called Docker-Registry on the main server. Create it by running.

mkdir ~/docker-registry
Copy the code

Navigate to it.

cd ~/docker-registry
Copy the code

Then, create a subdirectory called Data where your registry will store its images.

mkdir data
Copy the code

Create and open a file called docker-comemage.yml by running it.

nano docker-compose.yml
Copy the code

Add the following lines that define a basic instance of the Docker registry.

~/docker-registry/docker-compose.yml

version: '3'

services:
  registry:
    image: registry:2
    ports:
    - "5000:5000"
    environment:
      REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
    volumes:
      - ./data:/data
Copy the code

First, you name the first service Registry and set its mirror to Registry at version 2. Then, at ports, you map port 5000 on the host to port 5000 on the container. This allows you to send requests to port 5000 on the server and have the requests forwarded to the registry.

In the Environment section, you set the REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY variable to /data, specifying which volume it should store data on. Then, in the volumes section, you mapped the /data directory on the host file system to the /data directory in the container for penetration. The data will actually be stored on the host’s file system.

Save and close the file.

Now you can start the configuration by running.

docker-compose up
Copy the code

The registry container and its dependencies will be downloaded and started.

OutputCreating network "docker-registry_default" with the default driver Pulling registry (registry:2)... 2: Pulling from library/registry e95f33c60a64: Pull complete 4d7f2300f040: Pull complete 35a7b7da3905: Pull complete d656466e1fe8: Pull complete b6cb731e4f93: Pull complete Digest: sha256:da946ca03fca0aade04a73aa94b54ff0dc614216bdd1d47585f97b4c1bdaa0e2 Status: Downloaded newer image for registry:2 Creating docker-registry_registry_1 ... The done Attaching to docker - registry_registry_1 registry_1 | time = "of" the 2021-03-18 T12:32:59. 587157744 z level = warning MSG = "No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a  load-balancer. To provide a shared secret, Fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.11.2 Instance. Id = 119 a8d bb6 fe50b - 2-4-902 - d - dfa2db63fc2f service = registry version = v2.7.1 registry_1 | Time =" 2021-03-18T12:32:59.587912733z "level=info MSG ="redis not configured" go.version=go1.11.2 Instance. Id = 119 a8d bb6 fe50b - 2-4-902 - d - dfa2db63fc2f service = registry version = v2.7.1 registry_1 | Time =" 2021-03-18T12:32:59.598496488z "level=info MSG =" Using Inmemory Blob Descriptor cache" go.version=go1.11.2 Instance. Id = 119 a8d bb6 fe50b - 2-4-902 - d - dfa2db63fc2f service = registry version = v2.7.1 registry_1 | Time =" 2021-03-18T12:32:59.601503005z "level=info MSG =" Listening on [::]:5000" go.version=go1.11.2 Instance. Id = 119 a8d bb6 fe50b - 2-4-902 - d - dfa2db63fc2f service = registry version = v2.7.1...Copy the code

You will address the No HTTP Secret Provided warning later in this tutorial. Notice that the last line of the output shows that it has successfully started listening on port 5000.

You can press CTRL+C to stop its execution.

In this step, you have created a Docker Compose configuration that starts a Docker Registry listening on port 5000. In the next steps, you will expose it in your domain and set up authentication.

Step 2 – Set up Nginx port forwarding

As part of the prerequisites, you have HTTPS enabled in your domain. To demonstrate secure Docker registries here, you just need to configure Nginx to forward traffic from your domain name to the registry container.

You have set up the /etc/nginx/sites-available/your_domain file that contains your server configuration. Open it by running.net for editing.

sudo nano /etc/nginx/sites-available/your_domain
Copy the code

Locate the existing Location block.

/etc/nginx/sites-available/your_domain

. location / { ... }...Copy the code

You need to forward traffic to port 5000, where the registry will listen for traffic. You also want to add header information to requests forwarded to the registry, which will provide the server with additional information about the request itself. Replace the existing contents of the Location block with the following lines.

/etc/nginx/sites-available/your_domain

. location / {# Do not allow connections from docker 1.5 and earlier
    # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
    if ($http_user_agent ~ "^(docker\/1\.(3|4|5(? ! \.[0-9]-dev))|Go ).*$" ) {
      return 404;
    }

    proxy_pass                          http://localhost:5000;
    proxy_set_header  Host              $http_host;   # required for docker client's sake
    proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
    proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header  X-Forwarded-Proto $scheme; proxy_read_timeout 900; }...Copy the code

The if block checks the requested user agent and verifies that the Docker client version is above 1.5 and that it is not a Go application that is trying to access. For more explanation of this, you can find the configuration of Nginx headers in the Docker registry Nginx guide.

When finished, save and close the file. These changes are applied by restarting Nginx.

sudo systemctl restart nginx
Copy the code

If you get an error, double check the configuration you added.

To verify that Nginx is correctly forwarding port 5000 traffic to your registry container, run it.

docker-compose up
Copy the code

Then, in a browser window, navigate to your domain name and access the V2 endpoint like this.

https://your_domain/v2
Copy the code

You will see an empty JSON object.

{}
Copy the code

On your terminal, you will receive output similar to the following.

Outputregistry_1 | time = "of" the 2018-11-07 T17:57:42 Z level = info MSG = "response" completed. Version = go1.7.6 http.request.host=cornellappdev.com http.request.id=a8f5984e-15e3-4946-9c40-d71f8557652f http.request.method=GET HTTP. Request.. Remoteaddr = 128.84.125.58 HTTP request. The uri = "/ v2 /" HTTP. Request. Useragent = "Mozilla / 5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, Like Gecko) Version / 11.0.2 Safari / 604.4.7 "HTTP. Response. Contenttype =" application/json. Charset = utf-8 "HTTP. Response. Duration = 2.125995 ms. HTTP response. The status = 200 HTTP response. The written = 2 The instance. The id = 3093 e5ab - 5715-42 BC - 808 - e - 73 f310848860 version = v2.6.2 registry_1 | 172.18.0.1 - [07 / Nov / 2018:17:57:42 + 0000] GET /v2/ HTTP/1.0 200 2 "" Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7"Copy the code

As you can see from the last line, a GET request was made from your browser to /v2/, the endpoint from which you sent the request. The container receives your request from port forwarding and returns a response: {}. The code 200 in the last line of output means that the container successfully processed the request.

Press CTRL+C to stop its execution.

Now that you have port forwarding set up, you will continue to improve the security of your registry.

Step 3 – Set up authentication

Nginx allows you to set UP HTTP authentication for the sites it manages, which you can use to restrict access to the Docker registry. To do this, you will create an authentication file by using htpasswd and adding the combination of username and password that will be accepted.

You can get the htpasswd tool by installing the Apache2-utils package. Do so by running.

sudo apt install apache2-utils -y
Copy the code

You will store the authentication file with the certificate in ~/docker-registry/auth. Create it by running:.

mkdir ~/docker-registry/auth
Copy the code

Browse to it.

cd ~/docker-registry/auth
Copy the code

Create the first user and change username to the username you want to use. The -b flag command uses the bcrypt algorithm required by Docker.

htpasswd -Bc registry.password username
Copy the code

Enter the password when prompted and the combination of credentials will be appended to Registry.password.

** Note: ** to add more users, rerun the previous command instead of using -c, which will create a new file.

htpasswd -B registry.password username
Copy the code

Now that the credentials list is complete, you’ll edit docker-comemage. yml and command Docker to authenticate the user using the file you created. Edit by running open it.

nano ~/docker-registry/docker-compose.yml
Copy the code

Add highlighted lines.

~/docker-registry/docker-compose.yml

version: '3'

services:
  registry:
    image: registry:2
    ports:
    - "5000:5000"
    environment:
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/registry.password
      REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
    volumes:
      - ./auth:/auth
      - ./data:/data
Copy the code

You have added environment variables specifying HTTP authentication and provided the path to the file created by htpasswd. For REGISTRY_AUTH, you have specified htpasswd as its value, which is the authentication scheme you are using, and set REGISTRY_AUTH_HTPASSWD_PATH to the path of the authentication file. REGISTRY_AUTH_HTPASSWD_REALM indicates the name of the htpasswd realm.

You have also mounted the./auth directory to make the file available in the registry container. Save and close the file.

Now you can verify that your authentication work is correct. First, navigate to the home directory.

cd ~/docker-registry
Copy the code

The registry is then run by execution.

docker-compose up
Copy the code

In your browser, refresh your domain page. You will be asked for a username and password.

After providing a valid certificate combination, you will see an empty JSON object.

{}
Copy the code

This means that you have successfully authenticated and obtained access to the registry. Press CTRL+C to exit.

Your registration appears to be secure and can only be accessed after authentication. Now you will configure it to run as a background process, while being resilient to restarts through auto-start.

Step 4 – Start the Docker registry as a service

You can ensure that the registry container starts up every time a system starts up or crashes by instructing Docker Compose to keep the registry running at all times. Open docker-comemage. yml and edit it.

nano docker-compose.yml
Copy the code

Add the following line under the Registry block.

docker-compose.yml

. registry: restart: always ...Copy the code

Setting restart to always ensures that the container will still exist after a restart. When finished, save and close the file.

You can now start your registry as a background process by typing -d.

docker-compose up -d
Copy the code

When your registry is running in the background, you are free to close the SSH session without affecting the registry.

Since the size of Docker images can be very large, you will now increase the maximum upload file size accepted by Nginx.

Step 5 – Increase the file upload size for Nginx

Before pushing images to the registry, you need to ensure that the registry can handle large file uploads.

Nginx’s default file upload size limit is 1 MB, which is far from enough for Docker images. To increase this limit, you need to modify the main Nginx configuration file, which is located in /etc/nginx.conf. Open it for editing by running the following command.

sudo nano /etc/nginx/nginx.conf
Copy the code

Locate the HTTP section and add the following line.

/etc/nginx/nginx.conf

. http { client_max_body_size 16384m; . }...Copy the code

The client_max_body_size parameter is now set to 16384M, making the maximum upload equal to 16GB.

When finished, save and close the file.

Restart Nginx to apply the configuration changes.

sudo systemctl restart nginx
Copy the code

Now you can upload large images to the Docker registry without Nginx blocking the transfer or causing errors.

Step 6 – Publish to your private Docker registry

Now that your Docker registry server is up and running and accepts large file sizes, you can try pushing an image to it. Since you don’t have any ready-made images, you’ll use an Ubuntu image from Docker Hub (a public Docker registry) to test.

On your second client server, run the following command to download the Ubuntu image, run it, and get access to its shell.

docker run -t -i ubuntu /bin/bash
Copy the code

The -i and -t flags give you interactive shell access to the container.

Once you are in, create a file called SUCCESS by running it.

touch /SUCCESS
Copy the code

By creating this file, you have customized your container. You’ll use it later to check if you’re using the exact same container.

Exit the container shell by running.

exit
Copy the code

Now, create a new image from the container you just customized.

docker commit $(docker ps -lq) test-image
Copy the code

The new image is now available locally and you will push it to your new container registry. First, you have to log in.

docker login https://your_domain
Copy the code

When prompted, enter the username and password combination that you defined in step 3 of this tutorial.

The output will be.

Output...
Login Succeeded
Copy the code

Once you are logged in, rename the image you created.

docker tag test-image your_domain/test-image
Copy the code

Finally, push the image of the new tag to your registry.

docker push your_domain/test-image
Copy the code

You should receive output similar to the following.

OutputThe push refers to a repository [your_domain/test-image]
420fa2a9b12e: Pushed
c20d459170d8: Pushed
db978cae6a05: Pushed
aeb3f02e9374: Pushed
latest: digest: sha256:88e782b3a2844a8d9f0819dc33f825dde45846b1c5f9eb4870016f2944fe6717 size: 1150
Copy the code

You have validated your registry to handle user authentication by logging in and allowing authenticated users to push images to the registry. Now you will try to extract the image from your registry.

Step 7 – Extract from your private Docker registry

Now that you have pushed an image to your private registry, you will try to pull it from it.

On the main server, log in with the same username and password you set earlier.

docker login https://your_domain
Copy the code

Try to extract by running test-image.

docker pull your_domain/test-image
Copy the code

Docker will download the image. Run the container with the following command.

docker run -it your_domain/test-image /bin/bash
Copy the code

List existing files by running.

ls
Copy the code

You will see the SUCCESS file you created earlier and confirm that it is the same image you created.

SUCCESS  bin  boot  dev  etc  home  lib  lib64  media   mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
Copy the code

Exit the container shell by running.

exit
Copy the code

Now that you’ve tested push and extract images, you’ve finished setting up the security registry that you can use to store custom images.

conclusion

In this tutorial, you create your own private Docker registry and publish a Docker image to it. As mentioned in the introduction, you can also use TravisCI or a similar CI tool to automatically push directly to the private registry. By leveraging Docker containers in your workflow, you can ensure that the image containing the code will produce the same behavior on any machine, whether in production or in development. For more information about writing Docker files, you can visit the official documentation on best practices.