“

DockerHub recently changed its pricing to have 200 “effects” registered for free accounts and 100 “effects” registered for anonymous accounts. In this article, I show you how to use Cache to solve this problem.

“

background

DockerHub is the world’s earliest and largest container image warehouse, hosting a large number of operating system releases and various software Docker images.

As we move forward with business containerization, it is inevitable that we will need to use container images from DockerHub. Whether used in a personal local environment or for running test services

Here are two main solutions:

  • Build some common base images to store in the enterprise’s private image repository for business use:

    In this scenario, if the business side occasionally needs a niche/non-basic image, perhaps for temporary testing purposes, it is generally not necessary to maintain such an image as a base image.

    The result could be:

    • Use the image directly from DockerHub pull, when the network condition is poor, it is endless wait;
    • Pull the image first, and thendocker tagAfter the tag is reloaded, it is pushed to the enterprise’s private image repository. In this case, without good mirror management rules, there will be all kinds of meaningless mirrors in the mirror warehouse, resulting in a waste of storage resources.

  • Configure Proxy for Docker Daemon acceleration:

    • Many domestic image acceleration services only provide the acceleration service of Docker official images, individual/organization images do not provide acceleration service;
    • Even if the same image is downloaded on different nodes, it still needs to be accelerated through the network, resulting in additional overseas bandwidth costs.

In addition, DockerHub has recently revised its service price [1] and imposed the following restrictions on free users:

  • An unlogged user can pull only 100 times every six hours
  • A logged-in user can pull only 200 times every six hours

If we continue with the above two modes, the export IP is relatively fixed, so it is easy to trigger quota restrictions on DockerHub. The restrictions will be fully implemented on November 1.

In order to improve efficiency and save the cost of accelerated bandwidth, enterprises/individuals are in great need of a DockerHub full image accelerated service, also known as pull through cache.

Let me introduce how to use Docker open source project Registry :2 to achieve this requirement.

Start the service

Using Registry :2 to deploy the image caching service is simple. Here we start by executing docker pull Registry :2 to download the required image:

(MoeLove) ➜ docker pull Registry :22: Pulling from library/registry
cbdbe7a5bc2a: Pull complete 
47112e65547d: Pull complete 
46bcb632e506: Pull complete 
c1cc712bcecd: Pull complete 3db6272dcbfa: Pull complete Digest: sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d Status: Downloaded newer image for registry:2 docker.io/library/registry:2 Copy the code

To minimize the configuration of the DockerHub image caching service, use a single configuration item REGISTRY_PROXY_REMOTEURL:

Incidentally, I created a separate network named Hub-cache for it, along with the corresponding volume.

(MoeLove) ➜  ~ docker network create hub-cache
19a39f873a23150d3bdaf021e040ccccb092ee3071884d64d52a92df0397b220
(MoeLove) ➜  ~ docker volume create hub-cache
hub-cache
(MoeLove) ➜  ~ docker run --name=cache -d --restart=always --network=hub-cache -v hub-cache:/var/lib/registry -p 5000:5000  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io registry:2 
6cbdcbdcc2d62ec781479901c20be43184a48b2d73e06f04bd4693253c0c5a73 (MoeLove) ➜ ~ Docker ps-lCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6cbdcbdcc2d6 registry:2 "/ entrypoint. Sh/etc..."8 seconds ago Up 6 seconds 0.0.0.0:5000->5000/ TCP cacheCopy the code

Verify acceleration

Start a brand new Docker In Docker container for verification to avoid being affected by the local environment.

Set the registry you just started to mirror by passing –registry-mirror http://cache:5000.

(MoeLove) ➜ ~ docker run --network= Hub-cache-d -- Privileged Docker :dind --registry-mirror http://cache:500073c56ac25d68927c9f5b0e458f2babc0699cf8595df0d1e86c021fd03d477384
(MoeLove) ➜  ~ docker exec -it $(docker ps -ql) sh
/ # Check whether the configuration takes effect
/ # docker info --format '{{ .RegistryConfig.Mirrors }}' 
[http://cache:5000/]  / # time docker pull prom/prometheus Using default tag: latest latest: Pulling from prom/prometheus 76df9210b28c: Pull complete 559be8e06c14: Pull complete 6a4bb3319487: Pull complete 2cca90a64593: Pull complete d2014e464a99: Pull complete 70b42590e4a2: Pull complete 54645fcbd6cc: Pull complete 67d9943de656: Pull complete b9c749b1af90: Pull complete 9723d8eb5323: Pull complete 7d20502d5322: Pull complete 3e519cce6f63: Pull complete Digest: sha256:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971 Status: Downloaded newer image for prom/prometheus:latest docker.io/prom/prometheus:latest Real 0 m 42.71 sThe user 0 m 0.12 sSys 0 m 0.09 s  / # docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE prom/prometheus latest cdfc440228d0 8 days ago 168MB  / # docker rmi prom/prometheus  Untagged: prom/prometheus:latest Untagged: prom/prometheus@sha256:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971 Deleted: sha256:cdfc440228d01d7a94937d7a047fa6461efc1b1806bb20677043fee032810830 Deleted: sha256:c72f348fd2f923996ea80222feb77e34aba9de397bd96206ddc3c8651adc306d Deleted: sha256:e34df2c00334266a67bb846b958ba6eae3b1d5cdfe9d763707027a23e7c85100 Deleted: sha256:d2cb38310ada122064b7333bbfc12c67dc58acb30e29146b3ba1e24adc27a950 Deleted: sha256:7a87cd520d19a83b3582541aac4d95098ae5016b092e72eaf80dc54f587bf51e Deleted: sha256:f84c79dceed6b5a27234c1291d0bdccab5c459d587f13934d74db9b9e79471c6 Deleted: sha256:f542b0cffe0fe16c31c98e7eed934d5fea5e598c03b53b4efd308a62e0e9c6a9 Deleted: sha256:f746b4a525727bcb79367d009d707ef45d75bac09aaa18a68c20a19046df0897 Deleted: sha256:09b45653ee7062c7cd754885bf46ebe554d0794573fb2e200acea8644e64670f Deleted: sha256:867526c56b30e67493341ef33890aa242c1131e4bb4151e60011b4d450892d59 Deleted: sha256:86d629b358ee70bdb0f0a11c10915b8551e904fe337f9a8bfcad476977329532 Deleted: sha256:842455c528af7383ba4a0de424fc63664a0248581a191516d6dbf45195c69426 Deleted: sha256:1be74353c3d0fd55fb5638a52953e6f1bc441e5b1710921db9ec2aa202725569  / # time docker pull prom/prometheus Using default tag: latest latest: Pulling from prom/prometheus 76df9210b28c: Pull complete 559be8e06c14: Pull complete 6a4bb3319487: Pull complete 2cca90a64593: Pull complete d2014e464a99: Pull complete 70b42590e4a2: Pull complete 54645fcbd6cc: Pull complete 67d9943de656: Pull complete b9c749b1af90: Pull complete 9723d8eb5323: Pull complete 7d20502d5322: Pull complete 3e519cce6f63: Pull complete Digest: sha256:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971 Status: Downloaded newer image for prom/prometheus:latest docker.io/prom/prometheus:latest Real 0 m 5.27 sThe user 0 m 0.06 sSys 0 m 0.03 sCopy the code

As you can see, in the first pullprom/prometheusIt takes 42+s to mirror, but it only takes 5+s to pull again after deleting the downloaded image. The speed increase is very significant. Mirror acceleration effect achieved

Using a configuration

For Linux, just write the domain name of your image acceleration service in /etc/docker-daemon. json file (if you don’t have this file, you can create it directly) and restart the Docker daemon (reload is also optional).

{
        "registry-mirrors": [
                "https://hub-cache.moelove.info"
        ]
}
Copy the code

Or add registry-mirror configuration items to the startup parameters of the Docker Daemon.

For Mac and Windows users, registry-mirrors can be configured directly in the Docker Desktop system Settings.

Notice If HTTP_PROXY or HTTPS_PROXY is configured in the Docker Daemon, you need to configure the accelerated domain name in NO_PROXY to avoid proxy.

conclusion

This article introduces how to use Docker open source Registry :2 to build DockerHub image acceleration service. Only the simplest configuration is described here.

However, if deployed in an enterprise environment, more configuration is required. For example, Prometheus Metrics can be exposed by configuring REGISTRY_HTTP_DEBUG_PROMETHEUS_ENABLED to monitor service availability and view cache performance; You can configure logs and related fields. To avoid triggering traffic restriction to DockerHub after November, perform horizontal capacity expansion, prepare multi-egress IP addresses, and configure accounts.

Harbor V2.1 has a proxy cache feature, but it is different from the pull through cache feature described in this article. To use Harbor’s proxy cache feature, you need to mirror the image to be pulled. Set it to

/ /repo/name:tag. This saves the manual re-tag that was introduced at the beginning of this article. It’s not convenient, but it’s a nice feature.

Please feel free to subscribe to my official account [MoeLove]

The resources

[1]

DockerHub pricing: https://www.docker.com/pricing

Thanks for following MoeLove