Wechat official account: Operation and maintenance development story, author: Double Dong
1. Tool Description
- Install Docker >= 19.03: Since this Docker version includes Buildx.
- Install buildx: github.com/docker/buil…
- Linux kernel >= 4.8: Since the Linux kernel version binfmt_misc supports fix-binary (F) flag. The Fix_binary flag allows the kernel to use a bINFmT_misc registered binary format handler within a container or chroot, even if the handler binary is not part of the file system visible within that container or chroot.
Docker Buildx is a Docker CLI plug-in that extends the Docker command and supports the functionality provided by Moby BuildKit. Provides the same user experience as Docker Build and adds many new features.
BuildKit is the next generation of mirrored build components with many key features, and this article focuses on its ability to compile multiple systems architectures.
Address: github.com/moby/buildk…
Note that this feature only works with Docker V19.03 +.
This article shows you how to use Buildx to build images of multiple system architectures.
Before you start, you have a 64-bit Docker installed on Linux (major distributions) by default.
As of this writing, the Docker version number used is 19.03.11.
root@i-3uavns2y:~# Docker version Client: Docker Engine - Community version: 19.03.11 API version: 1.40 Go version: Git commit: 42e35e61F3 Built: Mon Jun 1 09:13:48 2020 OS/Arch: Linux/AMd64 Experimental: true Server Docker Engine - Community Engine: Version: 19.03.11 API Version: 1.40 (minimum Version 1.12) Go Version: Git commit: 42e35e61F3 Built: Mon Jun 1 09:12:26 2020 OS/Arch: Linux/AMd64 Experimental: false containerd: Version: 1.2.13 GitCommit: 7 ad184331fa3e55e52b890ea95e65ba581ae3429 runc: Version: 1.0.0 - rc10 GitCommit: Dc9208a3303feef5b3839f4323d9beb36df0a9dd docker - init: Version: 0.18.0 GitCommit: Fec3683 root@i-3uavns2y:~# uname -a Linux i-3uavns2y 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/LinuxCopy the code2. Operation principle
Buildx essentially calls the BuildKit API, and builds are done in a BuildKit environment. Support for multiple architectures depends on the buildKit environment. If buildKit support for multiple architectures is required, it needs to be done on the host (this is not required, but controlled by the build requirements).
BuildKit is designed to build for multiple platforms, not just the architectures and operating systems that the user who calls the build happens to be running.
When calling the build, you can set the –platform flag to specify the target platform for the build output (such as Linux/AMD64, Linux /arm64, or Darwin/AMd64).
When the current builder instance is supported by a Docker-Container or Kubernetes driver, you can specify multiple platforms together. In this case, it builds a manifest list that contains all the types of the specified schema. When you use this image in docker Run or Docker Service, Docker will select the correct image according to the platform of the node.
You can build multi-platform images using three different strategies supported by Buildx and Dockerfiles:
- Use QEMU emulation support in kernel
- Build on multiple native nodes using the same builder instance
- 3 cross-compile to different schemas using a stage in Dockerfile
If your nodes already support QEMU, QEMU is the easiest way to get started (for example, if you’re using Docker Desktop). It doesn’t require any changes to Dockerfile, and BuildKit automatically detects available secondary architectures. When BuildKit needs to run binaries for different architectures, it automatically loads them through binaries registered in the binfmt_misc handler.
For the QEMU binaries registered by bINFmT_misc on the host operating system to work transparently within the container, they must be registered with the Fix_binary flag. This requires kernel >= 4.8 and bINFmt-support >= 2.1.7. You can check for proper registration by checking to see if there is anything in /proc/sys/fs/binfmt_misc/qemu-*. Although Docker Desktop is pre-configured with bINFmT_MISc support for other platforms, for other installations it may need to be installed using tonistiigi/ BINFmt images.
root@i-tpmja312:~# docker run --privileged --rm tonistiigi/binfmt --install all
root@i-tpmja312:~# ls /proc/sys/fs/binfmt_misc/qemu-*
/proc/sys/fs/binfmt_misc/qemu-aarch64 /proc/sys/fs/binfmt_misc/qemu-ppc64le
/proc/sys/fs/binfmt_misc/qemu-arm /proc/sys/fs/binfmt_misc/qemu-riscv64
/proc/sys/fs/binfmt_misc/qemu-mips64 /proc/sys/fs/binfmt_misc/qemu-s390x
/proc/sys/fs/binfmt_misc/qemu-mips64el
Copy the codeUsing multiple native nodes can better support more complex situations that QEMU cannot handle, and generally has better performance. You can use the –append flag to add additional nodes to the builder instance.
2. Enable Buildx
Docker introduced a new feature in 19.03, which enables Docker to build images of different CPU architectures, such as ARM images. This is the original unified construction mechanism provided by Docker itself without introducing emulators, but it can only be used after setting. This article uses the experimental feature as an example to explain how to turn it on temporarily or permanently. (Starting from version V20.10, all experimental commands of the Docker CLI are enabled by default, and there is no need to configure or set system environment variables.)
The buildx command is an experimental feature, so you need to enable it first.
- Phenomenon of the problem
When used directly, the following problems may occur because the experimental feature is not enabled
root@i-3uavns2y:~# docker buildx version
docker: 'buildx' is not a docker command.
See 'docker --help'
Copy the code- Permanently turn on dockerd’s experimental features
Edit the ~/.docker/config.json file and add the following content (the following example applies to cases where the. Docker directory does not exist). Run the docker version command to view the version information, and the Server can be seen after the configuration takes effect: Docker Engine has Experimental: true:
root@i-3uavns2y:~# mkdir ~/.docker root@i-3uavns2y:~# cat > ~/.docker/config.json <<EOF { "experimental": "Enabled"} EOF root@i-3uavns2y:~# docker Buildx version github.com/docker/buildx v0.3.1-tp-docker 6db68d029599c6710a32aa7adcba8e5a344795a7Copy the code- Temporary open
On Linux/macOS or by setting environment variables (not recommended) :
$export DOCKER_CLI_EXPERIMENTAL=enabled $docker buildx version github.com/docker/buildx v0.3.1-tp-docker 6db68d029599c6710a32aa7adcba8e5a344795a7Copy the code3. Create a Builder instance
In Docker 19.03+ you can use the Docker buildx build command to build images using BuildKit. This command supports — the platform parameter can be used to build Docker images supporting multiple system architectures at the same time, greatly simplifying the build steps.
Docker does not support ARM image in Linux system architecture, so we can run a new container to support this feature, Docker desktop version does not need this setting (MAC system).
- Use QEMU emulation support in the kernel for multi-architecture image builds
$docker run --rm --privileged tonistiigi/binfmt:latest --install allCopy the codeNote: the docker/binfmt can reference site: hub.docker.com/r/docker/bi… Obtaining the latest image
Since Docker’s default Builder instance does not support specifying multiple — platforms at the same time, we must first create a new Builder instance. Due to slower domestic pull image at the same time, we can use the configuration mirror accelerate address [dockerpracticesig/buildkit: master] mirror (https://github.com/docker-practice/buildx) to replace the official image
If you have a private mirror accelerator, you can use github.com/docker-prac… Build your own BuildKit image and use it.
root@i-3uavns2y:~# docker buildx create --use --name= myBuilder-cn --driver docker-container --driver-opt Image = dockerpracticesig/buildkit: master # is suitable for tencent cloud environment (tencent cloud host, continuous integration coding.net root @ I - 3 uavns2y: ~ # docker buildx create -- use - the name = mybuilder - cn - driver docker - container - driver - opt image = dockerpracticesig/buildkit: master - tencent # use the default image root@i-3uavns2y:~# docker buildx create --name myBuilder --driver docker-container # root@i-3uavns2y:~# docker buildx use mybuilderCopy the codeView the existing Builder instance
root@i-tpmja312:~# docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
mybuilder * docker-container
mybuilder0 unix:///var/run/docker.sock inactive
default docker
default default running linux/amd64, linux/386
Copy the code4. Create a Dockerfile file
In order to build a variety of system architecture image, also need a support Dockerfile, which multi-architecture image is the most important is the base image and installed software need to support multi-architecture
Here is an example of a Dockerfile file that looks like this:
mkdir ~/demo
cd ~/demo
cat > Dockerfile <<EOF
FROM --platform=$TARGETPLATFORM alpine
RUN uname -a > /os.txt
CMD cat /os.txt
EOF
Copy the code$TARGETPLATFORM is a built-in variable whose value is specified by the –platform parameter.
Since it is based on Alpine’s image, alpine supports the following seven architectures, so our image supports them.
linux/amd64, linux/arm/v6, linux/arm/v7, linux/arm64/v8, linux/386, linux/ppc64le, linux/s390x
Copy the codeThe more friendly schema names are as follows:
amd64, arm32v6, arm32v7, arm64v8, i386, ppc64le, s390x
Copy the codeHere is an interlude for fun, a brief statistics, ARM system architecture has the following abbreviations:
arm64, armv8l, arm64v8, aarch64
arm, arm32, arm32v7, armv7, armv7l, armhf
arm32v6, armv6, armv6l, arm32v5, armv5, armv5l, armel, aarch32
Copy the codeThe comparison between Intel and AMD is much simpler:
x86, 386, i386, i686
x86_64, x64, amd64
Copy the code5. Build an image
Docker Buildx build parameters refer to the official documentation below
Docs.docker.com/engine/refe…
Use the $docker buildx build command to build the image, replacing myusername with your docker Hub username. The push parameter indicates that the built image is pushed to the Docker repository. Build a multi-system architecture image and push the built image to the Docker repository (aka hub.docker.com). To do this, you need to register an account (omitted for the demo) and log in.
The login command is as follows:
root@i-tpmja312:~/demo# docker login
Copy the codeEnter your username and password to log in.
Note that my username doubledong precedes the tag in the command shown below. If you want to make your own image, replace it with your own username.
The image built with the –push argument is pushed to the Docker repository.
The build command is as follows:
Build an image locally that supports seven platforms
root@i-tpmja312:~/demo# docker buildx build --platform linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/386,linux/ppc64le,linux/s390x -t doubledong/hello . --push # root@i-tpmja312:~/demo# docker buildx imageTools inspect doubledong/hello Name: docker.io/doubledong/hello:latest MediaType: application/vnd.docker.distribution.manifest.list.v2+json Digest: sha256:7fd51fbd9f5a478c751ab2138d87341da7937b82bbf2362b23d474727b2c7234 Manifests: Name: docker.io/doubledong/hello:latest@sha256:564098e26174ef2142fbb8bf21d3e57bc2cb31e31933e6e23c5ee8a7bea05219 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/amd64 Name: docker.io/doubledong/hello:latest@sha256:d378c84bcd8bce4b5d771be692bd251a8cb3bbaca9f203d20a5da6989d42c614 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/arm/v6 Name: docker.io/doubledong/hello:latest@sha256:1968399b3651bbcb0dc6218e6dfcb261995723decf39b9c80327624409158ff5 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/arm/v7 Name: docker.io/doubledong/hello:latest@sha256:62ac2af6e39ab10e77d83114931ff1abe449c30a86d0bf590d4bbf71836dcec1 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/arm64 Name: docker.io/doubledong/hello:latest@sha256:3ef9adac67717528ca95f89c184830aa072da155fc17e2a7e95dd9433d9aab51 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/386 Name: docker.io/doubledong/hello:latest@sha256:67e4c3e5c7eca8af5909373446e9e5fa6d1083223b3766bb8e9e6f41c01ca43b MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/ppc64le Name: docker.io/doubledong/hello:latest@sha256:f3dc2d058e4915a61281d945ae92e77b44e2b81a601c63470ce1912e4e29c53e MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/s390xCopy the codeAfter executing the command successfully, you will see the image you uploaded in Docker Hub. Here’s an example:
Now that you’ve done that, you’ve actually put your image in your local path, so let’s take a look at your existing Builder instance.
root@i-tpmja312:~/demo# docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
mybuilder * docker-container
mybuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default docker
default default running linux/amd64, linux/386
Copy the codeYou will find 8 architectures supported under MyBuilder (RISCV64 is not currently available, but it is supported).
If you look at the Docker image in action, you will see that there is a container named buildx_buildkit_mybuilder0 running.
This was created automatically when you built locally, so don’t stop it or delete it.
root@i-tpmja312:~/demo# docker ps -as|grep buildx_buildkit
e274b21faea2 moby/buildkit:buildx-stable-1 "buildkitd" 7 minutes ago Up 7 minutes buildx_buildkit_mybuilder0 0B (virtual 144MB)
Copy the code6. Write at the end
When making Docker images with multi-system architecture, it is recommended to use VPS with strong CPU or multi-core to build them, otherwise it will be very time-consuming. This article mainly focuses on manual construction of multi-architecture images, or cicD tool can be used for automatic construction, which will be explained in the following articles
Reference links:
Yeasy. Gitbook. IO/docker_prac…
Github.com/docker/buil…