1. Images and containers
1.1 mirror
A Docker image is similar to an exe application that is not running, or a VM that has stopped running. When the docker run command is used to start the container based on the image, the container application can provide services externally.
The mirror is essentially the file system that provides an isolated execution environment for the container process. Also known as root file systems (Rootfs). Note that rootfs is only the files, configurations, and directories contained within an operating system, not the operating system kernel. All containers on the same machine share the kernel of the host’s operating system.
Because rootFS encapsulates not only the application, but also all the dependencies it needs to run. This gives the container great consistency: whether locally, in the cloud, or anywhere else, the user simply unpacks the packaged container image and the complete execution environment that the application needs to run is reproduced.
We can understand the Docker image as a basic file system that contains the application program and its related dependencies. During the startup process of the Docker container, it is used in a read-only way to create the operating environment of the container.
1.1.0 Image Layering
If I need an Apache application running in a CentOS environment, I can package it as an Apache image. If I need another mysql application running in a CentOS environment, I package it as a mysql image…
These images have all CentOS environments, which will cause a lot of space usage and fragmentation problems.
Docker’s solution is to introduce the concept of layer into the image design. That is, each step the user takes to create an image generates a layer, which is an incremental rootFS.
Docker images are actually mounted by a set of image layers based on the UnionFS file system in turn, and each image layer contains changes to the previous image layer, and these changes actually occur during the operation of the container. Therefore, we can also understand the reverse, the image is the result of persistent storage of the container runtime environment.
1.1.1 Implementation of Mirroring
1.1.1.1 How does Docker build and store images
Each image in a Docker is made up of a series of read-only layers. Each command in a Dockerfile creates a new layer on top of the existing read-only layer, and each layer in the container makes only very minor changes to the current container
When the image is created by the Docker run command, a writable layer is added to the top layer of the image, namely the container layer. All changes to the runtime container are actually changes to the container’s read-write layer.
The picture above shows the assembly process very well. Each image layer is built on top of another image layer, and all image layers are read-only. Only the top layer of each container can be read and written by users directly. This assembly of containers, including namespaces, control groups, rootFs, and so on, provides great flexibility, and read-only mirror layers can also be shared to reduce disk usage.
1.1.1.2 Overview of Mirroring
All Docker images are packaged according to the logic set by Docker, and are controlled by the Docker Engine.
Our common virtual machine images are usually packaged in familiar ways by enthusiastic providers, downloaded from the Internet or otherwise obtained by us, and restored to the virtual machine’s file system. Docker images must be packaged by Docker, downloaded or imported by Docker, and cannot be directly restored to the file system in the container.
Although this loses a lot of flexibility, the fixed format means that we can easily transfer Docker images between different servers. With the image management function of Docker itself, it is very convenient for us to transfer and share Docker images in different machines.
For each mirrored layer that records file system changes, Docker generates a Hash code based on their information, a string of 64 lengths that is sufficient to ensure global uniqueness.
Since each mirror layer has a unique encoding, we can distinguish between different mirror layers and ensure that their contents are consistent with the encoding, which has the additional benefit of allowing us to share mirror layers between mirrors.
As a practical example, Docker provides two elasticSearch and Jenkins images that are modified on top of openJDK images, so these two images can share the image layer of OpenJDK in actual use.
1.1.2 Viewing a Mirror
To see which images are stored and managed in the docker daemon currently connected, use the Docker images command (this is the same on Linux, macOS, and Windows).
In the result of the Docker images command, we can see the IMAGE ID (IMAGE ID), CREATED time (CREATED), occupied space (SIZE) and other data.
1.1.3 Image Naming
The ID of the mirror layer can be used to identify each mirror layer or directly identify the mirror (because the top layer mirror can find all the dependent lower layers, so the top layer mirror layer ID can represent the ID of the mirror), but using such a meaningless super-long hash code is obviously against human nature. The image name makes it easier to identify the image.
To be precise, the image name can be divided into three parts: Username, repository, and tag.
- Username: identifies different users who upload images, similar to the user space on GitHub.
As for username, in the docker images results shown above, some images have the username part, while others do not. There is no image of username, which means that the image is maintained and provided by Docker officially, so the user will not be marked separately.
- Repository: Mainly used to identify ongoing content and form an ideographic description of the image.
The repository section of the image in Docker usually uses the software name. We prefer a container running a program, so the image of a natural container will only contain the program and some dependencies related to its running, so we use the name of the program directly applied to the image, not only eliminate the trouble of naming the image, but also directly express the content of the image.
- Tag: The primary user represents the version of the image, making it easy to distinguish the different details of the content being performed
Another important part of the image name is the image’s tag. The label of a mirror is a method to distinguish the same image at a higher level, and is also a key part of the final image identification.
In general, the label of an image is mainly used to distinguish between the different results of the construction process of the same image. Due to differences in time, space and other factors, the content of the image Docker builds will be different each time, which is reflected in the change of the image layer and their ID. Tags are the way to distinguish between images at the level of image naming.
Like the repository of images, image tags are usually named in reference to the application associated with the image. More specifically, we usually tag the image with the version number of the application in the image, along with some information about the environment, build style, and so on.
1.2 container
Containers are packages of software into standardized units for development, delivery, and deployment.
- Container images are lightweight, executable, stand-alone packages that contain everything you need to run your software: code, runtime environment, system tools, system libraries, and Settings.
- Containerized software is suitable for Linux – and Windows-based applications and runs consistently in any environment.
- Containers help reduce conflicts between teams running different software on the same infrastructure by giving software independence from external environmental differences (for example, differences in development and rehearsal environments).
1.2.1 Container life cycle
As Docker undertakes most of the work of container management, it only provides us with very simple operation interfaces, which means that some operation details of containers in Docker will be more strictly defined, including the life cycle of containers.
Here is a state flow diagram of the container running:
1.2.2, main process
When we start the container, Docker actually starts the corresponding program as defined in the image, and treats the main process of this program as the main process of the container (i.e., the process with PID 1). When we control the container to stop, Docker sends an end signal to the main process, telling the program to quit.
When the main process in the container shuts down (normally or erratically), the container also stops.
1.2.3 Copy-on-write
The copy-on-write of Docker is similar to programming, that is, when the container is run through the image, instead of immediately copying all contents of the image to the sandbox file system run by the container, UnionFS is used to mount the image to the sandbox file system in a read-only mode. Only when changes are made to the file in the container are the changes reflected in the sandbox environment.
In other words, during the creation and startup of containers, there is no need to carry out any file system copy operation, nor need to open up a large amount of disk space for containers. Compared with the operation of this process in other virtualization methods, the startup speed of Docker can be seen.
Docker’s container is a multi-layered structure. If we do the history operation on the image, we can see that each dockerfile command in it creates a new layer.
[root@ip-172-16-1-4 ec2-user]# docker image history nginx IMAGE CREATED CREATED BY SIZE COMMENT 8cf1bfb43ff5 6 days ago /bin/sh -c #(nop) CMD ["nginx" "-g" "daemon... 0B <missing> 6 days ago /bin/sh -c #(nop) STOPSIGNAL SIGTERM 0B <missing> 6 Days ago /bin/sh -c #(nop) EXPOSE 80 0B <missing> 6 days ago /bin/sh -C #(nop) EXPOSE 80 0B ["/docker-entr... 0B <missing> 6 days ago /bin/sh -c #(nop) COPY file: 0fd5fCA330dCD6a7... 1.04kB <missing> 6 days ago /bin/sh -c #(nop) COPY file: 0fd5fCA330dCD6a7 File: 1d0a4127e78A26c1... 1.96KB <missing> 6 days ago /bin/sh -c #(nop) COPY file:e7e183879c35719c... 1.6KB <missing> 6 days Ago /bin/sh -c set -x &&addgroup --system -... 63.3MB <missing> 6 days ago /bin/sh -c #(nop) ENV PKG_RELEASE=1~ BUSTER 0B <missing> 6 days ago /bin/sh -c #(nop) ENV NJS_VERSION= 0.4.2b <missing> 6 days ago /bin/sh -c #(nop) ENV NGINX_VERSION= 1.19.10 <missing> 6 days ago /bin/sh -c #(nop) LABEL maintainer=NGINX Do... 0B <missing> 6 days ago /bin/sh -c #(nop) CMD ["bash"] 0B <missing> 6 days ago /bin/sh -c #(nop) ADD file: 6ccb3bbCC69b0d44c... 69.2MBCopy the codeDocker has an important concept called Storage driver, which can help us to achieve hierarchical container and read and write. Currently, the default storage driver for Docker is overlay2. All container-related files are stored in /var/lib/docker. We can see that there are many different files in Overlay2
[root@ip-172-16-1-4 ec2-user]# ls /var/lib/docker/overlay2/
12597d435b78d470bed7cf3a4cc7d60691432e74f12c00fd44def7ecf6ab659f 6945f4530a5212bc3a8aa598dc88839d39b763799ccdfbd18a5f04180e3b676e d1f1bcc388595272f11f4ace02f9e6976dc96fd80cce8216d3626484ba114aad
145e76991ae57691ed94de5dd2ea950ff7b50dc26729605e711cd0f35f275e84 7c4d732c22b84e0df8c0d317df825623958d4eff59055827e6b2c76cbe8b0350 d1f1bcc388595272f11f4ace02f9e6976dc96fd80cce8216d3626484ba114aad-init
16da856b3acb71eea396f529a80cf728d664a4bf3eb042f828cb9651d82e90bf 9b6749a9a76ad9f76d7795ebbf8e47595dada7a06b9126f5d9c6e1084f1d0c02 f089b3fd763c560e1c398c9b432100cea56ad4dae3a6b760de42b45e856ce693
16da856b3acb71eea396f529a80cf728d664a4bf3eb042f828cb9651d82e90bf-init a60b187ea615a59402ad2fe6687a4dc87f1c037eafea6880167795c6e1b7f900 f089b3fd763c560e1c398c9b432100cea56ad4dae3a6b760de42b45e856ce693-init
658b1eab364b4523a8c7274e9b8a2bdc55095e9bd56dcb697f6456b4064abe66 a60b187ea615a59402ad2fe6687a4dc87f1c037eafea6880167795c6e1b7f900-init l
658b1eab364b4523a8c7274e9b8a2bdc55095e9bd56dcb697f6456b4064abe66-init backingFsBlockDev
Copy the codeA typical scenario is as follows: inside an image file, there are several layers, and at the bottom is the base image. This base image does not include the kernel file, and when executed, it calls the host kernel directly, so it does not have much space. On top of the base image, there are several layers, each representing a line of command executed in the dockerfile. The entire image file is read-only. Each container creates its own container layer through the image, and the container layer is readable and writable. Changes are stored in its own directory. So each container’s changes to itself do not affect the other containers.
In Docker, we use storage driver to carry out the so-called copy on write operation. The default storage driver is Overlay2
The basic principle of an OVERLAY is as follows
The container we created with the image consists of three layers. The bottom layer is a read-only mirror layer, the second layer is the container layer, and the top container mount layer is above it. The uppermost layer displays what we see directly on the container, which via UnionFS, or something like a soft link, the file path points to the container layer or mirror layer. When we try to read, modify, or create a new file, we always search from top to bottom. If we find it in the container layer, we open it. If not, open it from the mirror layer. If it is a new document, copy it from the image layer to the container layer and open it.
The resources
Docker Practice Guide for Developers