Docker architecture and the release of self-made images

One. What is a DockerDocker with automated testing and its testing practices

We all know about virtual machines, Windows installed a Linux virtual machine is a common solution for most programmers. Most production environments are also virtual machines (VMS), which virtualize physical hardware resources and allocate and use them on demand. VMS can be used exactly like real operating systems. If they are not used, you can delete VM files to reclaim resources, facilitating centralized management.

Because virtual machines are so large and consume so much hardware resources, Linux has developed another virtualization technology, Linux Containers (LXC for short), which do not emulate a complete operating system like virtual machines, but provide the same effect as virtual machines. If virtual machines are isolated at the operating system level, then containers are isolated at the process level. You can imagine the advantages of this level of isolation, which is undoubtedly fast and resource-saving.

Docker is the packaging of Linux container, providing simple and practical user interface, is the most popular Linux container solution.

Here’s the definition:

Docker is an open source application container engine based on the Go language and complies with the Apache2.0 protocol. Docker allows developers to package their applications and dependencies into a portable container and then distribute them to any popular Linux machine, as well as realize virtualization. Containers are completely sandboxed and have no interface with each other.

Ii. What problem does Docker solve

1. Resolve the problem of VM resource consumption.

Virtual machines run on the server operating system, virtual machines run on the guest operating system, and user applications run on the guest operating system. Eighty percent of the resource expenditure of a server is spent on hardware virtualization and the client operating system itself.

As shown in Figure 1, if docker container technology is adopted, the virtual server runs on the container, and the user’s application program runs in the virtual server. The operating system of the virtual server and the server use the same kernel, and the file system of the virtual server uses the file system of the physical server, but is isolated. It looks like each virtual server has its own separate file system; Virtual bridge devices are set up on physical servers, and each virtual server connects to the network through virtual bridge devices. Virtual servers use the CPUS, memory, and hard disks of physical servers and do not virtualize hardware. Therefore, there is no hardware virtualization or resource consumption of client operating systems. The performance of each virtual server is close to that of a physical server.

A typical home computer may be too tired to run a Linux virtual machine, but docker can be used to create dozens or even hundreds of virtual Linux servers. If you switch to a more powerful server, you can use Docker to provide private cloud services.

2. Rapid deployment.

The difficulty of software development lies in the configuration of the environment. Software that runs on your own computer may not run on another machine unless the operating system is set up correctly and components and libraries are installed correctly. For example, to deploy a Java developed Web system, the computer must have Java installed and the correct environment variables, and may also need to install Tomcat, Nginx. Switch machines and start all over again.

Using Docker can package the application and dependencies in a file (Docker image file), running this file will start the virtual server, start the application or service in the virtual server, just like running on the real physical machine, with Docker, you can deploy once, run everywhere, It can also be used for automated publishing.

3. Provide a disposable environment.

For example, testing other people’s software locally, providing unit testing and building environments for continuous integration, and starting or shutting down a virtual server is as easy and quick as starting or shutting down a process.

4. Provide resilient cloud services.

Because Docker containers can be opened and closed at any time, it is very suitable for dynamic expansion and shrinkage.

5. Build a microservice architecture.

With multiple containers, a single machine can run many virtual servers, so you can simulate a microservice architecture on a single machine, as well as a distributed architecture.

Iii. Docker installation, deployment and use

This article describes how to install and use Ubuntu 18.04. For other operating systems, see docs.docker.com/.

1. Install the Docker engine

Obtain the latest Docker installation package

aaron@ubuntu:~$ wget -qO- https://get.docker.com/ | sh

Copy the code

Run the preceding command and enter the current user password to automatically download the latest version of the Docker installation package and install it. After the installation is complete, there is a hint:

If you would like to use Docker as a non-root user, you should now consider
adding your user to the "docker" group with something like:

  sudo usermod -aG docker aaron

Remember that you will have to log out and back in for this to take effect!

WARNING: Adding a user to the "docker" group will grant the ability to run
         containers which can be used to obtain root privileges on the
         docker host.
         Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
         for more information.

Copy the code

This is required when you want to run Docker directly as a non-root user

sudo usermod -aG docker aaron 

Copy the code

Docker add user Aaron to docker group

docker: Got permission denied whiletrying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial Unix/var/run/docker. The sock: connect: permission denied. See'docker run --help'.

Copy the code

Run the following command to start the Docker engine

aaron@ubuntu:~$ sudo service docker start

Copy the code

After the installation is successful, the startup is automatically set upon startup by default. To manually set the startup, run the following command:

sudo systemctl enable docker
sudo systemctl start docker

Copy the code

A test run

aaron@ubuntu:~$ sudo docker run hello-world

Copy the code

2. Use the docker

1. Understand the architecture of Docker

Before using it, understand the architecture of Docker, as shown below:

Docker image is a file stored in The Docker Registry and a template used to create docker containers.

A Docker container is a single application or group of applications that run independently and can be understood as the virtual server described above.

A Docker host is a physical or virtual machine that executes docker daemons and containers.

Docker clients communicate with docker daemons using docker APIS through command lines or other tools. As users, we directly use the Docker client.

2. The docker command

View the help information about the docker command

docker --help  Docker command help information
docker COMMAND --help Docker COMMAND help information

Copy the code

View docker information

docker info

Copy the code

You can see the container’s pool, used data size, total data size, base container size, number of containers currently running, and so on.

Search for images. Search for container images made by others on the Internet.

docker search ubuntu
docker search centos

Copy the code

From this we can see that some images have integrated PHP, Java, Ansible and other applications. We can also make an image file containing our own application or service, and send this file to others. They can open the container directly using Docker, without any additional operations, and without consuming resources like virtual machines. How convenient is it to be able to run your application or service? !

If software testing, interface testing, automated testing, performance testing, LR script development, interview experience exchange. If you are interested, you can go to 175317069. There will be free information links in the group from time to time, which are collected and sorted out from various technical websites. If you have good learning materials, you can send them to me privately.

Download someone else’s container image from the Internet.

docker pull centos
docker pull ubuntu

Copy the code

Import the downloaded container image file

docker load < image_xxx.tar

Copy the code

Look at mirror

docker images
docker images -a

Copy the code

Check the mirror

docker inspect ubuntu

Copy the code

You can see the basic information about the container image.

To delete a mirror, specify the id of the mirror to be deleted

docker rmi ubuntu

Copy the code

Deleting All Mirrors

docker rmi $(docker images -q)

Copy the code

Display mirror history

docker history ubuntu

Copy the code

Run the container

Docker container can be understood as a process running in a sandbox, which contains resources necessary for the process to run, including file system, system class library, shell environment, etc. However, the sandbox does not run any programs by default. You need to run a process in the sandbox to start a container. The process is unique to the container, so when the process terminates, the container stops completely.

Run the Ubuntu container and enter the interactive environment

aaron@ubuntu:~$ docker run -i --name="ubuntu1" --hostname="ubuntu1"Ubuntu /bin/sh cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet FF00 ::0 Ip6-mcastprefix FF02 ::1 IP6-allNodes FF02 ::2 IP6-allRouters 172.17.0.2 ubuntu1 whoami root uname-aLinux ubuntu1 4.15.0-34 - generic#37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Copy the code

We created a container named ubuntu1 and set the host name of the container to ubuntu1. After entering the /bin/sh command, we printed the hosts file and checked the kernel version (consistent with the local OS version). Various Linux commands can be used here. Just like using commands in the new operating system. In the same way, we created a Ubuntu2 container on the new terminal and used it

docker ps

Copy the code

View the running container.

Type exit to exit the container.

docker run -d ubuntu 

Copy the code

A long alphanumeric string will appear for the container ID. Note that the container must have a continuously running process, otherwise the container will automatically exit soon.

Run the container and specify the MAC address

docker run -d --name='centos3' --hostname='centos3' --mac-address="02:42:AC:11:00:24"Docker - centos6.10 - hadoop - sparkCopy the code

List all containers

docker ps -a

Copy the code

Lists the containers that were last started

docker ps -l

Copy the code

Check the container

docker inspect centos1

Copy the code

You can get information about the container.

Get container CID

docker inspect -f '{{.Id}}' centos1

Copy the code

Obtaining the container PID

docker inspect -f '{{.State.Pid}}' centos1

Copy the code

Obtaining the CONTAINER IP address

docker inspect -f '{{.NetworkSettings.IPAddress}}' centos1

Copy the code

Get container Gateway

docker inspect -f '{{.NetworkSettings.Gateway}}' centos1

Copy the code

Obtaining the container MAC

docker inspect -f '{{.NetworkSettings.MacAddress}}' centos1

Copy the code

View the CONTAINER IP address

docker inspect -f '{{.NetworkSettings.IPAddress}}' centos1

Copy the code

Connect the container

IP address of the SSH containerCopy the code

Enter the password: 123456

Once the container is running, it can be accessed in another way

docker exec -it centos /bin/sh

Copy the code

View logs about the container running

docker logs centos1

Copy the code

List the files or directories in A container that have been changed. The list will display three types of events, A added; D. C changed

docker diff centos1

Copy the code

And the initial container image project, which directory files the user or system added/modified/deleted, can be viewed.

View the processes running in the container

docker top centos1

Copy the code

Copy the files/directories in the container to the local server

docker cp centos1:/etc/passwd /tmp/
ls /tmp/passwd

Copy the code

You can also copy the container files to the server using the network IP address.

Stop the container

docker stop centos1

Copy the code

Stop all containers

docker kill $(docker ps -a -q)

Copy the code

Start the container

docker start centos1

Copy the code

Deleting a single container

docker stop centos1
docker rm centos1

Copy the code

Stop the container before deleting it.

Delete all containers

docker kill $(docker ps -a -q)
docker rm $(docker ps -a -q)

Copy the code

3. Volume concept

To be able to store (persist) data and share data between containers, Docker introduced the concept of volumes. A Volume is a specific directory for a container. Files in this directory are stored on the host rather than in the container’s file system.

A data volume is a special directory that can be used by one or more containers. It bypasses the container’s default file system and provides many useful features:

(1) Data volumes can be shared and reused between containers;

(2) Changes to data volumes take effect immediately;

(3) Data volume updates do not affect mirroring.

(4) The data volume will always exist by default, even if the container is deleted.

Note: Using a data volume is similar to mounting a directory under Linux. The files in the directory specified as the mount point in the container will be hidden and the mounted data volume can be displayed.

Create and use data volumes

mkdir -p /root/volume1
mkdir -p /root/volume2
docker run -d -v /volume1 --name='centos5'Docker - centos6.10 - hadoop - spark docker run-d -v /root/volume1:/volume1 --name='centos6'Docker - centos6.10 - hadoop - spark docker run-d -v /root/volume1:/volume1 -v /root/volume2:/volume2 --name='centos7'Docker - centos6.10 - hadoop - spark docker run-d -v /root/volume1:/volume1:ro --name='centos8'Docker - centos6.10 - hadoop - sparkCopy the code

Use the docker run command to create a container, specifying the -v flag to create a data volume and mount it to the container; Multiple data volumes can be mounted. You can set the read-only property of a volume. Can not specify the server mapping directory, by the system automatically specify the directory, through docker inspect to view the mapping path.

Go to the containers and check the /volume1 and /volume2 directories.

Data Volume Sharing

To authorize a container to access another container’s data volumes, use the -volumes-from parameter.

Data volume container

If you have some continuously updated data that needs to be shared between containers, it is best to create a data volume container.

A data volume container is a normal container that provides data volumes for other containers to mount.

(1) Create a data volume container named dbData

docker run -d-v /dbdata --name dbdata docker-centos6.10- Hadoop-sparkCopy the code

Run the –volumes-from command to mount the volumes in the dbdata container

docker run -d--volumes from dbdata --name db1 docker-centos6.10 -- Hadoop-spark Docker run-d --volumes-from dbdata --name db2 docker-centos6.10-hadoop-spark

Copy the code

This enables data sharing between containers.

Go to the containers and check the /volume1 and /volume2 directories.

4. Create an image and publish it

Save the container changes and submit a new container image

docker commit centos1 centos111

Copy the code

Submit the existing container to form a new container image. Using Docker Images, you can see the Centos111 image. With this method, you can create a new container image.

Look at mirror

Docker Images REPOSITORY TAG IMAGE ID CREATED SIZE Centos111 latest D691a75EE371 23 minutes ago 501.5 MBCopy the code

Create a container based on the new container image

docker run -d --name='centos111' centos111

Copy the code

Check the container

docker inspect centos111

Copy the code

Export and import images

To migrate an image from one machine to another, you need to export and import the image.

Machine a.

Docker save docker-centos6.10-hadoop-spark > docker-centos6.10-hadoop-spark2.tarCopy the code

or

Docker save -o docker-centos6.10-hadoop-spark docker-centos6.10-hadoop-spark2.tarCopy the code

Copy docker-Centos6.10-hadoop-spark2. tar to machine B using SCP command and other methods

Machine B

Docker load < docker centos6.10 - hadoop - spark2. TarCopy the code

or

Docker load -i docker centos6.10 - hadoop - spark2. TarCopy the code

Publish container images

Docker push centos6.8 - lamp1Copy the code

Publish containers to the network.

5. Docker network

If software testing, interface testing, automated testing, performance testing, LR script development, interview experience exchange. If you are interested, you can go to 175317069. There will be free information links in the group from time to time, which are collected and sorted out from various technical websites. If you have good learning materials, you can send them to me privately.

When Docker starts up, a virtual network interface called Docker0 is created on the host machine. It randomly selects an unused host address and subnet mask from the private address defined in RFC 1918 and assigns it to Docker0. The default is 172.18.0.1/16. A 16-bit subnet mask provides 65534 IP addresses to the container.

Docker0 is not a normal network interface, but a virtual Ethernet bridge that automatically forwards packets between other network cards bound to it, allowing containers to communicate with hosts and containers to communicate with each other.

Each time docker creates a container, it creates Peer interfaces, similar to the two ends of a tube, where one side can receive packets from the other side. Docker connects one of the peer interfaces to the container as eth0 and holds the other with a unique name like vethAQI2QT, depending on the host’s namespace. Docker creates a shared virtual subnet between the host and all Docker containers by binding all veth* interfaces to the Docker0 bridge card.

Docker NAT network

By default, docker container accesses the network through NAT. When Docker is started, a virtual network interface named Docker0 is created on the host host. Docker0 is just a virtual Ethernet bridge that automatically forwards packets between other network cards bound to it. It enables containers and hosts to communicate with each other and between containers.

The gateway address of Docker0 is 172.18.0.1, and the mask is 16 bits, providing 65534 IP addresses.

In NAT mode, the VM container can access the Internet (outside the host), but the machines outside the host cannot access the container Intranet.

Docker Bridage network

Docker containers can access the network through Bridges.

In bridge mode, the vm container can access the Internet (outside the host), and other machines outside the host can access the container Intranet.

6. docker pipework

The network function of Docker itself is relatively simple and cannot meet many complex application scenarios. Therefore, there are many open source projects to improve docker network functions, such as Pipework, Weave, Flannel and so on.

Pipework is a docker network configuration tool developed by Docker engineer Jerome Petazzoni, implemented by more than 200 lines of shell, easy to use.

The installation pipework

git clone https://github.com/jpetazzo/pipework
cp pipework/pipework /bin/

Copy the code

or

Wget [http://172.17.1.240/docker/software/pipework] (http://172.17.1.240/docker/software/pipework) chmod + x a pipework cp pipework /bin/Copy the code

Run the container

docker run -d --net='none' --name='centos9'Docker - centos6.10 - hadoop - sparkCopy the code

Configure the container network and connect to the bridge docker0. The gateway is specified by @ after the IP address.

Pipework docker0 centos9 172.18.0.100/16 @172.18.0.1Copy the code

Docker network port mapping

If the docker0 virtual network is used, the network of the container is 172.17.0.0/16. The container can access the external network through NAT. However, the Internet cannot access the Intranet. If the container uses a BR0 virtual network, the container and the server can be in the same network address segment. Containers can access the extranet; Extranets can also access container networks. For containers that use docker0 virtual network, you can use port mapping to allow external networks to access certain ports of the container.

Run the container

docker run -d -p 38022:22 --name='centos10'Docker - centos6.10 - hadoop - sparkCopy the code

Connect the container

ssh localhost -p 38022

Copy the code

On other servers, you can access containers by accessing physical servers plus ports, and you can map multiple ports at once.

Run the container

docker run -d -p 38022:22 -p 38080:80 --name='centos11'Docker - centos6.10 - hadoop - sparkCopy the code

It is implemented on the server through iptables forwarding. It is also possible to forward the entire container IP address using iptables.

4. To summarize

Because containers are process-level, they have many advantages over virtual machines.

(1) Fast start

An application in a container is directly a process in the underlying system, rather than a process in a VIRTUAL machine. So starting the container is like starting a process on the machine, rather than an operating system, which is much faster.

(2) Less resource occupation

The container occupies only needed resources and does not occupy unused resources. The VIRTUAL machine is a complete operating system, so it inevitably takes up all resources. In addition, multiple containers can share resources, and virtual machines have exclusive resources.

(3) Small size

A container contains only the components used, whereas a virtual machine is a package of the entire operating system, so a container file is much smaller than a virtual machine file.

In short, containers are a bit like lightweight virtual machines that provide a virtualized environment at a much lower cost.