Write it up front
Docker Container Technology is a 5-minute tutorial on container technology. It has the following two features:
- Systematically explain the most popular container technology from the entire container ecological environment to a variety of specific technologies, from the whole to detail discussion.
- Focusing on practice and giving consideration to theory, I will lead you to learn container technology from the perspective of practical operation.
Why write this
The short answer is: container technology is hot, but the bar is high.
Container technology is the next hot technology after big data and cloud computing, and will be very popular for quite some time to come.
This is a very valuable technology for the IT industry. For IT practitioners, mastering container technology is the need of the market and an important way to improve their self-value.
For my own experience, the first few years after graduation were in J2EE application development. Then I moved to a large IT company whose products ranged from middleware to operating systems, from servers to storage, from virtualization to cloud computing.
My department is specialized in IT infrastructure implementation services. At the beginning, IT did traditional IT projects, including server configuration, two-machine HA, etc. As virtualization technology matures, my work also involves the planning and implementation of various virtualization technologies, including VMWare, KVM, and PowerVM. Later, with the rise of cloud computing, driven by the company’s business and personal interests, I began to learn and practice OpenStack. During this process, I wrote the “Playing OpenStack in 5 Minutes every Day” tutorial, which was recognized by everyone.
Now container technology represented by Docker comes, and more and more attention is paid to it, which can be clearly seen from the rising Trend of Docker searches in Google Trend (blue curve).
Each new round of technology is both an opportunity and a challenge for companies and individuals.
My personal view is that if a new technology is going to be mainstream in the future, it should be grasped early and fast. Because:
- New technology means new markets and new demands. In the early stage, there will not be many people who master this technology, but the market demand will become bigger and bigger. Therefore, there will be a seller’s market in short supply.
- Learning new skills takes time and effort. Early start makes early success
So opportunities, let’s look at challenges.
New technology often means technological breakthrough and innovation, there will be many new concepts and methods. Moreover, from the perspective of big data, cloud computing and container technology, these new technologies are all platform level, covering a wide range of technologies, including computing, network, storage, high availability, monitoring, security and many other aspects. IT is quite difficult for IT veterans to master these new technologies, let alone newcomers.
Due to my passion and dedication to technology, I began to investigate the PaaS technology stack after I mastered the IaaS technology related to OpenStack. At that time, Docker became more and more popular, so it was natural to start the study, research and practice of container-related technologies.
The process of learning container technology can be said to be a constant surprise, often amazed at the advanced container concept and the integrity and strength of the container ecological environment. Many of the challenges of traditional software development and operations can be easily solved in the container world, and I have come to understand why containers are so popular.
It’s no exaggeration to say that the container opened the door for me to another software world, which made me immersed and excited. In addition, I can’t wait to introduce what I have learned and thought about containers to more people, so that more IT engineers can benefit from container technology.
I hope this tutorial will also open the door to lowering the learning curve and systematically learning and mastering container technology.
For whom?
This tutorial is aimed at:
Software developer
It is believed that Microservice Architectur will gradually become the mainstream for developing application systems. The container is the cornerstone of this architecture. The market will need more software developers who can develop container-based applications.
IT implementation and operation engineer
Containers provide a better way to package and deploy applications. More and more applications will run as containers in development, test, and production environments. Mastering container related technology will become the core competence of implementation and operation engineers.
My own
I firmly believe that the best way to learn is to share. Writing this tutorial is also a summary of my own learning and practicing container techniques. Knowledge can only be mastered if it is written down and understood by others.
What does it include?
Docker Container Technology in 5 Minutes a Day and Docker Container Platform in 5 Minutes a Day
The following sections describe the contents of each section.
departure
Departures introduces the container ecosystem, giving you an overview of what technologies are involved in containers and how they relate to each other, and then looking at what parts of the ecosystem we will cover in this tutorial.
In order to make everyone have a perceptual understanding of the container as soon as possible, we will set up an experimental environment and run the first container to warm up for the subsequent learning.
Container technology
This is the main content of the tutorial, including “core container knowledge” and “advanced container knowledge” two parts.
Core knowledge mainly answers questions about what, why and how of containers. Focusing on how, architecture, mirroring, containers, networking, and storage will be discussed.
Advanced knowledge includes the techniques necessary to actually use containers for production, including multi-host management, cross-host networking, monitoring, data management, log management, and security management.
This will be discussed in detail in the book play Docker Container Technology in 5 Minutes a Day.
Container platform technology
As shown in the figure below, “Container platform technology” includes container choreography engines, container management platforms, and container-based PaaS. Container platform technology plays an important role in the ecological environment, which is crucial to the landing of container and its application in production.
We will discuss container choreography engines, container management platforms, and container-based PaaS in detail, learning and practicing some of the industry’s most iconic open source products, in the next book in this series, “Playing with Docker Container Platform in 5 Minutes a Day.”
How to write it?
I will continue to adopt the method of “Playing OpenStack in 5 Minutes a Day” to discuss and practice container technology from shallow to deep through a large number of experiments, and strive to achieve the following goals:
- Quick start: Get people to use containers in the most direct and efficient way possible.
- Step by step: Analyze the functions and configurations of containers in detail from easy to difficult and from shallow to deep.
- Understanding architecture: From the design principles and architecture analysis, in-depth discussion of the container architecture and operation mechanism.
- Focus on practice: based on a large number of practical operation cases, so that everyone can master real implementation skills.
In terms of content release, the wechat official account (Cloudman6) will be regularly shared on January, March and May every week. We welcome your questions and suggestions through the public account for technical exchange.
Why is it called Docker Container Technology in 5 Minutes a Day?
In order to reduce the difficulty of learning and take into account the characteristics of fragmented reading on mobile terminals, it only takes 5 minutes to finish reading the content pushed each time. Each content contains 1-3 knowledge points. That’s why I named this tutorial 5 Minutes a Day to Play With Docker Container Technology. It’s fragmented, but the tutorial is systematic, coherent and complete, just broken into pieces.
All right, today’s five minutes are a start, and next time we’re going to start playing with container technology.
Container ecosystem
Platform-level technologies such as containers often involve a wide range of knowledge, software, and solutions that beginners can easily get lost in.
So what to do?
The answer can be found in life experience. What do we do when we travel to a new city and want to get to know it?
I think most of you will turn on your phone and look at a map of the city:
- What is the approximate location and geographical shape of the city?
- Which districts or counties are they made up of?
- What are the main roads?
In the same way, we can learn about container technology from a bird’s eye view:
- What are the different levels of technology involved in a container ecosystem?
- What is the relationship between the different technologies?
- What are the core technologies and what are the ancillary technologies?
First of all, we need to have a general understanding of container technology, and then we can learn to be targeted, to be able to prioritize, to know, so that it is not easy to get lost.
Next, I will help you plan a learning route based on my experience and explore the container ecosystem together.
It’s very important to learn new technologies and get immediate feedback, so we’re going to set up our lab environment and run our first container to get a feel for what a container is.
A journey of a thousand miles begins with a single step, so let’s start by understanding the ecosystem.
Bird ‘s-eye view of the container ecosystem
Container ecosystem
When it comes to containers, Docker comes to mind.
Docker is now almost synonymous with containers. Indeed, Docker took container technology to the next level. At the same time, you need to know that there is an ecosystem around Docker. Docker is the cornerstone of this ecosystem, but a sound ecosystem is the decisive factor to ensure the real healthy development of Docker and container technology.
Broadly speaking, a container ecosystem consists of core technologies, platform technologies, and supporting technologies.
The following are introduced respectively.
Container core technology
Container core technologies are those that enable a Container to run on a host.
These technologies include container specifications, container Runtime, container management tools, container definition tools, Registry, and container OS, which are described below.
Vessel code
Containers are not just Docker, there are other containers, like RKT for CoreOS. To ensure a healthy Container ecosystem and compatibility between different containers, a group of companies including Docker, CoreOS, and Google has formed the Open Container Initiative (OCI), which aims to develop Open Container specifications.
OCI currently publishes two specifications: Runtime Spec and Image Format Spec. With these two specifications, containers developed by different organizations and vendors can run on different runtimes. This ensures container portability and interoperability.
Container runtime
Runtime is where the container actually runs. The Runtime works closely with the operating system kernel to provide an environment for the container to run.
If you are familiar with Java, you can understand the relationship between runtime and container as follows:
Java programs are containers, and the JVM is runtime. The JVM provides the running environment for Java programs. Similarly, a container can only run in runtime.
LXC, RUNC, and RKT are the three main container runtime types.
LXC is the older container Runtime on Linux. Docker also originally used LXC as runtime.
Runc is a container runtime developed by Docker, which conforms to the OCI specification and is now the default Runtime of Docker.
RKT is a container runtime developed by CoreOS that complies with the OCI specification and is therefore capable of running Docker containers.
Container Management tool
The runtime is not enough. Users need tools to manage containers. Container management tools interact with the Runtime internally and provide interfaces for users externally, such as the CLI. This is like providing Java commands that allow users to start and stop applications in addition to the JVM.
LXD is a management tool corresponding to LXC.
Runc is managed by the Docker Engine. Docker Engine consists of two parts: background Deamon and CLI. We usually refer to Docker, usually referring to the Docker engine.
The RKT management tool is the RKT CLI.
Container definition tool
Container definition tools allow users to define the contents and properties of containers so that containers can be saved, shared, and rebuilt.
Docker Image is a template for a Docker container. Runtime creates a container based on docker Image.
A dockerfile is a text file containing several commands that can be used to create a Docker image.
ACI (App Container Image) is similar to Docker Image except that it is the Image format of RKT Container developed by CoreOS.
Registry
Containers are created from images, and there needs to be a repository for storing images, called Registry.
Enterprises can use Docker Registry to build private Registry.
Docker Hub (hub.docker.com) is a hosted Registry provided by Docker for the public, on which there are many ready-made images, providing great convenience for Docker users.
Quay.io (quay.io/) is another public hosted Registry that provides services similar to Docker Hub.
Container OS
Thanks to the container Runtime, almost all Linux, MAC OS, and Windows can run containers. But that didn’t stop the advent of container OS.
Container OS is an operating system dedicated to running containers. Container OS is usually smaller and starts faster than regular OS. Because the OS is customized for containers, they usually run containers more efficiently.
There are a number of container OSS out there, with CoreOS, Atomic and Ubuntu Core being some of the best examples.
Container platform technology
The container core technology enables containers to run on a single host. Container platform technology enables containers to run as clusters in a distributed environment.
Container platform technologies include container choreography engines, container management platforms, and container-based PaaS.
Container choreography engine
Container-based applications typically use microservices architecture. In this architecture, applications are divided into different components and run as services in their own containers, providing services through apis. To ensure high availability of the application, each component may run multiple identical containers. These containers form clusters that are dynamically created, migrated, and destroyed based on business needs.
As you can see, such an application system based on microservices architecture is actually a dynamic and scalable system. This placed new demands on our deployment environment, and we needed an efficient way to manage container clusters. And that’s what the container Choreography engine does.
So-called orchestration typically includes container management, scheduling, cluster definition, and service discovery. Through the container orchestration engine, containers are organically composed into microservice applications to fulfill business requirements.
Docker Swarm is a container choreography engine developed by Docker.
Kubernetes is a Google-led open source container choreography engine that supports both Docker and CoreOS containers.
Mesos is a generic cluster resource scheduling platform that provides container choreography engine functionality along with Marathon.
The above three are the current mainstream container choreography engines.
Container Management Platform
The Container Management platform is a more general platform built on top of the container choreography engine. Typically, container management platforms can support multiple orchestration engines, abstracting the low-level implementation details of orchestration engines and providing users with more convenient functions, such as Application Catalog and one-click application deployment.
Rancher and ContainerShip are typical examples of container management platforms.
Container-based PaaS
Container-based PaaS provides a platform for microservice application developers and companies to develop, deploy, and manage applications, enabling users to focus on application development without having to worry about the underlying infrastructure.
Deis, Flynn, and Dokku are all representatives of the open source container PaaS.
Container support technology
The following technologies are used to support container-based infrastructure.
Container network
The emergence of containers makes network topology more dynamic and complex. Users need specialized solutions to manage the connectivity and isolation between containers and containers and between containers and other entities.
Docker Network is the native network solution of Docker. In addition, third-party open source solutions such as Flannel, Weave and Calico are available. Different schemes have different design and implementation methods, each with its own advantages and characteristics, and need to be selected according to actual needs.
Service discovery
Dynamic change is a major feature of microservice applications. When the load increases, the cluster automatically creates new containers. The load is reduced and excess containers are destroyed. Containers also migrate between hosts based on their resource usage, and the IP address and port of the container change accordingly.
In this dynamic environment, there must be a mechanism for clients to know how to access the services provided by the container. This is what the service discovery technique does.
Service discovery saves the latest information about all microservices in a container cluster, such as IP addresses and ports, and provides apis for service query.
Etcd, Consul, and ZooKeeper are typical solutions for service discovery.
monitoring
Monitoring is critical to infrastructure, and the dynamic nature of containers presents additional challenges. A number of monitoring tools and solutions have emerged for container environments.
Docker ps/ TOP /stats is a native docker command line monitoring tool. In addition to the command line, Docker also provides a STATS API that allows users to retrieve container status information via HTTP requests.
Sysdig, cAdvisor/Heapster, and Weave Scope are other open source container monitoring solutions.
Data management
Containers are often migrated between hosts, and ensuring that persistent data is also dynamically migrated is a capability provided by data management tools such as Rex-Ray.
Log management
Logs provide an important basis for troubleshooting and event management.
Docker logs is a native docker logging tool. Logspout provides routing for logs. It collects logs from different containers and forwards them to other tools for post-processing.
security
For young containers, security has always been a point of contention in the industry.
OpenSCAP can scan the container image for potential vulnerabilities.
The scope of knowledge covered by this tutorial
Earlier we had a bird ‘s-eye view of the entire container ecosystem and gained a comprehensive understanding of the technical architecture involved in containers. What will we cover in this series?
Covers 91.6% of the container ecosystem!
Run the first container
To give you a sense of what containers are, we’re going to get a container up and running as soon as possible. First we need to set up the experimental environment.
Environmental choice
The container requires a management tool, runtime, and operating system. Our options are as follows:
- Docker Engine because Docker is the most popular and widely used.
- Run-time runc Default Runtime for Docker
- Operating system – Ubuntu Although there are container OS such as CoreOS, considering that we are in the early stages of learning, it is more appropriate to choose the familiar operating system. It is more beneficial to wait until you have a solid knowledge of container basics before using container OS.
Install the Docker
In this section, we will install Docker in the Ubuntu 16.04 virtual machine. Because the installation process requires Internet access, the VIRTUAL machine must be able to access the Internet.
Docker supports almost all Linux distributions, as well as Mac and Windows. The operating system installation method can access: docs.docker.com/engine/inst…
Docker is divided into open source and free CE (Community Edition) version and Enterprise Edition (EE) version. Below we will follow the documentation to install the Docker CE version on Ubuntu 16.04.
Configure apt sources for Docker
1. Install the package and allow apt command HTTPS to access Docker source.
2. Add the official Docker GPG key
3. Add the Docker source to /etc/apt/sources.list
Install the Docker
Run the first container
When the environment is ready, run the first container immediately, executing the command:
The process can be simply described as:
- Download the HTTPD image from the Docker Hub. The Apache HTTP Server has been installed in the image.
- Start the HTTPD container and map port 80 of the container to port 80 of host.
Now we can verify that the container works using the browser. Enter http://[your Ubuntu Host IP] in your browser.
You can access the container’s HTTP service. The first container ran successfully! We easily had a WEB server. As we learn more, we will see more value in container technology.
summary
We have completed the first part of the tutorial.
We learned about the container ecosystem, and will continue to learn about most of the technologies in the ecosystem. We configured our experimental environment on Ubuntu 16.04 and successfully ran our first container, HTTPD.
The container door is open. Let’s explore.