The author | liu’s award

background

“Cloud native technologies enable organizations to build and run applications that scale flexibly in new and dynamic environments such as public, private and hybrid clouds. Cloud native technologies include containers, service grids, microservices, immutable infrastructure, and declarative apis.”

Container technology can’t avoid cloud native, and cloud native can’t avoid container technology. Container technology and cloud native are a pair of double spirals. Container technology gives birth to cloud native thought, and cloud native ecology promotes the development of container technology. From the birth of Docker (Container) technology in 2013 to the establishment of CNCF, a heavyweight alliance in the cloud native field, in 2015, it is not a historical coincidence but a historical necessity. As one of the key technologies of cloud native, container has been one of the focuses of the industry since its birth in 2013. To understand the historical background of container technology and the birth of cloud native, use a widely cited advanced diagram of cloud native container technology.

Let’s have a look at the history of the development of container technology table, first intuitively feel this hot land in full swing!

In 1979, Unix V7 supported chroot to build a separate virtual file system view for applications. In 1999, FreeBSD 4.0 supported Jail, the first commercialized OS virtualization technology. In 2004, Solaris 10 supported Solaris Zone, the second commercialized OS virtualization technology. In 2005, OpenVZ was released, the very important pioneer of Linux OS virtualization technology. From 2004 to 2007, Google used OS virtualization technology such as Cgroups on a large scale. In 2006, Google opened source the Process Container technology for internal use, later renamed cGroup. In 2008, Cgroups entered the Linux kernel mainline. In 2008, the LXC (Linux Container) project had a prototype of a Linux Container. In 2011, CloudFoundry developed Warden systems, a complete container management system prototype. In 2013, Google opened its internal container system with Let Me Contain That For You (LMCTFY). In 2013, the Docker project was officially released, and Linux container technology gradually took the world by storm. In 2014, the Kubernetes project was officially released, and container technology began to go hand in hand with choreography systems. In 2015, CNCF was co-founded by Google, Redhat, Microsoft, and some large cloud vendors, and the cloud native wave started. From 2016 to 2017, container ecology began to be modular and standardized. CNCF accepted Containerd, RKT projects, OCI released 1.0, and CRI/CNI was widely supported. From 2017 to 2018, container services were commercialized. AWS ECS, Google EKS, Alibaba ACK/ASK/ECI, Huawei CCI, Oracle Container Engine for Kubernetes; VMware, Redhat, and Rancher began offering kubernetes-based business services. From 2017 to 2019, container engine technology developed rapidly and new technologies emerged continuously. Kata Containers community was established at the end of 2017, Google open source gVisor code in May 2018, AWS open Source Firecracker in November 2018, Ali Cloud released security Sandbox 1.0. 2020-202X, container engine technology upgrade, Kata Containers start 2.0 architecture, Ali Cloud release sandbox container 2.0….

The development history of finishing container technology in the past 20 years can be roughly divided into four historical stages, which will be introduced in detail below.

Technological embryonic stage

One of the core issues container technology needs to address is runtime environment isolation.

Run time environment isolation for containers. The goal is to create an undifferentiated run time environment for containers to run their images anywhere at any time. Due to The preaching of Docker, people used to think that the runtime environment isolation of containers is OS virtualization, or container is equal to namespace + cgroup + security protection mechanism. I don’t quite agree with this view, this is just a historical period, one container runtime implementation technology, and there are many other possible technical solutions to implement container runtime environments. So, back to the source of the requirement: the container needs runtime isolation techniques to ensure that the container runs as expected. The component that implements container isolation is conventionally referred to as the container runtime.

From another point of view, container isolation technology solves the problem of resource supply. Why is container isolation needed to solve the resource provisioning problem? Success is also xiao, defeat is also xiao! Moore’s Law is so powerful that we have more and more computing resources at our disposal. 10 years ago, the typical specification of a minicomputer was a 32-channel 8-core CPU. Now, the computing capacity of a 4-channel PC server exceeds that of a minicomputer server 10 years ago. The typical use of minicomputers is to divide the whole machine into multiple partitions for use. Observing the current cloud service hardware development trend, more and more familiar with the feeling that we are in the minicomputer related technology “military to civilian”. Today we have a PC server with very powerful computing power comparable to minicomputers of ten years ago. Coincidentally, the typical usage of PC servers today is similar to minicomputers of ten years ago, cut into 1-8Vcpu virtual machines/containers.

Why do people always get used to dividing a large server into smaller partitions instead of developing software that can take full advantage of the computing power of a large server? Personally, there are two constraints behind it:

• The problem to be solved has a finite inherent degree of parallelism. With the increasing popularity of multi-core multi-processor systems, the IT industry began to upgrade from serial programming to parallel programming in 2004. At the beginning, the effect of parallel transformation for specific industry applications is very obvious, but later, it is found that with the increase of parallelism, the transformation cost is increasing and the benefits are decreasing. Constrained by Amdar’s law, the benefits of solving a particular problem will gradually decrease when the degree of parallelism exceeds a certain critical point. So it is not economical to increase system parallelism.

• Human intelligence is limited. Limited by human intelligence, the more complex the system, the higher the parallelism, the more prone to software failure, software maintenance costs grow exponentially. Therefore, from the perspective of software engineering, people also tend to interface, modular, unitary software architecture design, as far as possible to control the complexity of software, reduce the cost of engineering.

As a rule of thumb, 1-8 CPU parallelism is the comfort zone of software engineering and one of the driving factors behind technologies such as containerization and microservices.

A bit off topic… In short, resource supply based on isolation is not pseudo demand. The need to isolate the environment in which software runs has been around since the dawn of operating systems. Multi-task time-sharing operating system and process virtual address are designed to solve the resource sharing problem of multiple tasks running on the same host, so that each process thinks it owns the host. Of course, process isolation alone is not enough. Looking at the current resource isolation technologies, we can roughly divide them into five categories:

• Process isolation. The OS uses processes as an abstraction of the running process of tasks. Processes have independent address space and execution context. In essence, the OS virtualizes CPU and memory for processes. However, processes also share file system, network protocol stack, IPC communication space and other resources, and the interference between processes is very serious because of resource competition. This level of isolation is suitable for running different programs of a single user on different hosts. Users can ensure resource allocation and security protection by means of system management.

• OS virtualization. OS isolation, also known as OS Virtualization, is an advanced version of process isolation. Process isolation implements a separate address space and CPU context for each process, while OS isolation constructs an independent OS environment for each group of process instances to further virtualize OS resources such as file system, network protocol stack, IPC communication space, process ID, user ID, etc. OS isolation needs to address three core issues: separate views, access control, and security protection. Chroot and Linux Namespace provide an independent view for a process group, cgroup provides access control for a process group, and Capabilities, Apparmor, seccomp and other mechanisms provide security protection. Of course, OS is a very complex and dynamic system, although the OS split-body makes the process group feel that there is an independent OS, but the real implementation is still an OS instance, so the overall protection ability is still a worry.

• Hardware virtualization. OS virtualization is the realization of OS kernel, and hardware virtualization is the realization of hardware device. The advent of hardware virtualization enables multiple operating systems to run simultaneously on the same physical server, each thinking it is managing an entire server. Different operating systems are strictly isolated from each other. The hardware access of Guest operating systems is strictly regulated by the VMM or CPU. Hardware virtualization has good security and isolation, but the disadvantage is that the introduction of a hardware virtualization layer leads to additional performance overhead.

• Hardware partitioning. This is the resource separation technology adopted by the traditional minicomputer system, which is to completely separate a large server into multiple hardware units from the hardware or firmware layer, thus achieving the highest level of security and isolation. However, as a declining technology route, minicompus has obvious disadvantages: inflexible granularity of resource separation, high system cost and limited system scalability.

• Language runtime isolation. For Java, NodeJS, and other managed languages that require Language Runtime, we also have the option of implementing isolation in the Language Runtime. For cloud native services such as functional computing, implementing an isolation mechanism at language runtime is theoretically the best path. However, there are many practical constraints on the implementation of this route, so most of the current function calculation or the use of container/VM technology to achieve isolation.

The biggest technical contribution to OS virtualization comes from Google.

From 2003 to 2006, Google successively released the “Troika”, which laid the framework of big data computing and then further created the concept of “cloud”. It was also during this period that process isolation technology entered a more advanced stage. Under the cloud computing framework proposed by Google, isolated process is not an isolated Jail but won’t give itself, they need more like a lightweight container, besides can be isolated, but also can be control and allocate, so as to realize cross-platform, high availability of distributed application scenarios, such as extensible features.

In 2006, Google launched Process Containers, which are used to constrain, account for, and isolate resources (CPU, memory, disk I/O, network, etc.) for a set of processes. Due to its more mature technology, Process Container entered the Linux kernel backbone in 2006 and was officially renamed Cgroups the following year, marking the beginning of the rethinking and implementation of the concept of “Container” in the Linux camp.

In 2008, a complete Container technology, LXC (Linux Container), appeared in the Linux kernel by combining the resource management capabilities of Cgroups with the view isolation capabilities of Linux namespaces, This is the basis for the container technology that is widely used today.

In general, before docker was invented in 2013, Linux operating system had basically solved the operating environment isolation technology, one of the core technologies of containers, or Linux OS virtualization technology had basically taken shape. While container operating environment isolation technology is almost in place, we still have to wait for another key technology to take off.

Technological explosion

Before 2013, the cloud computing industry was worried about the proper opening position of cloud native. Platform as a Service (PaaS) looks like a promising direction. In 2006, Fotango released Zimi service, which can be said to be the originator of PaaS industry, with typical cloud native features such as pay-per-use, Serverless, API-based configuration and services. Google launched GAE in 2008; In 2011, Pivotal released Cloud Foundry. These early PaaS platforms made very useful explorations and promoted the healthy development of the cloud computing ecosystem, but these early explorations did not form a big industry trend, but were limited to a few specific areas. Until Docker open source, everyone woke up, it was not the wrong direction, but the means of application distribution and delivery.

The real core innovation of Docker is Docker Image, a new application packaging, distribution and operation mechanism. Container images package the application runtime environment, including code, dependency libraries, tools, resource files, and meta-information, into an immutable package independent of the operating system distribution.

• Container images package the environment on which the entire container runs to avoid relying on the operating system of the server running the container, thus achieving “Build once, run anywhere”.

• Once the container image is built, it becomes read Only and part of the immutable infrastructure.

• The kernel addresses the container process’s dependence on libraries, tools, and configurations contained in the operating system, but the container image does not address the container process’s special dependence on kernel features. This one often jumps into this pit when actually using containers:

Docker’s tagline is “Build, Ship and Run Any App, Anywhere”. We have understood the mechanism that Docker uses container Image to solve “Run Anywhere”, so how is “Run Any App” implemented? It also relies on the Container Image. Users can package any environment that the container process depends on without having to adapt the application to the environment defined by the PaaS. The “Run Any App” broke through the PaaS industry’s dilemma and created endless possibilities, driving the growth of cloud native. Let’s pay tribute to this great idea!

At this point, the container technology architecture has solved the two core problems: how to distribute software and how to run software. In 2014, my former boss said to me, “Why don’t you stop working on the Linux kernel and watch Docker?” After a little research, I gave my former boss a simple and clear answer, “Nothing but packing tools!” Because of this answer, cloud native opened a door for me quietly closed. Looking back on the history, I sometimes regret that I was too young to see the power of container technology + choreography system, let alone realize the coming breath of cloud native!

Docker, as a stand-alone software package, release and operation system, has great value. However, the maximum value of this innovative technology cannot be brought into play only by limiting Docker technology to a single machine. Naturally, the industry hopes to build a cloud-based cluster system based on Docker technology to arrange and manage business containers in the next step.

When it comes to container orchestration systems, we need to start with Google. In 2008, Google launched its first App hosting platform, GAE (Google App Engine), based on LXC, offering the development platform as a service for the first time.

GAE is a distributed platform service that provides users with a development environment, server platform, and hardware resources through Virtualization technology. Users can customize their applications on the platform and distribute them through Google’s servers and Internet resources. Google uses Borg (a predecessor of Kubernetes), a tool for orchestrating and scheduling LXCS, in GAE. Borg is a large-scale cluster management system used internally by Google, capable of hosting 100,000 tasks, thousands of different applications, and managing tens of thousands of machines simultaneously. Borg uses permission management, resource sharing, and performance isolation to achieve high resource utilization. It supports high availability applications, reduces the probability of failure through scheduling policies, and provides task description language, real-time task monitoring, and analysis tools. If isolated containers are containers, Borg can be said to be the earliest port system, and LXC + Borg is the earliest container arrangement framework.

After docker was launched in 2013, it quickly swept the world. In 2014, Google created an open source project Kubernetes (K8s for short) based on Borg system for internal use, which is used to solve the problems of container deployment, operation and management of large-scale clusters. Kubernetes has added a new layer of management abstraction Pod on top of the container to make better use of the container for functional module segmentation of applications. Born out of Borg, K8s has quickly become the industry standard, a must-have tool for container choreography, thanks to Google’s massive cluster infrastructure.

In response, Docker company also added Docker Swarm, a container cluster management system, and supporting Docker Machine, Docker Compose and other tools in Docker 1.12 released in 2015. To build a perfect container orchestration system, and Kubernetes to compete head-on. Since then, the container community has been divided into two camps: Google and Docker; The container orchestration system is Kubernetes, Docker Swarm, and Apache Mesos. The competition of various factions intensified, and gradually extended to the establishment of industry standards. Let us recall the history of this section of the turbulent rivers and lakes!

After Docker launched Docker in 2013, CoreOS came into being. CoreOS is a lightweight Operating system based on the Linux kernel, designed for the infrastructure construction of computer clusters in the era of cloud computing, with automation, easy deployment, security, reliability, scale and other features. It had a very visible label at the time: an operating system designed for containers. With the help of Docker, CoreOS quickly became popular in the cloud computing field, and Docker + CoreOS became the golden partner of container deployment in the industry. At the same time, CoreOS has also made great contributions to the promotion and community building of Docker. The growing Docker, however, seems to have bigger ambitions. Docker, unwilling to just do “a simple basic unit”, developed a series of relevant container components by itself. Meanwhile, it acquired some container technology companies and began to build its own container ecological platform. Obviously, this is a direct competition for CoreOS. In late 2014, CoreOS launched its own container engine Rocket, or RKT, in an attempt to compete with Docker. RKT is similar to Docker in that it can help developers package applications and dependencies into portable containers, simplifying deployment tasks such as setting up environments. The difference between RKT and Docker lies in that RKT does not have the “friendly functions” that Docker provides for enterprise users, such as cloud service acceleration tools and cluster systems. What RKT wants to do, on the other hand, is a purer industry standard.

The above material is referred to as “From virtualization to cloud native: The History of container Technology”. Why do you refer to this material in paragraphs? The most critical context is the business competition between technology companies, and the search for balance between competition and cooperation leads to the birth of standard specifications, and the birth of standard specifications is the most important cornerstone of the whole cloud ecosystem.

The result of competing container engines (Docker vs Rocket) and container choreography (Docker swarm vs Kubernetes vs Apache Mesos) is that everyone sits down to talk about interface standards. In June 2015, Docker took the lead in establishing OCI, which aims to “develop and maintain OCI Specifications for container image format and container runtime”. Its core outputs are OCI Runtime Spec (Container Runtime Spec), OCI Image Spec (Image Format specification), and OCI Distribution Spec. So the OCI organization solves the problem of building, distributing, and running containers.

A month later, Google spearheaded the launch of the Cloud Native Computing Foundation (CNCF), which aims to “build Cloud Native Computing — an infrastructure-centric architecture around the dynamic scheduling of microservices, containers, and applications — and promote its widespread adoption”. So the CNCF organization solves application management and container choreography problems. These two foundations around containers play a very important role in the development of the cloud’s native ecosystem. Rather than competing, they complement each other and jointly set a set of industry standards. The establishment of these industry standards has injected infinite vitality into various industries, and the concrete implementation of standards based on interfaces has emerged continuously, presenting a scene of a hundred flowers blooming together.

The most important containers-related specifications include CRI, CNI, CSI, OCI Distribution Spec, OCI Image Spec, OCI Runtime Spec, and Shimv2. The CRI, OCI Image Spec, OCI Runtime and Shimv2 specifications are closely related to ali Cloud sandbox containers.

So, thanks to this golden age of cloud-native, container technology, a group of creative people came together to create these key specifications that opened up the possibility for vendors to implement their own unique and compliant technologies.

Commercial exploration period

After 5 years of technical development, the container technology is basically mature, and the cloud native system is in its infancy. From 2017, major cloud vendors began to experiment with container services and advanced cloud native services. From the current business forms, container-related public cloud services can be roughly divided into three forms:

1. Generic container Choreography service. Before the result of The Three Kingdoms killing of container choreography system, the container choreography service system is constructed based on multi-bet strategy. AWS is a self-developed programming system, Azure ACS supports Docker Swarm, DC/OS and Kubernetes at the same time, Ali Cloud ACS supports Docker Swarm and Kubernetes. Google and Huawei firmly support Kubernetes and do not offer container services that support other container orchestration systems. With Kubernetes dominating the container layout, this line of container services is gradually declining, Azure is directly terminated ACS services earlier this year.

2. Kubernetes Container Choreography service. Google is certainly the first to test the water Kubernetes container choreography service, but also earlier to carry out K8s container choreography service. In 2017, with the major factories in the CNCF negotiation table reached a Kubernetes compatibility certification process, Kubernetes choreography service market ushered in a big outbreak, to 2018, the K8s container choreography services of the major cloud manufacturers will be complete in place.

3. Serverless Container instance service. Starting in 2017, the industry began to experiment with Serverless Container instance services, freeing users from the heavy task of maintaining container infrastructure to focus on the business itself. The core goal of Google Cloud Run is to support Knative, so many constraints are attached to its usage form.

As can be seen from the figure above, public cloud container services have been explored since 2014, especially after the two years of false start in 2017-2018, the basic business form of container services has been relatively clear. The development trend can be summarized as follows:

• Industry acceptance of containerization has been high, the popularity of containerization is also increasing year by year.

• Container choreography has become a dominant system, with K8s becoming the de facto king of container choreography.

• Serverless Container instance services are popular in the market with a growing customer base.

• In the long run, managed container Choreography service and Serverless container instance service will co-exist to meet customer demand for service cost and flexibility.

During the exploration of commercial model, the core goal is to quickly guide and confirm customer needs by trial and error, and build suitable product form. The construction idea of product technical architecture during this period is to use existing mature technology to quickly build commercial form, and keep moving forward in the process of trial and error.

Among them, the typical architecture of container choreography hosting service node level is to use IaaS system to generate VM, and then deploy kubelet, Docker, Containerd, runC and other container service components in VM, which is the technical architecture of VM + container. A VM can host multiple container/Pod instances for the same user. The node-level architecture of the Serverless Container instance service is more straightforward, with only one container/Pod instance deployed within a VM to achieve Serverless. This short and quick approach has played a very important role in the exploration of commercial models, but its historical limitations in terms of flexibility, deployment density, and resource cost are still very large.

Commercial expansion period

By 2019, the business pattern and market trend of container service have been obvious. The industry as a whole has entered the stage of business expansion, attracting more customer groups through external publicity and improving product technology competitiveness through internal efforts. The industry is undergoing technological upgrading from “having” to “excellent”. The industry is going through this historical phase of technological upgrading, so we can only talk about trends and predictions. The focus of this series is on container isolation technology, so instead of talking about business expansion and container choreography, we will focus on container engine technology trends. So far, we can roughly divide container engine technology into two generations:

1. The Container on the VM. In other words, container services are constructed by IaaS + PaaS architecture based on layered design, which is a typical architecture in the commercial exploration stage. VMS are produced based on the mature IaaS infrastructure of major cloud vendors, and container service components are deployed on VMS. This architecture uses a lift and shift strategy, shifting the responsibility for the operation and maintenance of container services from the user to the cloud vendor. Using the same software components as users only shifts the responsibility of operation and maintenance, which helps guide customers to gradually move to the cloud and accept the cloud native thinking. However, the services provided by cloud vendors in this period are simple operation and maintenance hosting, which have no obvious technical advantages over the user-built container services. Even the user experience of the part restricted by multi-tenant isolation is inferior to that of the user-built container services.

2. Container with Hardware Virtualization. With the layered design architecture of Container on VM, it is difficult for cloud vendors to build unique technological advantages. For Serverless container instance services, the service delivery plane has moved from the hardware interface of IaaS to OS Syscall, so do not follow the hierarchical design of VM + containers. We need to start from the source of requirements. Container services need a container engine with high performance, strong isolation, sufficient security and low cost. How to build such a container engine is one of the current research and development hotspots in the industry. Please pay attention to the following series of articles for specific technical ideas.

summary

To sum up, container service ecology has roughly gone through four stages, solving or trying to solve different problems respectively:

1. Technology embryonic stage: solved the isolation problem of container operating environment
2. Technical burst: solved software distribution and container choreography problems
3. Commercial exploration: The commercial service form of the container is confirmed
4. Commercial expansion period: expand applicable scenarios and deployment scale, and enhance product competitiveness through technological innovation

gossip

So much history, let’s talk about docker, the company and docker technology.

On November 13, 2019, Mirantis, a private cloud infrastructure company, announced in its official blog that it has acquired Docker’s enterprise business, including taking over more than 700 of its customers, marking the complete failure of Docker’s commercial exploration that began in 2013. For those who do not know the history of container development, this result is hard to understand. Docker is the pioneer of container boom, and container is the initiator of this round of cloud computing technology evolution. Why is it that it is on the cusp, but still cannot fly?

In fact, Docker’s fate today was decided four years ago.

When Kubernetes was released in 2014, it quickly attracted a number of heavyweights, including Redhat, and launched Kubernetes 1.0 a year later to support full commercial use. In response, Docker led the establishment of OCI, aiming to develop an open standard for container image format and runtime, so as to continue to occupy the discourse power of container ecology. However, after the establishment of CNCF in July 2015, the company opened up a new battlefield by quickly overtaking cars, focusing on container arrangement and application management. Then in 2016, Kubernetes community developed the container runtime interface specification CRI. As long as the container runtime of this CRI specification can be ecological docking with K8s, which triggered the research and development boom of container engine. Cri-containerd, Cri-O, Frakti and other engines continue to emerge. With the addition of the original RKT engine, Docker has become a member of the general population of container engines. Where it came from, Docker is back to its original form, a standalone software package that largely missed the cloud native wave.

However, docker, a client-side container management tool (UI), will still exist for a long time, after all, there are strong user groups. However, in the technology stack of cloud service vendors, Docker’s position will become weaker and weaker, gradually replaced by k8S-specific container engine. Although docker still has a strong mass base, the spark has been ignited and the trend has emerged. It is only a matter of time!

reference

• Cloud Native and Container Technology Landscape
• A Brief History of Containers: From the 1970s Till Now
• From Virtualization to Cloud Native: The History of Container Technology
• Why 2019 is the Age of Container Technology?
• Alibaba’s Practice and Exploration in Safe Containers
• Practice of Secure Containers in Edge Computing Scenarios
• Vision 2020: Traditional Containers are Dead, Secure Containers will Become cloud native standard

Course recommended

Last year, CNCF and AliYun jointly released “Cloud Native Technology Open Course”, which has become a “required course” for Kubernetes developers. Today, Ali Cloud again gathered a number of technical experts with rich experience in cloud native practice, officially launched the cloud native technology Practice Open Class. The course content is advanced from simple to profound, focusing on “landing practice”. It also creates real and operable experimental scenes for learners, which is convenient to verify learning results and lays a solid foundation for subsequent practical application. Click on the links to view the course: developer.aliyun.com/learning/ro…

“Alibaba Cloud originator focuses on micro-service, Serverless, container, Service Mesh and other technical fields, focuses on the trend of cloud native popular technology, large-scale implementation of cloud native practice, and becomes the public account that most understands cloud native developers.”