Kubernetes architecture goals
Kubernetes is a production-grade infrastructure for deploying, extending, managing, and composing application containers across hosts. Kubernetes is more than just “container choreography”. Its main solution direction is to eliminate the burden of coordinating computing resources, network and storage, so that developers are container-centric and realize their own leading resource operation process. Kubernetes also provides a stable, portable platform for building custom workflows and higher levels of automation.
Kubernetes is targeted at applications that consist of multiple containers. So Kubernetes uses pods and labels to build loose or tight formations of containers for management and discovery.
What does Kubernetes include architecturally?
First it has to be extensible and pluggable. Thus, architecturally, Kubernetes is built as a collection of pluggable components and layers that can replace the schedulers, controllers, storage systems, and distribution mechanisms of daily development. If the design is good enough, there is no need to change the source code when using higher-level Paas functionality or multi-tier clustering. For this reason, he designed the API not only for the eventual users of Kubernetes, but also for tool and extension developers. Kubernetes’ API is intended as an open ecosystem of tools, automation systems and higher API levels. With this plug-in construction, Kubernetes has no such thing as an “internal” component API. All apis are visible and available, including scheduler scheduler, node controller, replication-Controller, and Kubectl controller. In order to achieve more complex functionality, it is perfectly possible to combine access to these low-level apis.
What is kubernetes trying to achieve architecturally?
The first is “run everywhere”, whether it’s a public cloud, a private cloud, a bare-metal machine or a laptop, Kubernetes has the same functionality. With this goal, Kubernetes-related components and tools are portable across the entire ecosystem as well as between development and production environments.
The second is “general purpose”, where Kubernetes has the ability to host all types of workloads. These include stateful, stateless, microservices or monolithic services, batch, cutting-edge services or traditional applications.
Then there’s “No migration at all.” Kubernetes is not only suitable for pure native cloud applications, but also for all user needs. Kubernetes focuses on microservices and cloud native, but provides mechanisms to facilitate the flexible migration of monolithic applications and legacy legacy applications. Many features of Kubernetes can be added to work sequentially, and there is the flexibility to use custom solutions to replace the built-in features.
Also “extensible”, Kubernetes adds additional functionality by providing the same interface as the built-in functionality.
And “automation”, kubernetes aims to greatly reduce the burden of manual operations. Declarative control is supported by specifying the user’s desired intents through apis, as well as advanced choreography and automated imperative control. Declarative methods are key to the self-healing and autonomous functions of the system.
Finally, “more cutting-edge.” Kubernetes wants to support non-cloud native applications, but also wants to improve cloud native and DevOPs-related technologies, such as allowing applications to participate in Kubernetes’ own management. But it also doesn’t force applications to bind to apis, because convention configuration is a better solution than API compatibility.
Kubernetes architecture details
Kubernetes contains the node agent (Kubectl) and the cluster control plane (AKA Master), and the cluster state is supported by ETCD.
Cluster Control plane (AKA Master)
The Kubernetes control plane is segmented into sets of components that can run on a single master node, be replicated to support a high availability cluster, or even run on Kubernetes itself (also known as self-hosted).
Kubernetes provides a REST API that primarily supports CRUD operations on persistent resources, which are the center of its control plane. Kubernetes’ API provides IaAS-like container-centric primitives such as PODS, services and portals, and lifecycle apis to support orchestration (self-healing, scaling, updating, termination) of common types of workloads such as ReplicaSet (simple replaceable or stateless app-Manager). Deployment (orchestrating updates to stateless applications), jobs (batch), Cron Jobs (Cron), daemons (cluster service), and StatefulSet (stateful applications). We deliberately separate service naming/discovery and load balancing from the application implementation because the latter is diverse and open.
Both the user client and the component containing the asynchronous controller interact with the same API resource, which acts as a coordination point, common intermediate representation, and shared state. Most resources contain metadata, including labels and comments, detailed required states (specs), including default values and observed states (status).
The controller continues to work, driving the actual state to the desired state, while reporting the currently observed state to the user and other controllers.
Although controllers are based at the base level to maximize fault tolerance, they typically monitor changes to related resources to minimize response delays and redundant work. This enables decentralized and detached choreographed coordination without the need for a message bus.
api server
The API server provides the Kubernetes API. It is intended to be a relatively simple server, with most of the business logic implemented in a separate component or plug-in. It mainly handles REST operations, validating them, and updating the corresponding objects in ETCD (and possibly eventually updating other storage). Note that Kubernetes deliberately does not support atomic transactions across multiple resources for many reasons.
Kubernetes would not run without this basic API mechanism, which includes:
- REST semantics, monitoring, persistence and consistency assurance, API versioning, default Settings, and validation
- Built-in license control semantics, synchronous license control hooks, and asynchronous resource initialization
- API registration and discovery
In addition, the API server acts as a gateway to the cluster. By definition, the API server must be accessed by clients outside the cluster, while nodes (and of course containers) may not be accessible. The client authenticates the API server and uses it as a fortress and proxy/tunnel for nodes and PODS (and services).
cluster state store
All persistent cluster states are stored in one instance of ETCD. This provides a reliable way to store configuration data. With Watch support, coordination components can be notified of changes quickly.
controller-Manager server
Most other cluster-level functions are currently performed by a separate process called the Controller manager. It performs life cycle functions (for example, namespace creation and life cycle, event garbage collection, terminated POD garbage collection, cascading delete garbage collection, node garbage collection) and API business logic (for example, pod scaling controlled by replication sets).
The application management and composition layer provides self-healing, scaling, application life cycle management, service discovery, routing, and service binding and provisioning.
These functions may eventually be split into different components to make them easier to extend or replace.
scheduler
Kubernetes allows users to request a cluster to run a set of containers. The scheduler component automatically selects the host on which these containers will run.
The scheduler monitors unscheduled pods and binds them to nodes through the/Binding POD child resource API, based on the availability of the requested resource, quality of service requirements, association and disassociation specifications, and other constraints.
Kubernetes supports user-provided schedulers and multiple concurrent cluster schedulers, using the shared state approach pioneered by Omega. In addition to the disadvantages of pessimistic concurrency described in the Omega paper, the two-level scheduling model hides information from the upper-level scheduler and requires the same functionality required by all upper-level schedulers to be implemented in the lower-level scheduler to ensure that their scheduling requests can be satisfied by the required available resources.
Kubernetes node
The Kubernetes node has the services necessary to run the application container and manage it from the main system.
kuberlet
The most important and prominent controller in Kubernetes is Kubelet, which is the main implementer of the Pod and node apis that drive the container execution layer. Without these apis, Kubernetes would simply be a CRUD-oriented REST application framework supported by key-value stores (perhaps the API machine would eventually be spun off as a separate project).
Kubernetes executes a separate application container as its default native execution mode, rather than processes and traditional operating system packages. Application containers are isolated not only from each other, but also from the hosts on which they execute, which is critical to separating the management of individual applications from each other and from the management of the underlying cluster physical/virtual infrastructure.
Kubernetes provides pods that can host multiple containers and volumes as their basic execution primitives to facilitate packaging a single application per container, separation of deployment-time concerns from build-time concerns, and migration from physical/virtual machines. The Pod primitive is key to gathering the main benefits of deployment on modern cloud platforms such as Kubernetes.
API licensing controls can reject a POD or add additional scheduling constraints to a POD, but Kubelet is the final arbiter of what a POD can and cannot run on a given node, not a scheduler or daemon.
Kubelet is also currently linked to the cAdvisor resource monitoring agent.
Container runtime
Each node runs a container runtime, which is responsible for downloading images and running the container.
Kubelet is not linked in the base container runtime. Instead, we define a container runtime interface to control the underlying runtime and facilitate pluggability of that layer. This separation is needed to maintain clear component boundaries, facilitate testing, and facilitate pluggability. Currently supported runtimes, whether upstream or forks, include at least Docker (for Linux and Windows), RKT, Cri-O, and Frakti.
kuber Proxy
Service abstraction provides a way to group PODS under common access policies, such as load balancing. This implementation creates a virtual IP that can be accessed by clients, and it transparently proxies to the Pods in the service. Each node runs a Kube proxy process that programs the Iptables rules to capture access to the service IP and redirect it to the correct back end. This provides a highly available load balancing solution that reduces the performance overhead by balancing client traffic from a node on the same node.
Service endpoints are found mainly through DNS.
So that’s the architecture of Kubernetes. Here’s a simplified summary:
The core functions of each kubernetes component:
- Etcd: indicates the cluster status
- Kube-apiserver: Resource operator entry, authentication and authorization control, API registration and discovery
- Kube-controller-manager: maintains cluster status
- Kube-schedluer: Resource scheduling
- Kubelet: Maintains container lifecycle, volume (CVI) and network (CNI) management
- Container Runtime: The actual running of the container (CRI)
- Kube-proxy: SVC provides service discovery and load balancing within a cluster