The author

Xu Di, Tencent cloud container technology expert.

Ru Yingzhe, senior product manager of Tencent Cloud.

Abstract

Over the past few years, the cloud computing space has undergone a number of dramatic changes, with more and more organizations now deploying applications on multiple infrastructure platforms, both on-premises and on the cloud. These platforms may be two public cloud service providers, or two private clouds, or multi-geographic edge clouds.

As a result, the traditional technology architecture and management approach have increased complexity and risks, making it difficult to meet the challenges of application service deployment and governance across multiple platforms. Therefore, the Clusternet project, which represents the latest concept in the industry, came into being.

Open source Clusternet project

Clusternet (Cluster Internet) is a cloud native project of Tencent cloud open source, which has both multi-cluster management and cross-cluster application choreography, making multi-cluster management as simple as accessing the Internet. Regardless of whether your Kubernetes cluster is running on public, private, hybrid or edge clouds, you will have a consistent management/access experience to centrally deploy and coordinate multiple clusters of applications and services using the K8s API.

Clusternet uses the Addon plug-in for easy installation, maintenance and integration, making it easy to manage millions of Kubernetes clusters, making cloud computing as ubiquitous as the Internet, free and convenient.

Based on the Clusternet project, a multi-cloud and multi-cluster management platform is implemented to provide users with cross-cloud, cross-cluster, and cross-region /zone distributed container services, which can better meet the requirements of various scenarios.

Low multi-tenant

● High availability and Dr

● Multi-cloud multi-center service

● Edge calculation

Architecture concept

Clusternet is designed for the future of cloud native, multi-cloud and multi-cluster. Its leading architecture enables users to manage clusters and applications from a global perspective, enabling users to easily distribute their services to the world and run them everywhere at once.

Cluster Management Cluster

Clusternet follows the concept of cloud native, and all management components are flexibly deployed in Kubernetes cluster through Addon, realizing K8S-on-K8S cluster management cluster.

Compared to other multi-cluster management solutions in the community, there is no need to learn and maintain complex platform software and systems, and no need to maintain additional control plane components and ETCD. Clusternet Cluster management reuses existing Kubernetes clusters and ports, and Aggregated APIServer (AA), which Aggregated information about information, greatly reducing o&M complexity and resource consumption.

Clusternet is architecturally lightweight and streamlined, comprising:

1, clusternet – the hub

The clusternet-Hub component is deployed and Aggregated in the parent cluster, and Aggregated APIServer (AA).

Main responsibilities:

● Approve registration requests for subgroups and create dedicated resources for them, such as namespaces, Service Accounts, and RBAC rules.

● Maintain long links between the parent cluster and subgroups

● Provide Kubernetes-style REST APIS for accessing subsets, especially edge subsets, and support mutual access between subsets of services

● Support multi-cluster application distribution and governance

2, clusternet – agent

Component Clusternet-Agent is deployed in each subset.

Main responsibilities:

● Automatically registers the current cluster with the parent cluster as a subset.

● Establish a TCP full-duplex Websocket security tunnel with the parent cluster. FeatureGate SocketConnection is used to determine whether to establish a security tunnel. Turning this feature off means that the parent cluster can access subsets directly.

● Reports cluster heartbeat information, including Kubernetes version, platform information, HealthZ/ReadyZ/LiveZ health status, cluster capacity, and node status

Based on the lightweight and flexible Clusternet architecture, clusterNet-Hub supports self-registration of the parent cluster. Clusternet-hub can publish applications to its own cluster. This method maximizes the resources of the parent cluster, rapidly expands existing clusters, and easily manages massive public clouds, private clouds, and edge clusters.

Multi-cluster distribution of any resource type

Clusternet support to different cluster distribution and management of various applications including native Kubernetes all kinds of resources (Deployment/StatefulSet/ConfigMap/Secret, etc.), all kinds of CRD resources, And the HelmChart app and so on.

The following figure shows Clusternet’s multi-cluster application distribution model, in which the green modules need to be created by users, and the purple modules are resource objects for circulation within Clusternet. Clusternet provides the Kubectl plug-in, which allows you to create resources by using the “Kubectl Clusternet apply” command.

Clusternet resource distribution model USES the design of loosely coupled, users do not need to change or to rewrite the existing resource objects, only need to define additional distribution strategy (Subscription) and differential configurations (Localization/Globalization) can realize the application of cluster distribution.

● Compatible with any resource type

Fully compatible with K8s standard resources such as Deployment, StatefulSet, DaemonSet, and various custom CRDS, no need to learn the complex CRD definition of multi-cluster resources.

Low Subscription

Define the resources you want to install into the cluster. For each matching cluster, a corresponding Base object is created in its own namespace

Low Localization and Globalization

When distributing multi-cluster applications, Localization and Globalization differentiation strategies can also be used for differential configuration among different clusters. Localization describes the namespace-scoped differentiated configuration policy, and Globalization describes the cluster-scoped differentiated configuration policy. Both support management and configuration based on Priority. This feature is very useful for scenarios such as multi-cluster oriented blue-green releases, Canary releases, and version upgrades.

Low Base & Description

Clusternet automatically generates Base and Description objects to observe and track the distribution of application resources. Description refers to the Base object rendered by Localization and Globalization differentiation configuration, that is, the object definition that describes the final deployment to the target subset.

Cloud native ecology is fully compatible

Clusternet has been fully compatible with the cloud native ecosystem since its inception.

Clusternet provides multi-platform version support, including common Linux platforms such as x86, ARM64, ARMV6, PPC64LE, s390X.

Clusternet also provides kubectl plug-in (Kubectl-ClusterNet), directly through the Kubectl command line operation, without the introduction of additional CLI tools, also without switching kubeconfig context, It can create, update, edit, delete, expand and shrink multiple cluster resources, which is convenient and compatible with users’ existing CICD publishing system.

Clusternet also provides client-Go support, which is fully compatible with all versions of Client-Go. Clusternet provides Clusternet multi-cluster docking capability in a non-intrusive manner through wrapperFunc. Developers can refer to the documentation guide to implement multiple clustering and application management capabilities in your system in as little as three lines of code.

Clusternet supports cloud-native software and projects and has supported multi-cluster distribution of Helm Charts since v0.2.0. Based on Clusternet’s native support for any K8s resource, all K8s ecosystem projects and software can achieve multi-cluster capability through Clusternet projects.

Note: Kubectl-clusternet will soon be added as an official Kubectl plugin. Please refer to PR for details. You can install kubectl krew install Clusternet in the future. At this stage, you can download binary files directly to use.

future

Clusternet is Tencent’s latest open source multi-cluster management and application governance project. The new architecture follows the concept of cloud native, fully compatible with native K8s resources and the expansion of multiple clusters, enabling users to connect to the K8s cloud native ecosystem quickly and low-cost, and facilitating the transformation of your business to the future distributed cloud.

Clusternet is still in the process of rapid development, with the upcoming release of V0.4.0, which will bring more excellent features. Please keep your eyes on it. Clusternet is ready to work with colleagues in all fields to seek truth and innovation, forge ahead, and jointly usher in a new era of intelligent connection, co-existence of reality and reality.

Join us

While the Clusternet project is open source, visit github.com/clusternet/… Like support.

Interested developers and partners are welcome to participate in the Clusternet project by clicking on the link to participate in the Clusternet Survey (wj.qq.com/s2/8944628/… , we have prepared 10 Tencent customized technical t-shirts for students to fill in the questionnaire, now participate!

We are gathering the first group of Open source technology fans for Clusternet in limited quantity. If you are interested in Clusternet and related technologies, you are welcome to join us. How to participate: add the following QR code and answer: Clusternet, Xiao Yun will invite you to join the group.

Our open source technology fans will enjoy the following special benefits

1. Exclusive open source technology fan communication group

2. Communicate directly with project authors and industry experts

3. More community activities and benefits await you