The author

Lingfei Kong is a senior engineer of Tencent Cloud. He has experience in r&d and architecture of large-scale Kubernetes cluster and microservices, and currently focuses on infrastructure development of cloud native hybrid cloud.

Zhu Xiang, senior product Manager of Tencent Cloud Container Services, is currently responsible for the design of cloud native hybrid cloud products.

preface

Enterprise digital transformation has become the core strategy of enterprises. Cloud computing as the core of the new generation of IT technology, has become an important support for enterprise digital transformation, the cloud has become the only way of enterprise digital transformation. Due to reasons such as data security, privacy, resource utilization, and service disaster recovery, enterprises usually adopt the hybrid cloud architecture when they log on to the cloud. The hybrid cloud has become the new normal of cloud services on the enterprise.

In recent years, with the popularity of cloud native technology in the cloud computing market, the combination of hybrid cloud and cloud native has become a trend. Cloud native technologies represented by Docker and Kubernetes shield the difference of underlying infrastructure and become the actual basis of enterprise hybrid cloud management. At the same time, CNCF community provides a number of cloud native tools and software to help enterprises further reduce cloud/cloud differences. Cloud native and hybrid clouds are a “match made in heaven”.

TKE Everywhere is a new feature launched by Tencent Kubernetes Engine (TKE) for hybrid cloud scenarios. TKE Everywhere supports users to run managed TKE services in their own infrastructures.

Customer cloud challenge

Resources of the old

The enterprise has made investment in the data center. If it wants to migrate to the cloud, how to deal with this part of the stock of servers? There is no extra cost for the cloud. How to amortize the cost of resources in the process of the cloud becomes a difficult problem to effectively use these resources.

Consistent experience

When an enterprise uses the cloud, the cloud and THE IDC are independent of each other. Therefore, it is easy to form an isolated island of O&M deployment, resulting in “mixing” but not “integration”. By “mixing” rather than “merging”, enterprises do not enjoy the value of hybrid cloud architecture, but bring additional complexity and problems. Under the business architecture of hybrid cloud, customers need a consistent and unified experience.

TKE Everywhere

TKE Everywhere is a new deployment option of TKE Container Service (Tencent Kubernetes Engine), which enables users to run Kubernetes container services on their own infrastructure, including virtual machines and physical machines, consistent with TKE on the cloud. TKE Everywhere uses a managed control plane to centrally manage the life cycle of clusters, such as creating, upgrading, and monitoring clusters. Users only need to provide computing resources.

TKE Everywhere provides for users

Mixed Node deployment

TKE Everywhere offers a hybrid deployment of multiple types of nodes. You can run IDC nodes, CVM/ Blackstone physical servers, and EKS, the elastic container service, in a cluster.

On the one hand, IDC’s resources can be effectively utilized; On the other hand, you can easily extend your business from the cloud to the cloud when you need it.

The extreme elastic

Supports rapid expansion of IDC resources by using cloud resources, enabling unlimited expansion of IDC resources to effectively cope with traffic bursts. You can quickly scale up CVM on the cloud or use an out-of-the-box flexible container serving EKS.

Seamlessly expand services on the cloud

Seamless integration of Tencent Cloud native related services, including monitoring, security, audit, log, service grid, image distribution acceleration, etc., you can connect the monitoring indicators of clusters and services to Tencent Cloud native Prometheus monitoring service, saving the cost of self-built monitoring system. Logs can also be delivered to the cloud native log service CLS for storage and analysis. Seamless integration with TCM, the service grid on the cloud, and construction of micro service governance capabilities.

Directions for use

In order to ensure the stability of the cluster, we recommend using Tencent Cloud Networking to Connect users’ IDC Network with the VPC Network of public Cloud. Cloud Connect Network (CCN) provides Intranet interconnection services between private networks (VPCS) and between VPCS and local data centers (IDCs) on the Cloud. It is capable of multi-point interconnection, route self-learning, link optimization, and fast fault convergence.

TKE Everywhere supports Cilium Overlay network based on VxLAN tunnel mode and Cilium Underlay network based on direct routing to shield the differences between underlying network environments.

Overlay networks can be used when customers do not want to change their IDC infrastructure. When customers have high performance requirements for hybrid cloud container networks, Underlay networks based on direct routing can be used.

The premise

  1. The TKE cluster has been created.
  2. Kubernetes cluster version 1.18 or later.

Enable the third-party node function

In the cluster [Basic Information] – [Node and Network Information], enable the function of third-party nodes and fill in relevant parameters as required, as shown in the following figure:

Example Create a third-party node pool

Tencent Cloud TKE uses the capability of third-party node pools to register non-Tencent cloud servers as TKE cluster nodes. Users provide computing resources, and container service TKE is responsible for the management of the whole life cycle of the cluster.

Node initialization

Create a node, obtain the node initialization script, and run the script on the IDC server to be registered with the cluster.

Add a node

Run the following command to add a node

./add2tkectl-cls-m57oxxxp-np-xxxx install
Copy the code

Customer case – research on the cloud

Tencent’s internal self-research business on the cloud has become the main direction, here we select a representative case of a self-research business on the cloud. At present, the self-research service is deployed in the IDC room of the self-research. Most of the services are containerized, and rich experience in containerization has been accumulated.

The appeal of the cloud

Resources of the old

At present, the self-research business holds a large number of server resources in the self-research IDC room, because there is no extra cost. If you want to migrate to the cloud, how to deal with this part of the stock of servers. How to amortize the cost of resources in the process of cloud, and effectively use this part of resources has become a difficult problem.

Unification of operation and maintenance system

Service deployment and O&M systems are unified in the on-cloud and off-cloud IDC basic environments. The inconsistency of operation and maintenance management system will bring additional burden to the cloud of self-research business.

The cloud solution is TKE Everywhere

In view of the current situation and demands of self-research teams, we recommend TKE as the environment for the deployment of cloud services, adopting TKE Everywhere features. TKE Everywhere enables self-developed IDC rooms to run the managed TKE service. The control plane of a cluster is completely managed by TKE, eliminating the need for users to manage clusters.

In order to ensure the stability and high performance of the service, the network between the IDC room developed by ourselves and the public cloud VPC is connected through Tencent Cloud Networking.

In a cluster of self-developed services, both the resources under the cloud and those under the cloud are deployed in the cluster. In this way, IDC computing resources are effectively utilized and all services are migrated to the cloud as IDC computing resources are gradually phased out.

TKE Everywhere seamlessly integrates logging, monitoring, auditing, service grid and other products and capabilities around Tencent Cloud TKE. Self-developed services The monitoring and log systems originally built by IDC need to be operated and maintained by special personnel. To make full use of cloud advantages, the self-built monitoring and log systems are gradually migrated from the cloud to the cloud native Prometheus monitoring service and cloud native log service CLS provided by Tencent Cloud to reduce o&M costs.

With the help of TKE Everywhere, the self-research business migrated hundreds of thousands of nuclear resources to the cloud within a month, effectively supporting the self-research business cloud.

conclusion

TKE Everywhere offers a new solution for the hybrid cloud scenario, enabling users to run managed TKE services on their own infrastructure. TKE Everywhere is fully hosted, and users only need to provide computing resources instead of a local O&M cluster.

Private invitation

The characteristics of the TKE Everywhere now open beta testing, we sincerely invite you to participate in the beta testing application, please click on the submit work orders (console.cloud.tencent.com/workorder/c…