Guide language | controls the clouds and micro edge calculation, in a local area network service enterprise how to do? Tencent Youtu service combines Tencent cloud edge container TKE@edge to make Devops, and makes customization to the service to support the corresponding business scenarios. The next part of this article will show the details of practice implementation in detail, hoping to bring inspiration to everyone.
background
The so-called private cloud, in fact, is to play services on multiple lans, basically equivalent to the development of the operation and maintenance of the whole package. Each LAN requires a jumper, LAN environment (each LAN environment is different), hardware, software, etc., and then requires a large number of human operation and maintenance deployment upgrade services (traditional practices such as Ansible, Fabric, SCP, such as copy, configuration update, version maintenance are very troublesome, so it is abandoned). Moreover, different LAN services need to be configured differently, so configuration management is also a problem.
The author has also thought about doing a set of LAN automatic deployment, the idea is to deploy agent on the LAN to get through with the cloud, and then send various data to send commands. At this time, I suddenly saw my colleague writing TKE@edge. After reading it, I felt that it was what I wanted, so I started to do it.
The status quo
Batch Deployment problemSCP and Fabric deployment tools are used for batch deployment. Each LAN uses a different configuration. To change the configuration, you need to log in to the machine one by one.Differential configuration problem: In order to solve the configuration problem, the use of the configuration center, all the configuration centralized management, but each LAN configuration center is still different, although has converged to a service, or feel very tired and prone to error;Services are invoked upstream and downstream: Uses a self-developed service discovery component, which combines the DNS function of Consul to address upstream and downstream services through DNS. This problem can be solved very well.
TKE @ edge profile
Is there a product that can control service deployment upgrades in the cloud? It is understood that TKE@edge is one of them, which can better solve this problem.
In addition, there is an open source solution, K3s, which can run K8s on devices with limited resources through tailoring, but it still fails to solve several problems I care about most, such as:
1) Cloud operation and maintenance;
2) Manage edge nodes across networks and regions in a cluster;
3) Simplify the problem of differentiated configuration management in different regions.
Next, let’s take a look at how K3s and TKE@edge work and how they differ.
K3s working principle diagram
TKE @ edge architecture diagram
From TKE Edge Container Series edge computation and edge container introduction.
As can be seen from the above architecture diagram, TKE@edge adds a tunnel to get through the external network and transmit data and commands, which is the agent I mentioned before. In addition, hub-edge, an autonomous component of edge nodes, is added, which corresponds to cloud control one by one.
TKE@edge makes a few highlights:
1. The ServiceGroup: In addition, the business system can be automatically copied, which is a set of business system, not a single Pod. For example, if a LAN Zone is added, it can be automatically copied to a new LAN without intervention.
2. Distributed health detection: In order to avoid network problems in weak network environment and cloud management, autonomous decision can be adopted to determine which PODS are really expelled.
3. Supports heterogeneous nodes.
My Core Problem (Q) and Solution (A)
1. The service can control deployment and upgrade in the cloud
tke@edge provides TKE console like Tencent cloud container service, which can be operated in batches.
2. The service cannot be accessed across lans
ServiceGroup, services in the same Zone communicate with each other and services in different zones are isolated by labeling nodes as zones. TKE@edge Creates Deployment with the resources of Deploymentgrid.
3. The K8s service performs a complex LB policy such as consistency hash
First, import the Pod NodeName into the Pod environment variable with the K8s API; then filter the Label through the Node zone information and the CLIENT-Go API; this needs the support of the upper level service discovery component. Why not use K8s Ingress and Service? Sorry, it is not supported.
4. Service traffic injection
NodePort exposes the service. In order to avoid network adapter bursting, multiple host nodeports are required to receive traffic. Consul is used to register the service, similar to Tencent cloud CLB solution
5. Export service traffic
A small problem
6. Differentiated service configuration by region: one set of code, cloud customized configuration, Configmap management for service configuration through zone association, and Configmap is mounted to Pod container directory through Volume mechanism. How to decide which region to use which configuration? Select by passing in NodeName. After this problem is solved, the new shopping mall (LAN), only need to configure the corresponding configuration in the cloud, you can automatically expand the capacity, the bunker is simply.
7. Some minor issues are not enumerated here
Results show
The author did deployment in Xi ‘an shopping mall and Hebei shopping mall, and cut part of the flow of Xi ‘an field.
The cloud deployment
The Deploymentgrid console is not yet developed and resources can only be created through Kubectl
Configuration management
Once these two knotty problems are solved, we’re done.
Cost and benefit comparison
** Past: ** deployed a set of multiple services in the mall, and a team of 7 or 8 people worked for a week (sometimes two weeks), connecting upstream and downstream;
Now: seconds!! And it works automatically!! Really cow!! Finish, have a premonition to feel oneself want to be cut, awesome programmer, is to change the life of ordinary programmer.
Conclusion outlook
At present, I think the problem is that tke@edge should be customized based on K8S, which takes up a lot of resources. There are some requirements for AI equipment, such as the ability to run Docker, as well as hardware platform and operating system. In addition, in the process of node addition, there is no function of batch labeling for nodes, and the scheduling rules for node label modification need to be clarified. The node limit and large-scale Pod scheduling performance of tke@edge single cluster can be managed have not been thoroughly studied.
With the arrival of 5G, more and more devices will be marginalized, as will computing, and edge containers and scheduling will be a promising direction.
[Tencent cloud native] cloud said new, cloud research new technology, cloud travel new live, cloud appreciation information, scan code to pay attention to the public account of the same name, timely access to more dry goods!!