In KubeCon2021, Wang Jiezhang, huawei Cloud native technology development engineer, delivered the keynote speech “Extension and Expansion of Service Mesh in Edge Computing Scenarios”, and shared the project overview of KubeEdge EdgeMesh.

The speech mainly consists of four parts:

1) Network challenges in edge computing scenarios

2) EdgeMesh project overview, structure and core capabilities

3) EdgeMesh performance test

4) EdgeMesh RoadMap

The following is the full text of the speech

First of all, let’s take a look at the cloud native container network. The cloud native container network has gone through three stages:

Stage 1: Docker container network. After the appearance of Docker, it also has its own four container network models, such as Host mode, Content mode, None mode and Bridge mode. However, the native Docker container cannot solve the problem of cross-level communication between containers. Later Docker introduced CNM and the corresponding implementation libnetwork to solve this problem.

The second stage: Container network interface (CNI). Later, due to various reasons, Kubernetes’ main push of CNI exceeded THE popularity of CNM. CNI is a container network interface specification with simpler interface and higher compatibility.

The third stage: Service grid + CNI. With the development of service network, it will cooperate with CNI. Some service grid plug-ins will inject Sidecar agent into POD at the start of each POD to provide layer 4 or 7 traffic management functions.

Kubernetes service discovery

The development of Kubernetes service is actually dependent on the container network. First, the user will create a set of back-end instances of the application through Deployment to manage it. However, the life cycle of POD is very short, and its POD IP may change with pod update, upgrade, migration, etc. This phenomenon is called POD IP drift.

To solve this problem, Kubernetes community proposed the concept of Service, each service corresponds to a set of back-end application instance, by providing a constant Cluster IP address to solve the problem of Pod IP drift. A proxy component is also provided to maintain the conversion rules from Cluster IP to Pod IP based on the information provided by the control plane. When a client needs to access the service, it usually only needs to access the same Cluster IP address. This traffic passes through the local network stack and is replaced with the Pod IP of the mysql backend instance, which is sent via the container network request.

Key challenges in edge scenarios

1) Edge computing has many subdivisions and poor interoperability

2) The edge cloud communication network has low quality and high delay, and the edge is often located in the private network, making it difficult to achieve two-way communication

3) Limited edge resources require lightweight component management to run edge applications

4) When the edge is offline, service autonomy and local fault recovery are required

5) Edge nodes are highly dispersed. How to efficiently manage them and reduce operation and maintenance costs

6) How to standardize management and flexibly configure heterogeneous resources

The edge computing platform KubeEdge can solve all of these key challenges and solve almost all of them. But in addition to these issues, there are other issues, such as having a video streaming application on the edge that needs to interact with an AI application on the cloud. First of all, the network on the edge cannot be directly connected with the network on the cloud, so it is impossible to access the cloud from the edge. This problem can be solved by configuring a public IP address for the AI applications on the cloud. However, if each application on the cloud is configured with a public IP address, the IP address cannot be converged. Moreover, cloud applications that want to actively access edge applications cannot actually do so, because edge applications are generally on private networks and do not have public IP addresses, so they cannot achieve correct routing and two-way communication.

In general, there are the following problems:

1) The edge cloud network is fragmented, and micro services cannot communicate directly across subnets

2) The edge side lacks service discovery ability

3) Complex network configuration management in an edge environment

Based on the above problems, EdgeMesh came into being.

EdgeMesh project overview, architecture, core competencies

Cloud native edge computing platform — KubeEdge

KubeEdge is an extension of Kubernetes in edge scenarios. The goal is to extend Kubernetes’ container orchestration capabilities to the edge. KubeEdge mainly consists of two components, CloudCore in the cloud and EdgeCore on the edge node, as well as a Device module for managing a large number of edge devices.

EdgeMesh’s position in KubeEdge architecture: EdgeMesh enables network communication between all edge applications, whether they are in the same component or different components. In addition, EdgeMesh can communicate between applications on the cloud and those on the edge.

EdgeMesh design principles

Lightweight: Each node only needs to deploy a very light Proxy component, and the edge side does not rely on CoreDNS, Kube-Proxy, CNI plug-in and other native components

Cloud-native experience: Provide cloud-native service discovery and traffic forwarding experience for container applications in the KubeEdge cluster

Cross-subnet communication: Shielding complex edge network environment and providing cross-subnet edge and edge cloud communication between containers

High reliability: Point-to-point direct connection is established through holes, and the forwarding efficiency is very high. When holes are not supported, traffic is forwarded through trunks to ensure normal communication between services

Layered architecture: With a layered design architecture, modules can be compatible with native components and support dynamic shutdown

What is EdgeMesh?

define

As the data plane component of KubeEdge cluster, EdgeMesh provides simple service discovery and traffic proxy functions for applications, thus shielding the complex network structure in edge scenarios

advantage

EdgeMesh meets the new requirements of edge scenarios (such as limited edge resources, unstable edge cloud network, complex network structure, etc.), that is, it achieves high availability, high reliability and extreme lightweight:

1. High availability

1) Use P2P hole-making technology to open up the network between edge nodes

2) The communication between edge nodes is divided into intra-LAN and cross-LAN

  • Communication within the LAN: Direct communication
  • Cross-lan communication: If the tunnel is successfully drilled, a direct channel is established between agents. Otherwise, the tunnel is forwarded through the Server trunk

2. High reliability (offline scenario)

1) Control surface metadata is delivered through KubeEdge edge cloud channel

2) EdgeMesh internal implementation of lightweight DNS server, domain name requests within the node closed loop

3. Extreme lightness: Each node has only one Agent, saving edge resources

4. User value

1) Users have the ability of cross-subnet communication across different Lans from side to side/side to cloud/cloud to side

2) Compared with the edge deployment of kube-proxy + CNI + Nodelocaldns components, users only need to deploy One Agent on the node to complete the target, all-in-One deployment

Key functions currently implemented:

EdgeMesh architecture

EdgeMesh-Agent

  • Proxier: configures the kernel’s iptables rules to intercept requests to the EdgeMesh process
  • DNS: a built-in DNS resolver that resolves domain name requests within nodes into the cluster IP address of a service
  • Traffic: a Traffic forwarding module based on the Go-chassis framework that forwards Traffic between applications
  • Controller: Get Services, Endpoints, Pods and other metadata through KubeEdge’s edge side Local APIServer capability
  • Tunnel-agent: provides cross-subnet communication by using trunk and hole-making technologies

EdgeMesh-Server

  • Tunnel-server: establishes a connection with the Edgemesh-agent, assists in tunneling, and provides the relay capability for the Edgemesh-Agent

How EdgeMesh works

EdgeMesh performance test

  • In the case of equal throughput and equal test application scale, the memory consumption of edgemesh-agent is about 30% of that of ISTIO-Proxy, and the CPU consumption of Edgemesh-Agent is about 10% of that of ISTIo-Proxy

  • The memory consumption of the control plane component Edgemesh-server is about 4.5% that of ISTIOD, and the CPU consumption of edgemesh-Server is about 0.15% that of ISTIOD

  • EdgeMesh is the agent of node mode. The total resource consumption of data plane is a multiple of the number of nodes in the cluster.

    Istio is a sidecar agent. The total resource consumption on the data plane is a multiple of the number of applications in the cluster

  • In the same network environment, when the throughput is 20RPS(Request per second) and 100RPS concurrent requests, EdgeMesh’s forwarding performance is better than THAT of Istio (the forwarding rate increases by about 10 times).

    The EdgeMesh end-to-end delay in this case was almost identical to the benchmark group test results

  • The variance of EdgeMesh end-to-end delay increases with the increase of the number of concurrent requests. When the number of concurrent requests exceeds 500RPS, the end-to-end delay of EdgeMesh fluctuates significantly (it is being optimized).

  • Benchmark Test tool:

    Github.com/Poorunga/se…

EdgeMesh RoadMap

2021.06: EdgeMesh is open source

2021.09: EdgeMesh supports microservices across LAN side cloud/side communication

2021.12: EdgeMesh supports one-click deployment

2022.03: EdgeMesh supports Pod IP traffic forwarding across the edge cloud

2022.06: EdgeMesh connects to standard Istio for service governance control

2022.09: EdgeMesh supports cross-cluster service communication

Finally, welcome more cloud native enthusiasts to join us and explore the direction of edge computing together!

Attached: technical exchange address

Website: kubeedge. IO

EdgeMesh Github github.com/kubeedge/ed…