In May 2019, CNCF set up the Universal Data Plane API Working Group (UDPA-WG) to develop the standard API for Data Plane.

At that time, I wrote a blog post “CNCF is preparing a Common Data plane API working group to develop a standard DATA plane API” to introduce this. At that time, UDPA was in its very early stages of preparation and very little information was available.

Now that nine months have passed, I have recently collected and sorted out the current situation and information of UDPA to give you the latest progress of UDPA (as of February 24, 2020).

In addition, Ant Financial’s open source cloud native network agent MOSN currently supports xDS V2 API, and will gradually evolve towards UDPA in the future, which is compatible with the standard Istio. Interested readers can go to know about it.

MOSN：github.com/mosn/mon

UDPA introduction

First, a quick overview of what UDPA is:

  • UDPA: short for Universal Data Plane API, Universal Data Plane API
  • Udpa-wg: short for “Universal Data Plane API Working Group”, a Working Group under CNCF responsible for formulating UDPA;

Objectives of UDPA, as described by github.com/cncf/udpa:

The goal of the Common Data Plane API Working Group (UDPA-WG) is to bring together industry stakeholders interested in common control and configuration apis for data plane agents and load balancers.

UDPA’s vision, also quoted:

Vision for common Data Plane API (UDPA) at blog.envoyProxy. IO /the-univers… Set forth. We will seek a set of apis that provide a de facto standard for L4/L7 data plane configuration, similar to the role played by L2/L3/L4 OpenFlow in SDN. These apis will be formally defined in Proto3 and evolve from the existing Envoy xDS API with well-defined, stable API versioning policies. The API will cover service discovery, load balancing allocation, route discovery, listener configuration, security discovery, load reporting, health check delegation, and more. We will refine and shape the API to support client lookaside load balancing (e.g. Grpc-lb), data plane proxies outside Envoy, hardware LB, mobile client, and other scopes. We will strive to be as vendor-independent and implementation-independent as possible, while maintaining support for UDPA projects already in production (Envoy and GRPC-LB so far).

Students who are interested in UDPA can learn more about it through the following two ways:

  • UdPA@github: UDPA projects on GitHub, UDPA API defined code here;
  • Universal Data Plane API Working Group (UDPA-WG) : CNCF UDPA Working Group, you can join the Working Group to learn more information;

UDPA and xDS relationship

Before going into the details of UDPA, it is important to explain the relationship between UDPA and xDS, as this can be very helpful in understanding UDPA.

At EnvoyCon in November 2019, Google’s Harvey Tuch, developer at Envoy and one of the current UDPA principals, addressed this question in great detail and clarity in a talk titled: “The Universal Dataplane API (UDPA): Envoy’s Next Generation APIs”.

Note: HERE I directly quote part of the speech. The following two pictures are from [the PPT of the speech](static.sched.com/hosted_file… UDPA 2019. PDF). Thanks Harvey.

The following chart shows the evolution of THE xDS protocol in recent years and its future plans:

  • XDS V2 introduced Proto3 and gRPC in 2017, the same year Istio project was launched.
  • In 2018 and 2019, the xDS V2 API continued to evolve, with the introduction of new API definitions such as HDS/LRS/SDS. In particular, incremental push mechanisms were introduced to improve Pilot delivery performance.
  • The xDS V3 API was originally scheduled to be released by the end of 2019, but the technology is currently delayed. V3 is still Alpha1, and more V3 apis are expected to be introduced in the upcoming Istio 1.5 release. At the same time, V3 API also introduced part of UDPA content, but because UDPA is slow at present, the impact on xDS is not big, mainly xDS itself development, such as API and technical debt clearing;
  • But in 2020, expect UDPA to come a long way, especially after the design of UDPA-TP and UDPA-DM, which we will unfold below, begins to be formalized as apis. XDS V4 is expected to be based on UDPA, so xDS V4 is likely to see some big changes;

To summarize: xDS will move closer to UDPA and will be based on UDPA in the future.

The following image shows the Envoy’s timeline on xDS version support:

There is currently a slight delay in the implementation of this plan, and the stable version of V3, originally scheduled for late 2019, was actually finalized in mid-January. (Note: See Envoy PR API: Freeze V3 API for details). Then the currently widely used V2 API will be marked as depreated. By the end of 2020, the V3 API is expected to be replaced by the V4 API (note that THE V4 API will be based on UDPA), while the current V2 API is scheduled to be removed from support by the end of 2020!

The figure above also shows how major versions of the xDS protocol will evolve and replace in the future. Generally speaking, the pattern is as follows:

  1. One large version per year: 2019 V2 -> 2020 V3 -> 2021 V4 ->2022 V5;
  2. Each major release goes through alpha -> stable -> Deprecated -> removed phases, each lasting a year;
  3. Three large versions of the API will exist simultaneously in the stable post Envoy: a stable version in use, a deprecated previous stable version, and a new and next large version ready for development (but only Alpha);
  4. A major release of a new stable will be deprecated for the previous stable major release and remove for the previous deprecated major release.

As the saying goes, “The waves behind the Yangtze River push on the previous ones, and the former ones die on the beach.” Or, “The new versions of the Yangtze River and Mountain dynasties have been leading the pack for 12 months.”

Remark: For detailed Stable API versioning strategies, see Envoy’s design document “Stable Envoy API Versioning”, which is a bit too long. Those who are too long can refer to the shortened version of this document, API Versioning Guidelines.

UDPA API progress

Without further ado, let’s take a look at the latest development of UDPA. From github.com/cncf/udpa, you can see some of the API content defined in the UDPA so far:

The type definition

Currently, only one type TypedStruct is defined.

TypedStruct contains any Protocol Buffer message that has been serialized from JSON and a URL that describes the type of serialized message. This is very similar to Google.protobuf.any in that it uses Google.protobuf.struct as a value instead of the Protocol buffer binary.

Message TypedStruct {// URL used to uniquely identify the type of serialized Protocol buffer messages // This has the same semantics and format as described in Google.protobuf.any:  // https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto string type_url = 1; // JSON representation of the type specified above. google.protobuf.Struct value = 2; }Copy the code

TypedStruct is defined in the context of how to embed an opaque configuration in statically typed packets of the Protocol buffer. This is a common requirement that involves the difference and tradeoff between Google.protobuf. Any and Google.protobuf. Struct. See my previous translated blog post “Dynamic Extensibility and Protocol Buffers” for a good introduction and discussion.

TypedStruct is arguably the best practice for this requirement by far, and is a formal end to this topic.

Data definition

The data definition also defines only one data OrcaLoadReport.

ORCA stands for Open Request Cost Aggregation, and OrcaLoadReport is used to submit a load report that summarizes the Cost of the Request.

A brief introduction to ORCA

Today, simple load-balancing decisions can be made in an Envoy by considering local or global knowledge of back-end loads such as cpus. More complex load balancing decisions may require application specific knowledge, such as queue depth, or combining multiple metrics. This may for limited resources on the multiple dimensions of the service (for example, CPU and memory can be the bottleneck, depending on the application of the load and execution environment, which is unable to determine first hit a bottleneck) and these dimensions is not predetermined categories is very useful when the position of (for example, resources may be the number of threads available in a “pool”, disk IOPS, etc.).

For more detailed information on Orca, see the design document Open Request Cost Aggregation (Orca).

Currently Envoy is implementing SUPPORT for ORCA, and this feature is defined directly in the UDPA API as part of the UPDA standard. The following is the OrcaLoadReport definition, which contains CPU/ memory utilization and RPS information:

Message OrcaLoadReport {// CPU utilization is represented as a portion of available CPU resources. Should be from the latest samples or measurements. double cpu_utilization = 1 [(validate.rules).double.gte = 0, (validate.rules).double.lte = 1]; // Memory utilization is represented as a portion of available memory resources. Should be from the latest samples or measurements. double mem_utilization = 2 [(validate.rules).double.gte = 0, (validate.rules).double.lte = 1]; // Total RPS served by the endpoint. All services for which the endpoint is responsible should be covered. uint64 rps = 3; . }Copy the code

The service definition

The service definition also defines only one service, OpenRcaService.

OpenRcaService is an out-of-band (OOB) load reporting service that is not on the request path. OpenRcaService periodically samples reports with sufficient frequency to provide relevance to requests. OOB reporting compensates for the limitations of in-band reporting.

service OpenRcaService { rpc StreamCoreMetrics(OrcaLoadReportRequest) returns (stream udpa.data.orca.v1.OrcaLoadReport);  }Copy the code

Annotations to define

UDPA currently defines four annotations:

  • MigrateAnnotation: Identifies migration-related API changes in previous and subsequent releases, including rename/oneof_promotion/move_to_package semantics;
  • SensitiveAnnotation: Marks a field as “sensitive,” such as personally identifiable information, password, or private key;
  • StatusAnnotation: Mark the state of a file, such as “work_in_progress/ in progress”;
  • VersioningAnnotation: Used to record version information, such as previous_message_type indicating the current message type in the previous version;

There was also a ProtodocAnnotation that was divided after coming up with the design and had not yet formally joined the UDPA. The purpose of this annotation is to mark UDPA messages that are not currently implemented;

Total UDPA API

As you can see from the UDPA API list above, there are very few apis officially released in UDPA, namely:

  • A TypedStruct type definition;
  • An OpenRcaService service definition and accompanying OracLoadReport data definition;
  • 4 notes;

This is a bit of a surprise given that UDPA was launched in May 2019, nine months to date.

Looking through the content on github.com/cncf/udpa, including all the commit and PR, I found that the active developers are mainly two students: Htuch from Google and Lizan from Tetrate. However, the number of STARS of CNCF /UDPA project is also very low, only 55 stars, so it can be considered that the community pays little attention to it.

However, when I looked at the UDPA design document later, I realized that the best of UDPA was in the design, but the API had not been formally developed for schedule reasons.

UDPA design

Let’s focus on the design of UDPA. There are two main design documents:

  • Udpa-tp Strawman: TransPort designed by UDPA, which is the next generation TransPort protocol for UDPA;
  • Udpa-dm: L7 Routing strawman: Data Model designed by UDPA, which is the definition of the resource transmitted through UDPA-TP;

The UDPA explains this:

The Envoy V2 xDS API is currently moving to the Universal Dataplane API (UDPA). The focus is on the separation of concerns between transport protocols and data models.

A typical example of the separation of concerns between transport protocols and data models is “container transport mechanism” (analogous to UDPA-TP) and “standard in containers” (analogous to UDPA-DM). In the design of UDPA, the definition of the data model and the implementation of the transport protocol are separated, which means that a unified transport protocol can be reused as long as different data models are designed. As a result, UDPA is very scalable.

In this regard, I was somewhat surprised, because at the end of last year, when Yanlin and I discussed the integration of registry and control plane through MCP/xDS/UDPA protocol, we found that the three work mechanisms are very similar. Considering that a variety of different resources may need to be defined and synchronized and distributed on this working mechanism, I had a similar idea at that time, hoping to standardize the underlying resource synchronization mechanism for reuse:

So far, the UDPA-TP appears to have taken solid steps towards this goal. Of course, it would be nice to take this one step further: the underlying resource synchronization mechanism does not need to be confined to UDPA, but can become a more general purpose mechanism.

Here’s a closer look at the udPA-TP and UDPA-DM designs, from two UDPA design documents and personal interpretation.

UDPA – TP design

The udPA-TP design document lists the key design motivations for udPA-TP at the beginning, including:

  • Advanced features such as LRS/HDS are supported while retaining the conceptual PUB-Sub model that exists in Core V2 xDS;
  • Support extensive extensions to Envoy Mobile and other DPLB clients;
  • Simple client and simple management server implementation;
  • Make incremental update resources efficient and simple (note: Incremental updates are a key feature Istio/Envoy is working on right now);
  • Support for resource federation (note: in line with my previous idea of aggregating multiple registries/configuration centers through the MCP protocol, when multiple sources of resources exist in reality, mechanisms must be provided to aggregate these resources and provide a global view);
  • Make subresources of API resources simple and cheap to implement, and make them updatable incrementally and syndication;
  • Maintain support for conformance models;
  • Eliminate v2 xDS oddities;
  • Merge of ACK/NACK and subscription messages. In V2 xDS, DiscoveryRequest is both a subscription request and a potential acknowledgement of a previous message. This leads to some complicated implementation and debugging experiences (note: this can make UDPA interaction patterns different from xDS);
  • CDS/LDS is a different API layer than EDS/RDS. In V2XDS, EDS/RDS is quasi-incremental, while CDS/LDS is the latest state.
  • Conceptually, small scale changes based on the Envoy V2 xDS API. We do not want to impose significant conceptual and implementation costs on UDPA administration server implementers (Note: Personally, this is one of the reasons why UDPA doesn’t have an extensive set of apis at the moment, UDPA and xDS, including the actual implementer of xDS Envoy, are too close to consider the transition from xDS to UDPA, and can’t simply roll out a whole new set of apis);

Here are UDPA terms that will help you understand UDPA later:

  • DPLB: Data Plane Load balancer. Covers use cases such as proxies (such as Envoy) or client-side RPC libraries (such as gRPC and Envoy Mobile). DPLB is the UDPA client. They are responsible for starting UDPA streams to the admin server (Note: Note that DPLB includes not only the Service Mesh Sidecar represented by Envoy, but also SDK libraries such as gRPC, which already supports the xDS interface);
  • Management Server: An entity capable of providing UDPA services. The Management server can be just a UDPA server or, in the federated case, a UDPA client and server;
  • UDPA: Common data plane API, including data model (UDPA-DM) and Transport protocol (UDPA-TP);
  • Udpa-tp: UDPA API benchmark transport protocol.
  • Udpa-dm: Data model of UDPA API;
  • UaaS: UDPA-as-A-Service, cloud hosted UDPA management service (note: Google Traffic Director provided by Google on GCP and App Mesh provided by AWS can be regarded as the prototype of UaaS);

Then there are federation-related terms:

  • Federation/ Federation: Interoperation of multiple UDPA management servers to generate UDPA configuration resources;
  • Direct federation/ Direct federation: when the DPLB is directly connected to multiple UDPA management servers and is able to synthesize its configuration resources from these streams;
  • Indirect Federation: When the DPLB connects to at most one UDPA administration server at a time (for some given resource types) and the UDPA administration server performs all federation-related work;
  • Advanced DPLB/ Advanced DPLB: DPLB that supports direct federation (requires UDPA-FeD-TP);
  • Simple DPLB/ Simple DPLB: DPLB that does not support direct federation, that is, only baseline UDPA-TP;
  • Udpa-fed-dm: A superset of UDPA-DM with additional resource types for federation;
  • Udpa-fed-tp: a superset of udPA-TP that supports federation.

The following chart helps to understand these terms:

(Note: The value between simple UDPA management Server and Advanced UDPA Management Server should be UDPA-TP instead of UDPA-Fed-TP.)

The UDPA-TP transport protocol provides the means to transfer named and versioned resources between the management server and the DPLB client, and we call these entities UDPA-TP endpoints. Udpa-tp endpoints can be client and administration server, or both administration servers (for example, in federated configuration). The figure above illustrates the various ways in which UDPA can be used to transfer resources between UDPA-TP endpoints.

UDPA management servers are divided into two types:

  • Simple: It is actually just a cache of opaque resources (almost no udPA-DM is known). Its main function is to obtain resources from the upstream advanced UDPA management server and distribute resources to DPLB, without generating resources itself.
  • Advanced: Aware of UDPA-DM, usually used to capture information and transform it into a standardized resource (typically, Pilot in Istio). You can either send resources directly to the DPLB or to the simple UDPA management server, which then distributes the resources to the DPLB.

The corresponding DPLB clients are also divided into two types:

  • Simple: A simple DPLB can talk to at most one management server at any point in time for any configuration source;
  • Advanced: Supports UDPA-FeD-TP and can participate directly as a federated endpoint;

The simple DPLB can only connect to one management server, but it can also be federated: the simple DPLB connected to the management server can be federated, and then indirect federation for THE DPLB (Note: The simple DPLB shown in the figure above is federated between two Advanced UDPA Management Servers.

Udpa-tp design interpretation

Before reading the udPA-TP design, let’s review the classic component and architecture diagrams for Istio. Here are the architecture diagrams for Istio 1.0 and Istio 1.1, respectively:

Considering that Mixer and Citadel components have less to do with UDPA, we focus on Proxy/Pilot/Galley:

  • Proxy in Istio is Envoy (our MOSN is working on becoming a candidate), a direct counterpart to the CONCEPT of DPLB in UDPA. However, Proxy in Istio is only functionally aligned with Simple DPLB in the preceding figure. In contrast, the UDPA-TP design adds an Advanced DPLB that can connect to multiple UDPA management Servers for federation;
  • Pilot and Galley correspond to Simple UDPA Management Server and Advanced UDPA Management Server in UDPA-TP design respectively in terms of their connection to DPLB. However, in terms of actual functions, the responsibilities of Pilot are far beyond the design of Simple UDPA Management Server, while the functions of Galley are far less than those of Advanced UDPA Management Server. If the architecture and components of Istio in the future are to resemble UDPA’s design, it is clear that the responsibilities of Pilot and Galley will have to change dramatically;
  • The biggest difference is that UDPA-TP introduces the concept of federation and supports federation on both DPLB and Advanced UDPA Management Server. Especially, if DPLB wants to realize federated function, it will increase the function of DPLB greatly, and the corresponding communication protocol between DPLB and UDPA Management Server (xDS at present) will also increase a lot of content for federated implementation.

Compare udPA-TP design:

There is probably no specific implementation product for udPA-TP design, and I haven’t found a detailed API design for UDPA-Fed-TP. There are few sources of data, so I can only make some preliminary personal interpretations:

  • First, the introduction of the Simple UDPA Management Server is a highlight, Simple enough and focused on distributing data (represented as resources in UDPA) to DPLBS (known as the data plane). Undoubtedly, the Simple UDPA Management Server will focus on key performance indicators such as capacity/performance/delivery efficiency/stability to make up for the weak performance of Pilot delivery in the current Istio design. From this perspective, I tend to think of Simple UDPA Management Server as a new component, somewhere between DPLB and Pilot. Summary: Address capacity and performance issues.

  • Second, Advanced UDPA Management Server introduces the concept of federation, The above image shows federation between two different Cloud providers (Cloud Provider X and Cloud Provider Y) and on-premise, which is a typical hybrid Cloud scenario. My understanding is that federation can not only be used for multi-cloud, but also for multi-data sources, such as opening up multiple registries and solving the problem of heterogeneous communication. Summary: Solve the global aggregation problem of multiple data sources.

  • Then, the concept of Advanced DPLB is introduced, and the usage scenarios are very complicated from the diagram: 1. The first DPLB is a typical indirect federation scenario and is relatively simple. 2. The second DPLB does indirect federation in the same way, Direct federation is also implemented through udPA-Fed-TP and on-premise Advanced UDPA Management Server. The third DPLB is more complex and federates three management Servers. Summary: Complex scenarios inevitably lead to complex mechanics, and the driving force behind them remains to be seen.

I personally don’t understand the design of UDPA-TP, mainly for federated usage scenarios. My question is: is there really such a complex scenario? In particular, the introduction of federated functionality into DPLB will inevitably force xDS/UDPA to provide federated API support for this, and Envoy/MOSN and others will become more difficult to implement. Therefore, unless there is a particularly strong need and scenario-driven, it is best to strike a balance between complexity and functionality.

I personally prefer something like the following:

  • Maintain the simplicity of DPLB and Simple UDPA Management Server;
  • DPLB connects to the Simple UDPA Management Server only, the protocol is UDPA-TP, and the federation is transparent to DPLB (only indirect federation is implemented).
  • Simple UDPA Management Server focuses on capacity and performance improvements, including ongoing incremental updates to support various resources.
  • Advanced UDPA Management Server is responsible for performing federation, including federated with Advanced UDPA Management Server in other environments and aggregating data from other local registries.

In summary, I personally prefer to keep DPLB and Simple UDPA Management Server Simple and high performance, leaving complex functions to Advanced UDPA Management Server. In the future, I will focus on the implementation of UDPA in the federal function. If there is any new progress, I will try to share it in a timely manner.

UDPA resource definition and design

In the design of UDPA-TP, resources are divided into two types based on their source:

  • Configuration Resources/ Configuration Resources: The control plane is generated and distributed to the DPLB by the management server. Familiar resources transmitted through xDS protocol, such as Listener/Route/Cluster/Endpoint, belong to this type.
  • Client Resources: The DPLB is generated and ready to be used by the control plane. The OracLoadReport defined in UDPA above is a typical example. It is a new feature that DPLB collects statistics and status and reports it to the control plane so that the control plane can refer to how complete it can be when making decisions.

When a resource is defined, it has three important properties:

  • Name /Name: unique for a given type, and the resource Name is structured, with a path hierarchy as follows: /, for example, com.acme.foo/service-a. Note the introduction of namespace as the basis for resource federation and ownership delegation;
  • Type /Type: The resource Type is represented by a string provided by the Type URL;
  • Version /Version: For a given named resource, it may have different versions at different points in time. After the version is added to the resource definition, it is very convenient to check whether the resource has been updated;

Here is an example of a resource definition (just a hint, not yet officially part of the UDPA API) :

Message Resource {// The name of the Resource to distinguish it from other resources of the same type. // Follow the reverse DNS format, such as com.acme.foo/listener-astring name = 1; // Resource level version string version = 2; // Resource payload. // pass AnytypeURL Specifies the resource type google.protobuf.Any resource = 3; // TTL of the resource. // After this time period, resources will become invalid on DPLB. Graceful degradation of resources, such as endpoint allocation, is supported when the connection to the management server is lost. // The same resource name/version/ is received with the new TTLtypeWill not cause any state changes other than refreshing the TTL. On-demand resources may be allowed to expire and may be recaptured when the TTL expires. // TTL Refresh message resource field may be empty, name/version/typeUsed to identify the resource to be refreshed. google.protobuf.Duration ttl = 4; // Provenance of resources (ownership, provenance, and completeness). Provenance origin_info = 5; }Copy the code

Other aspects of the UDPA-TP design, such as security/error handling/transport/user stories, are not covered in detail, but are still a bit far from being an API. If you are interested, you can browse the UDPA-TP design documentation directly.

UDPA – DM design

One of the core contents of UDPA design is the separation of concerns between TransPort and Date Model. The design of UDPA-TP was introduced earlier, and it can be said that it is still in progress and not fully finalized.

The DESIGN of udPA-DM, however, feels much earlier than udPA-TP, which is somewhat surprising: It was expected that UDPA would build on xDS ‘existing mature API definitions, quickly roll out a set of apis that covered common functionality, or even just clean up parts of xDS and move them over. But here’s the thing: At present, there are very few DEFINED API contents in UDPA-DM, only L7 Routing, which is still under design. Other familiar APIS such as Listener/Cluster/Endpoint/Security/RatingLimit are missing.

However, the design and implementation of UDPA-DM is not clear due to the lack of data. At the beginning of the UDPA-DM design document, there is the following description:

As a starting point, We recognize that the UDPA-DM is not required to be as expressive as any given DPLB client’s full set of capabilities, instead it should be possible to translate from UDPA-DM to various DPLB native configuration formats. We envisage UDPA-DM as a lingua franca that captures a large amount of useful functionality that a DPLB may provide, making it possible to build common control planes and ecosystems around UDPA capable DPLBs. First, we realized that UDPA-DM does not need to be expressive in all its capabilities like any existing DPLB client, but should be able to convert from UDPA-DM to various DPLB native configuration formats. We envision UDPA-DM as a general-purpose language with a number of useful features that DPLB should provide, making it possible to build a common control plane and ecosystem around UDPA-enabled DPLBS. The UDPA-DM will be a living standard. We anticipate that it will initially cover some obvious common capabilities shared by DPLBs, while leaving other behaviors to proxy specific API fields. Over time, we expect that the UDPA-DM will evolves via a stable API versioning policy to accommodate functionalities as we Negotiate a common representation. Udpa-dm will become the de facto standard. We expect that it will initially cover some of the obvious common features common to DPLB, while leaving the rest of the behavior to agent-specific API fields. Over time, we expect that UDPA-DM will evolve through a stable API versioning strategy to accommodate functionality, and that we will negotiate common representations.

I am a little confused about the interpretation of these two text descriptions, mainly about the clear relationship between the definition of UDPA-DM and specific native implementations of DPLB, such as Envoy xDS. Here’s a picture I drew:

  • The figure on the left is the conversion scheme: as described in the first paragraph, UDPA-DM is a generic definition that is then translated into the native configuration of the various DPLBS. This means that the UDPA-DM API and the actual DPLB native configuration format can be very different, and may even have completely different format definitions. This relationship is similar to that between Istio API and xDS API, and SMI (Microsoft’s Service Mesh Interface) and xDS.
  • The figure on the right is a subset scheme: as described in the second paragraph, UDPA-DM is a generic definition, but does not transform. Other DPLBS will simply reuse these UDPA-DM API definitions and supplement them with their own proprietary API definitions. In this way, UDPA-DM will appear in the form of a general subset and gradually expand its scope. Then, other apis will gradually replace the overlapping parts of their own apis with UDPA-DM apis, keeping only their own extension apis.

In terms of xDS evolution, V3 / V4 will move closer to UDPA, and v4 in particular will be based on UDPA. As the UDPA API is still in its infancy, and the USE of the UDPA API in xDS V3 is minimal (mostly annotation definitions), it is unclear which solution it is.

The following are the key udPA-DM designs described in the UDPA-DM design document:

  • Describes the L7 routing layer, which describes common behavior between major L7 DPLBS, including Envoy, HAproxy, Nginx, Google Cloud Platform URL Mapping, and Linkerd;
  • Provides flexibility for agent /LB specific extensions for behaviors that do not fit into a generic abstraction. Escape hatch (note: similar to SQL dialect)
  • Dsmt4 Provides L7 routing table scalability and effective logarithmic evaluation. For example, Envoy V2 xDS is a strictly linear evaluated routing table with significant scaling limitations. For DPLBS that support UDPA-TP, it should be possible to obtain routing table segments on demand.
  • Legacy users who support linear matching routing tables in the V2 Envoy xDS API;
  • Remove the need for multiple xDS style apis, such as RDS, VHDS, and SRDS;
  • The composability of resources; It should be possible to support UDPA federated use cases with independently managed resources such as virtual hosts;

The following focuses on the design of L7 Routing.

Routing API design

There are three terms in the Routing API:

  • Service /Service: An opaque string that describes a back-end Service (or in Envoy terms, Cluster /Cluster). The current documentation does not provide any further detail on service representation, which will be left for later work;
  • Routing table /Route Table: A set of matching conditions used for HTTP requests and related operations. Actions can include redirecting or forwarding to a specific service;
  • L7 routing /L7 routing: Evaluates the given HTTP request against the routing table;

In the design of UDPA-DM’s Routing API, a significant change has been made in the way of request matching compared with xDS, which is mainly reflected in supporting layered matching in addition to linear matching.

Here is an explanation of the request matching methods used in linear matching and hierarchical matching:

  • Linear: The routing table is similar to the [([Match], Action)] type. In this model, the routing table is an ordered list of matching criteria-action conditions. Each matching criteria is matched by the logic “with”, for example:
    • If :authority == foo.com AND path == / AND x-foo == bar THEN route to X
    • If :authority == bar.com AND x-baz == wh00t THEN route to Y
  • Hierarchical: Where routing tables are like tree structures and each node has (MatchCriteria, [(Value, Action)]) type. With this structure, any given node will evaluate only a single matching condition, such as the value of authority. Layered matching can provide relatively efficient implementation.

XDS V2 API currently uses linear matching, while UDPA-DM’s Routing API will introduce layered matching to form a linear-layered mixed matching, as shown in the following figure:

The design document describes this mixed-match model as follows:

The model does not map directly to any given DPLB today but borrows from some Envoy concepts and should have an efficient realization. It may be possible to use HAproxy ACLs to model this topology. Currently the model does not map directly to any given DPLB, but borrows some concepts from Envoy and should have a valid implementation. This topology might be modeled using HAproxy ACLs.

Since the Routing API has not been designed yet, nor has it officially become an UDPA API, in the newly finalized xDS V3 protocol, RoutesDiscoveryService and ScopedRoutesDiscoveryService didn’t introduce the new model, so the model is expected to continue to design and the finalized version in 2020, probably by the end of the xDS will be reflected in v4. Then, it becomes clear whether udPA-DM and xDS will be a transformation model or a subset model.

Since the Routing API has not been designed, the definition of the Routing API will not be expanded in detail here. The Routing API has been updated for a long time.

  • Issue Discussion [WiP] L7 Routing straw man;
  • In the PR submitted Routing API definition file udpa/config/v1alpha/Routing. The proto: can do a self and xDS v3 API;

conclusion

UDPA is still in the early stages of design, and the key UDPA-TP and UDPA-DM designs are in draft form but far from complete, and far from what we would like to see as a complete common data plane API. Moreover, the progress of the project is not ideal, and I feel that the level of attention and human input are limited.

P.S.

Recently, IN order to understand the progress of UDPA, I did research and study of UDPA. Unfortunately, UDPA materials are very scarce, except for several official websites and design documents listed in this paper, there is only Harvey’s speech.

After the survey was completed, UDPA’s progress was not satisfactory, especially with recent work almost stalled, the design of the key UDPA-TP and UDPA-DM was not completed, and only a few UDPA API definitions were referenced in xDS V3. There are too many unknown/pending/unfinished contents, and due to the lack of data input, much of the information is just my personal understanding and ideas, which should not be a rigorous in-depth introduction of the attitude of the article.

However, considering the lack of UDPA data at present, WITH the idea of “something is better than nothing”, I bravely finished this article. If there is any new input, I will timely improve or revise this paper. Students who are interested in and understand UDPA are also welcome to contact me for discussion and guidance.

The resources

  • github.com/cncf/udpa: UDPA’s project on Github, where the API definitions come from
  • Universal Data Plane API Working Group (UDPA-WG): UDPA design document, but very little content
  • Universal Data Plane API Working Group (UDPA-WG) : CNCF UDPA Working Group, you can see some information after joining
  • Udpa-tp strawman: Design document for UDPA-TP
  • Udpa-dm: L7 Routing strawman: UDPA-DM design document
  • Stable Envoy API Versioning: Official document for Envoy that describes the Stable API versioning strategy for Envoy
  • CNCF is working on a Common Data Plane API group to develop a standard API for the data plane: AN introduction to UDPA I wrote last year
  • The Universal Dataplane API (UDPA): Envoy’s Next Generation APIs: A talk by Harvey Tuch to help understand xDS and UDPA

Financial Class Distributed Architecture (Antfin_SOFA)