Author | Zhang added wings
Source |ERDA official account
Cloud original birth of opportunities and challenges
Standards and ecological significance
Since the release of Kubernetes v1.0 on July 21, 2015, and the subsequent establishment of the CNCF organization, both Kubernetes and CNCF have experienced rapid growth in the following years. Today, Kubernetes has entered the mature period, the cloud native era is just beginning. Although cloud native is not just around Kubernetes’ ecology, there is no doubt that Kubernetes is the cornerstone of cloud native ecology. By standardizing APIs and CRD standards, Kubernetes has established a cloud-native PaaS ecosystem and has become the de facto standard in the PaaS space.
This layer of factual standards has huge implications for enterprise delivery. Before the emergence of K8S ecology, there was a lack of unified standards for things like screws and nuts, like civil engineering. If Party A only focused on the upper business functions, it would be easy to build the platform on floating sand, which would lead to the overturning of the business. It is no exaggeration to say that in the field of enterprise delivery, it is really “the sky does not grow K8S, forever is like the night”.
Taking API routing function in API management as an example, if K8S is not used, enterprises may choose various gateway software such as F5/Nginx/ HAproxy /Zuul to do the corresponding routing configuration. Some software provides console UI, others may be human script operation and maintenance, lack of standards, operation and maintenance skills can not be deposited, the departure of key personnel may bring disaster. K8S abstracts the ability of API routing to the Ingress resource, defines the standard, and shields the underlying software details. Personnel certified by CNCF CKA will have the ability of API routing operation and maintenance. In the field of API management, in addition to API routing, there are also links such as API traffic governance strategy, API open authorization, and dosage observation and audit. K8S and ISTIO and other ecologies have provided some standard definitions. Although many of them are not yet mature, the future of standards and ecologies has become increasingly clear. Erda Cloud has been firmly committed to this path, providing enterprises with standards-compliant, trusted API management products.
Cloud native gateways are in contention
The gateway is a key component of the API management product. In the cloud native ecology, many different Ingress Providers have emerged for the K8S Ingress standard. Someone compiled the following table:
As can be seen from the table, the low-level gateway implementations of the Ingress Provider are: Nginx, HAProxy, Envoy, Traefik, Skipper.
Kubernetes Nginx Ingress, as the official implementation, has the largest user ecological group and is now the most used in mass production deployment. Before the appearance of cloud native ecology, Nginx has been the first gateway in the Web 2.0 era with high performance and easy expansion, and has a good development, operation and maintenance ecology, so it was selected by K8S as the default Ingress Provder, and then with Ingress, The ecology will be further expanded.
HAProxy has a similar experience with Nginx. Both of them emerged in the era of Web 2.0 and occupy a place in the gateway ecosystem with high performance and very low resource occupancy. Before combining with K8S Ingress, HAProxy and Nginx play more of a load-balancing role similar to ADC (Application Delivery Controller) access layer in the software system architecture. Even if the micro-service architecture is adopted, It is not directly responsible for the traffic exposure of the microservice, but usually adds another layer of microservice Gateway forwarding, such as the Spring Cloud Gateway. Since the microservice Gateway usually needs to communicate with the registry in the microservice architecture to realize service discovery, which is not the strength of HAProxy and Nginx, there is no better choice in the Spring Cloud microservice architecture than the Spring Cloud Gateway. However, K8S changes all this by providing a Service discovery mechanism with Ingress/Service/Endpoint and a health check mechanism built into the code, which enables a more generic and extensible Service discovery without language restrictions. This helps HAProxy and Nginx extend their tentacles into the microservice governance space.
In fact, before K8S, there had been gateway software trying to expand from the traditional load balancing field to micro-service governance, Kong is the most prominent one. With the help of OpenRESTY, based on the high performance and reliability of NGINX, it can realize the rapid development of business function plug-ins. So Kong has implemented its own service discovery and health check mechanisms.
However, it is a pity that after the appearance of K8S, the independent health inspection mechanism has been relatively weak, and Kong Ingress still retains this weak mechanism, thinking it is not a wise choice. In addition, Kong Ingress only supports its own Lua plug-in ecology in routing policy configuration, rather than the native NGINX configuration instruction. Although Kong’s plug-in ecology is already perfect, for example, one line of NGINX configuration instruction such as adding HTTP reply header can be done. Kong’s plugins are a bit of low-level duplication, jumbled and inefficient. However, Kong’s plug-in ecology is rich, especially the authentication and authentication functions are quite mature, which is also the main reason why Erda Cloud does not choose Kong Ingress, but still uses Kong to achieve part of the API management ability.
A software life cycle, if it is in line with the cycle of architecture ecological development, will burst out dazzling light. If Kong did not meet K8S at the best age, “I was born before you were born, you gave birth to me already old”. Envoy and Istio, complementary to each other, can be called a perfect pair. ISTIO extends the capabilities of an Envoy from managing the portal north-south to managing the traffic between micro-services east-west. The dynamic update mechanism of configuration based on XDS protocol is also better than HAProxy and NGINX. However, Envoy’s configuration files are described in JSON and are not very descriptive. So some things like Istio/Ambassador/Gloo abstract the configuration twice, which is easier to operate. Although Istio + Envoy has the most vision of the future, Gloo is currently the best in terms of gateway coverage based on configuration abstraction, but it still has a gap with Nginx Ingress, especially the ADC-like features. Because the abstraction level of the Envoy itself is not very high, the Ingress Providers which abstract to the Envoy for two times also cut off the ecology to some extent. To match the ecology of Nginx Ingress, the Providers of the Nginx Ingress are different from that of the Nginx Ingress. There’s still a long way to go.
Up-and-comers like Traefik and Skipper, based on Golang, can’t be ignored either. While the user base isn’t as good as the old guard, they have a good developer base, thanks to the language Golang makes it so easy to deal with concurrency and IO. Catering to developers also has two sides. For example, due to the existence of GC, it will become very difficult to maintain an ideal memory footprint under a large number of concurrent connections and requests. In order to achieve the performance of C/C++ gateway, the complexity of code design will be increased to a certain extent, and it will become no longer simple and reliable.
The reason is that no one’s products can fully cover the needs of users, from ADC-like products to micro-service discovery products to more business-related API management domains such as authentication. The Cloud native API management solution of Erda Cloud also adopts the two-tier architecture of Nginx Ingress + Kong. However, it is believed that with the development of standards and the expansion of gateway boundaries, there will certainly be a single-layer full-stack gateway software that can not only fully meet the requirements of low-level network operation and maintenance, but also well expand and support the business strategy of the upper layer, and it must be cloud-oriented native architecture.
The API manages the product’s cake and eat it
Around 2010, the API management products of Apigee and Mulesoft were introduced, and the concept of API lifecycle management was introduced, aiming to achieve the whole process of API monetization from design and development, to test and deployment, and then to open integration. To this day, consulting firms such as Gartner also regard API lifecycle management as a key part of enterprise digitization, and provide consulting services for enterprises by producing annual analysis reports. Here is the Gartner API Magic Quadrant for the full lifecycle management in 2020:
As you can see in the chart above, Apigee and Mulesoft are firmly at the top of the Leader Quadrant because of their first-mover advantage. It can be seen that Kong is also in the Leader Quadrant, even ahead of Apigee and Mulesoft on a strategic level, largely because of Kong’s rich business plug-in ecosystem and cloud-native design such as Kong Ingress, which is ahead of established API management products. However, compared to mature APIs like Mulesoft to manage SaaS products, Kong has not been as successful in productization.
Existing more mature API management SaaS products, all have their own implementation of the API gateway as the core. However, different from the cloud native Ingress gateway, these API gateways are not directly routed to the K8S service, but are built on the whole business system to do API management at the application system level. This also means that once you choose this type of SaaS product, you have to accept the overhead of an additional layer of forwarding over the network, possibly across the public network, and incurring additional TLS encrypted transmission overhead, which can add tens to hundreds of milliseconds of latency to the network transmission.
The full life cycle API management product provided by Erda Cloud not only utilizes Kong’s mature API management plug-in ecology, but also combines with Nginx Ingress to connect Cloud native applications more quickly and easily. Let users not have to make a choice between latency and API management capabilities. This is the biggest difference between Erda Cloud’s cloud-native API management product and other products on the market.
Erda Cloud Cloud native gateway
Introduction of architecture
Considering the function coverage, user ecology and enterprise production practice scale, Erda Cloud’s gateway product has chosen Nginx Ingress as the K8S Ingress Provider. Erda Cloud can host K8S clusters built by users (or using Cloud vendor K8S services). As long as the NGINX Ingress Controller version is higher than 0.25 installed in the cluster, you can directly use the Erda Cloud Cloud native gateway products.
Erda Cloud uses Kong as an extension that users can choose to install when they have business level API management needs such as API Open Authentication. When users configure the functional requirements of the corresponding domain name /API on Erda Cloud, Kong will automatically take over the corresponding Ingress and route it to the corresponding K8S Service again. Based on methods such as UNIX Domain Socket/EBPF SOCKMAP, the latency of NGINX INGRESS forwarding to KONG can be reduced to negligible.
Of course, adding a layer of proxy forwarding will certainly bring additional resource occupation, which is the deficiency of this architecture. However, it has the advantages of cloud native Ingress Provider for smooth access of users, the ability of full coverage from ADC-like function to business-layer API management function, and the flexible design that users can independently manage which traffic is forwarded through Kong. This trade off is worth doing.
Young Ingress Annotation length
NGINX Ingress can implement powerful traffic governance functions based on the Ingress Annotation, such as the ability to limit the request rate:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-example-host annotations: # each IP can access nginx 5 times per second. The ingress. Kubernetes. IO/limit - RPS: 5 # as the calculating speed of bucket algorithm into the size, and limit the multiplication coefficient of RPS nginx. Ingress. Kubernetes. IO/limit - into - multiplier: 5 # with limit - window said global speed limit of 100 times per second (need to rely on memcached) nginx. Ingress. Kubernetes. IO/global - rate - limit: 100 nginx.ingress.kubernetes.io/global-rate-limit-window: # 1 s limit sent to the back-end service delivery rate of 1 MB per second nginx. Ingress. Kubernetes. IO/limit - rate: 1024 # 10 MB data before sent to the back-end service without limit nginx. Ingress. Kubernetes. IO/limit - rate - after: 10240 spec: rules: - host: "foo.bar.com" http: paths: - path: "/bar" backend: service: name: bar port: number: 80
The request speed limit is the tip of the iceberg for Ingress Annotation capabilities. This simple configuration is not possible with many other gateways. For example, when using the Istio Ingress Gateway, it has to be achieved through the complicated EnvoyFillter configuration. Gloo, by contrast, simplifies some of the configuration, but also makes it available in the commercial version, which is not available in the open source version.
Erda Cloud further enhances the ability of Ingress Annotation. Taking the request rate limit as an example, the global traffic speed limit provided by Nginx Ingress relies on external memcached for synchronization between multiple instances of Nginx Ingress. Erda Cloud can be aware of the number of instances of NGINX INGRESS and automatically update the configuration of the speed limit. For example, if the Gateway Policy in Erda Cloud is configured with a request limit of 100 times per second, and the number of NGINX Ingress instances is currently 2, then the speed limit in Ingress is actually configured with 50 times per second. Not only that, but Erda Cloud also reduces the mental cost of doing such a configuration. Those familiar with Nginx should know that request rate limits are implemented based on the leaky bucket algorithm. Burst size is the key to achieving peak-filling. If the burst size is zero, the occasional request interval will also be denied. But ordinary users might not understand this mechanism, the Nginx Ingress Nginx. Ingress. Kubernetes. IO/limit – into – multiplier configuration items will be.
Let’s take a look at how Erda Cloud lets users do the configuration.
The gateway configuration for Erda Cloud replaced the flow-limited Burst configuration with a configuration for maximum additional latency based on a simple equation:
Burst Size = maximum additional latency (seconds) * maximum throughput (requests per second)
Maximum additional delay is much easier to explain: When the request rate exceeds the limit, peak filling is performed, and additional delay is added to the current request. If the additional delay required exceeds the limit, access is denied directly.
Nginx Ingress does not provide ready-made annotations that support the global rate limit of this local mode, but it is still possible to customize Nginx configuration based on snippet annotations:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-example-host annotations: # will add the following under the corresponding server {} nginx configuration snippet nginx. Ingress. Kubernetes. IO/server - the snippet: | ###ERDA-AUTO-CONFIG### location @server-guard-65feeb { more_set_headers 'Content-Type: text/plain; charset=utf-8'; return 429 "System is busy, please try it later."; } # # # # # # ERDA - AUTO - CONFIG# will be in the corresponding location under add {} to nginx configuration fragments nginx. Ingress. Kubernetes. IO/configuration - snippet: | ###ERDA-AUTO-CONFIG### limit_req zone=server-guard-65feeb burst=5; limit_req_status 581; error_page 581 = @server-guard-65feeb; ###ERDA-AUTO-CONFIG### spec: rules: - host: "foo.bar.com" http: paths: - path: "/bar" backend: service: name: bar port: number: 80
When a request is rejected, it will return a 581 status code internally, jump to a custom named location via error_page, and then return the rejection status code and the rejected response body in the named location.
Since the limit_req_zone directive can only be configured within NGINX’s HTTP block, the Erda Cloud updates the corresponding Ingress Annotation, You also need to update the corresponding Configmap nginx-configuration:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configuration
namespace: kube-system
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
data:
http-snippet: |
###ERDA-AUTO-CONFIG###
limit_req_zone 1 zone=server-guard-65feeb:1m rate=100r/s;
###ERDA-AUTO-CONFIG###
As such, the configuration generated by Erda Cloud is wrapped in the ### erda-auto-configurable ### comment blocks. Outside of these comment blocks, users can still add custom configuration items that are not affected by the configuration of Erda Cloud.
In this example, you’ve already seen the power of NGINX and Ingress in combination, using snippets to make the most of the power of NGINX native configuration on Ingress. This is also the appeal of the cloud native, which takes Nginx configuration operations to a whole new level, better decoupling and better adapting to microservice architectures. Erda Cloud takes this one step further by making these configurations even less costly to the minds of R&D and operations personnel through product packaging.
Take Kong’s strength for Ingress’s weakness
Although Nginx Ingress can realize micro-service discovery and routing relying on K8S Service, it lacks the ability of micro-service API management because it cannot support routing control at the level of HTTP Method. For example, if a service only wants to expose a specific API GET/API /assets/{assetID} to the public, the routing mechanism based on Nginx Ingress cannot be implemented. This requirement can be easily implemented on Erda Cloud’s Cloud native gateway.
First, add the API to be exposed for the corresponding service in the microservice API management. The following figure exposes only one API GET/API /assets/{assetID}.
Then specify the domain URL routing to the micro service, such as domain name for the helloworld. Erda. Cloud, according to the following configuration, when the request the helloworld. Erda. Cloud/API/example/assets / 100, The gateway rewrites the request path to/API /assets/100 and forwards it to the back-end microservice.
This functionality is achieved with the help of KONG. When the user creates the microservice API, it will associate the microservice with Kong’s Route/Service. When the user configures a route on a specific domain name, an Ingress is created that is forwarded to Kong:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /c517f0f2/api/assets$1
nginx.ingress.kubernetes.io/upstream-vhost: dev.gateway.inner
nginx.ingress.kubernetes.io/use-regex: "true"
name: erda-route-ed71b6-6de9f1
namespace: addon-kong-pb8210
spec:
rules:
- host: helloworld.erda.cloud
http:
paths:
- backend:
service:
name: kong
port:
number: 8000
path: /api/example/assets(.*)
pathType: ImplementationSpecific
With the help of Kong’s ability, Nginx Ingress also makes up for its lack of authentication ability. Kong provides rich AuthN plug-ins, and Erda Cloud mainly uses Oauth2, Key Auth, and HMAC Auth three plug-ins. You can uniformly manage the calling credentials of callers to different AuthN plug-ins in the Erda Cloud:
You can delegate the entire domain name, or a specific route under the domain name, to different callers:
Open validation of APIs is a key capability for enterprises to manage APIs. Erda Cloud hopes to better meet this requirement and make it easier for providers to open their APIs. The caller can apply for the call independently, automatically obtain the certificate and SLA quota limit; Thus, the efficiency of API management can be improved as a whole. Erda Cloud has launched a product for API lifecycle management based on this Cloud native gateway capability.
The Erda Cloud API manages products
The Erda Cloud API management product consists of three modules: API Design Center, API Marketplace, and API Access Management.
The interface provider designs the interface in the API Design Center, and then publishes it to the API Marketplace. The caller of the interface looks for the interface in the API Marketplace and applies for the calling certificate and SLA quota. The interface provider, or manager, authorizes caller access in API access management and can audit and analyze request traffic from individual callers.
API Design Center
Some API design products on the market will require users to fully understand the Swagger 2.0/ OpenAPI 3.0 protocol, to type out a document word for word; Other products support APIs designed in a more friendly way, but do not use a standard protocol, and their output documentation loses its generality.
The Erda Cloud API Design Center is based on visual editing, intuitive and friendly interface, users do not need to know any REST API specification standards, or any knowledge of the API description language, can easily write a professional level API document. At the same time, the OpenAPI 3.0 protocol standard can be delivered and removed at any time, once designed, everywhere used; API documents hosted on other platforms, swagger files generated in code, etc., can also be easily migrated.
The Erda API Design Center hosts API documentation in a code repository, which is designed to correlate interface descriptions with interface implementation code. The developer enters the code repository, selects the appropriate branch of code, and maintains the interface documentation, which is a good way to keep the documentation in sync with the newly developed functionality. This philosophy follows the idea that GITOPS configuration is code.
Hosting documents into a repository also means that documents can be collaborated on a branch-based basis. When different users write the same document, they simply cut a new branch from the source branch, edit the document on the new branch, and then merge the branches. Principals of different interfaces of the same service can design their own interfaces at any time, and merge them back at any time, without mutual influence and blocking.
The API mart
API marts are portals to APIs for publishing, documenting, exposing, and debugging APIs. The API provider publishes the designed document to the marketplace for subsequent management and monitoring. API users access, explore, debug, and subscribe to APIs in the API Marketplace. Marketplaces are places where API owners and users complete “transactions.”
API Marketplace uses semantic versioning mechanism to implement API document versioning. The format of version number is Major. Minor. Patch, where:
- A major is a major version number. A change in the major version number usually indicates a significant change or a change that is not backward compatible.
- Minor is the minor version number, and a change in the minor version number usually indicates that new features have been added and are still backward compatible.
- Patch is a revision number, and the change of revision number usually represents a minor and partial revision to the existing version.
In addition to the semantic version number, there is a version tag called “version name,” which is typically a self-explanatory word or phrase that indicates the name of the current document version. The version name uniquely corresponds to the major in the semantic version number. The version name can be considered an alias for the major version number. For example, a MacOS version number, Big Sur 11.2.2, where “Big Sur” is the version name of the operating system, “11” is the major version number corresponding to the version name, and 2.2 is the minor version number and revision number respectively. The benefit of versioning in this way is that it treats the growth of the API document as much as the growth of the application, allowing you to look at the functionality of the application from an API perspective. The version number explains the compatibility and dependencies between changes in the service, so that owners and consumers can clearly understand the changes in the service based on the version number semantics.
API resources can be associated with specific service instance addresses on the Erda Cloud. Through such association, the API provider can further implement the access management of the API, and the caller can apply for calling and testing the interface in the API Marketplace.
API Access Management
API access management refers to the management, protection, and audit of the API being called while opening the interface to the outside world. When an API is exposed to the outside world, it is often necessary to identify, constraining, and audit client behavior, and then create access management for API resources. The specific means of access management are client authentication and SLA quota management.
After associating an API resource with a specific service instance address on Erda Cloud in the Marketplace, the API provider creates an access management for the API resource, so that callers can apply for calling the API in the API Marketplace. The provider receives the invocation request and approves it, setting SLA quota for the client; Approved clients gain access credentials and can access the interface from the outside. The caller can then switch API versions in access management and forward requests to service instances of different versions, thus upgrading or rolling back the API versions without the customer being aware of it.
The functionality of API access management is based on the capabilities of the Erda Cloud native gateway product, which is much simpler than using the configuration capabilities of the gateway directly — users only interact with the API.
API lifecycle management best practices
The Erda Cloud based API management product implements best practices for API lifecycle management. As shown in the figure below, you can view the API from both the provider’s and caller’s perspectives.
From the perspective of the API provider: firstly, it is necessary to update the documents of the API Design Center in time as the service function changes. Because the documents are also based on code warehouse management, the timely synchronization of the API documents can be ensured through code review. During the development co-tuning phase, the API provider can publish the API document to the marketplace, and other module functions that rely on this interface can be developed in parallel. If there is a need to open the API to the outside world, the API provider will set up the access management function for the corresponding API resource, and the external call traffic can be observed in real time in the access management console.
From the point of view of the API caller: If you are a test engineer, you should design the test cases for the automated interface based on the API documentation provided by the developer, rather than maintaining a test specific interface documentation. If it is an external integrator, the required functional interface should be found through the API Marketplace. After the successful application call, a simple interface access test should be conducted in the API Marketplace to confirm that the function meets expectations. Then, according to the API document, the code of the integration module is written and deployed. Finally, you can view the invocation traffic in My Access.
Software iterates and changes over its life cycle, and so do APIs. Both API providers and callers need to pay attention to the impact of API iteration. The provider should strictly follow the semantic version mechanism of the API Marketplace. When Breaking Change occurs, an independent access management entrance should be created for the new Major version, and the old version mark should be discarded to guide the caller to use the new version. Callers should keep an eye on subscription notifications to see the latest version of the API documentation being used.
Erda Cloud’s DevOps feature provides CI/CD capabilities in Cloud native scenarios, and API management should be considered part of CI/CD. You can use the automated test platform of Erda Cloud to connect the API marketplace and add automated interface tests to the CI process. You can use Erda’s pipelined extension to automatically publish the API version after the CD process and automatically associate the K8s Service address with the Service.
Erda Cloud provides one-stop service for enterprises to build their system architecture based on Cloud native. The API management of Erda Cloud is also a product that grows naturally on the soil of Cloud native. API lifecycle management is a key link of enterprise digitalization. If an enterprise adopts the cloud-native architecture, it must choose the corresponding API management products, otherwise it may lead to the increase of adaptation cost and low management efficiency.
Learn more about the latest information about Erda Cloud products from our official website, erda. Cloud. Also welcome to follow our GitHub Repo, our code has been all open source, looking forward to your star ~~~
Welcome to Open Source
As an open source one-stop cloud native PaaS platform, Erda has platform-level capabilities such as DevOps, micro-service observation governance, multi-cloud management and fast data governance. Click the link below to participate in open source, discuss and communicate with many developers, and build an open source community. Everyone is welcome to follow, contribute code and STAR!
- Erda Github address:https://github.com/erda-project/erda
- Erda Cloud Website:https://www.erda.cloud/