Abstract: Now we are talking about “distributed”, “micro-service”, “cloud native” these concepts, more in the support of “software services” elastic scaling and load balancing. API Gateway as the first gate and its important component, let’s take a look at its development history, current situation and future direction.

As an important part of modern distributed, microservice, cloud native systems and as an important topic of discussion, WE will also talk about load balancing and the current state of API Gateways.

Nowadays, when people talk about the concepts of “distributed”, “microservice” and “cloud native”, they are not only talking about the “software service” function itself, but also talking about how to make the service can be better extended to support large-scale applications. Load balancing and API gateway are the first gate and an important component of it. Let’s take a look at its development history, current situation and future direction.

Load balancing

When it comes to gateways, you have to talk about load balancing, which usually comes in two different ways: server load balancing and client load balancing (such as Spring CloudRibbon & Eureka). Client load balancing is a common choice for non-REST services with high real-time requirements, such as Honor of Kings and League of Legends, where the daily activities entering the game are based on REST services, and teams playing matches are usually non-REST services. There are also online collaboration products, such as Welink’S IM/Presence function, which can be used as REST services. Welink Meeting’s real-time collation service cannot meet the real-time requirements of RESE. Most of them use client load balancing. The basic process is as follows:

1. Registration or discovery mechanism between load balancing gateways and servers.

2. The client requests a list of conference servers from the gateway, where the server has some business logic to calculate a list of servers to return to the client.

3. After receiving the server list, the client sends probe packets to the server and decides which server to choose according to the response time of probe packets (the network status between the client and the server) and the load of the server.

4. The client establishes a service connection with the server through a specified protocol.

5. If any part of the client or server is abnormal, the client will go through the process 2-4 to recover the service connection.

It can be seen from the above that customer load balancing will have a relatively complex process, but once the connection is established, its performance must be optimal. However, client load balancing cannot guarantee the REST servitization of the server. Once the server fails, the service will be interrupted for a short time. However, this scheme is suitable for some applications with high real-time requirements (some applications mentioned above). For REST services, L4 load balancing (F5, LVS, Nginx, HaProxy, etc.) /L7 load balancing (HaProxy, Nginx, Apache, Mysql Proxy, etc.) is a common method. At this Arch Summit, some vendors introduced their 4-tier load balancing solutions. Let’s take a look at the overall architecture and development history of load balancing in distributed systems.

From the perspective of the development of load balancing, its evolution process is roughly as follows according to its application scale:

The state of the API Gateway

With the popularity of microservices, API Getway has flourished. The API Gateway’s job is to deliver messages and provide customers with a unified API entry. Backend for Frontend API Gateway And SingleAPI Gateway are used. Is there a third model? The basic gateway architecture of a product I made before is as follows:

1. The client has login requirements. The response of login authentication contains the URL list of apis of different services.

2, the client in different API request is, use the corresponding API URL, so that the client to achieve most of the API distribution work.

3. At this point, the main task of the API gateway is not the original API forwarding function. It is to simplify the back-end services and implement some common functions of the back-end services.

4. Even in this scenario, there can be no API gateway, and the API can directly face the application service, which then schedules the microservice for business processing.

Back to the API Gateway itself, the core design concept is to ensure uninterrupted business on the data side. Because of the diversity of services that connect to the API gateway and the uncontrolled design of client apis and applications, it is difficult to require fault tolerance for every service and client that you access. This requires the gateway to ensure the normal processing of each request as far as possible, and meet a high SLA. Currently, the mainstream use of Nginx system API gateway in the industry is mainly considered as follows:

  • Hot restart

Upgrading components on the data side is a high-risk operation that can lead to a broken connection and system failure unless your front-end LB (load balancers) is capable of draining quickly, but even then it can force a break on the requests being processed. So a hot restart on the data side is critical.

  • Supports subscription dynamic routing

API routing changes frequently and requires high timeliness. If the regular synchronization scheme is adopted, the simultaneous synchronization of tens of thousands of data will slow down your system. Therefore, it is very critical to add a subscription routing service center. And it’s not too stressful to just take incremental data for performance.

  • Support plug-in management

Nginx provides a rich ecosystem of plug-ins. Different apis and different users require different processing flows. If each request is processed according to the same flow, redundant operations are bound to occur. Plug-in management can improve performance to a certain extent and also ensure that processing chains can be added quickly during the upgrade process.

  • High-performance forwarding capability

API gateways generally work in the multi-back-end API reverse proxy mode, and many self-developed API gateways are prone to bottlenecks in performance, so nGINx’s excellent performance and efficient traffic throughput are its core competitiveness.

  • Stateless can scale horizontally

API gateway is the collection of all requests of the whole system, which needs to be flexibly scaled according to the business scale. Service center and Nginx configuration management can be used to quickly add and delete existing clusters, and synchronize to LVS to achieve rapid horizontal expansion.

With the increasing complexity of today’s systems, many service modules, in addition to handling their own business, also take on some non-business responsibilities, such as authentication and authorization, traffic limiting, caching, logging, monitoring, retry, fuse, etc. A number of open source API gateway implementations have sprung up.

  • Tyk: Tyk is an open source, lightweight, fast and scalable API gateway that supports quotas and speed limits, authentication and data analysis, supports multiple users and multiple organizations, and provides full RESTful apis (GO language).

  • Kong: Kong can be considered an OpenResty application, and OpenResty runs on top of Nginx, extending Nginx with Lua. (Kong = OpenResty + Nginx + Lua)

  • Orange: Orange is an OpenResty-based API Gateway. It monitors and manages apis and user-defined rules, such as access statistics, traffic segmentation, API redirection, API authentication, and WEB firewall. (Tencent is using it)

  • Netflix Zuul: Zuul is an edge service that provides dynamic routing, monitoring, resiliency, security and more. Zuul is a Netflix load balancer based on JVM routing and server side. (Spring Cloud)

  • Ambassador: Ambassador is an open source microservices API gateway built on Envoy Agents to support rapid publishing, monitoring and updates for multiple teams of users, handling functions such as Kubernetes Ingress Controller and load balancing. Seamless integration with Istio. (Kubernetes-Native)

  • The other… Apiaxle: an API gateway implemented by Nodejs; Api-umbrella: an API gateway implemented by Ruby.

In addition to the above capabilities, as API gateway services take on more responsibilities, their performance bottlenecks become more prominent. In this ArchSummit, some companies showed some features of their own, and in the evolution of products, they have made some optimization in the architecture, mainly including:

  • Implementing gateways in C++ to improve performance (*)

— at this conference, two or three companies are proposing C++ implementations to improve performance. This is basically less about architecture and more about programming language differences.

  • Proprietary protocols speed up the mapping between apis and services

— Several companies are doing this, such as Tencent.

  • The gateway achieves hierarchical isolation of changeability and changeability, and realizes the architecture evolution of value-added services of the gateway (*)

— This is the architecture side of things, as I understand it is more about architecture evolution and operations. The part facing the front-end customer (stable) and the back-end service (unstable) is implemented and deployed in layers, so that the customer-facing gateway service is basically unchanged. When back-end services are extended or reconstructed, system upgrades have minimal impact on customers (details are not covered).

  • The gateway implements traffic limiting, making back-end services more stable and simpler.

— This is easy to understand and the general practice. This makes back-end application service/microservice design and implementation easier. Of course, different product implementations are different. When Tencent introduced the game API gateway, it mentioned that the service started static creation to maximize connection Session, and removed dynamic creation and recycling mechanism to achieve optimal performance.

  • Gateway implements authentication, simplifying the business process of back-end services (suitable for authentication, not permission)

This is also a common practice to make the back-end application service/microservice design and implementation easier. This service is suitable for authentication, but permission management may not be suitable for some special application scenarios. For example, a certain function in some applications has a fine division of permissions, which is already content-level access control. Gateways generally cannot be implemented in place of services and need to be implemented through services themselves.

conclusion

From the perspective of the development of gateway, it has experienced generation after generation of evolution, from the evolution of its own architecture, to the superposition of its functions, to promote the iterative evolution of its architecture. In this era of everything in the cloud, the concept of cloud native has been accepted by various industries and raised to a high level. Even some traditional network-equipment businesses are moving to the cloud.

For the development and evolution of products, we will “copy, learn, change”.

  • For the same business, mature and excellent solutions, we should be able to “copy”, directly take over to use, do not build their own wheels behind closed doors.

  • For the transformation and evolution of different businesses, there is not much experience that can be used for reference, but the architecture and knowledge of related fields. We should not copy, but “learn”, learn its ideas, its essence.

  • Finally, there is change. Any common solution or architecture can solve common problems, but because of common problems, there are inevitably some “common problems”.

Appendix: Arch Summit API Gateway architecture diagram:

Click to follow, the first time to learn about Huawei cloud fresh technology ~