Comparison and selection of common API gateway, and explain our company’s own micro service gateway, dry goods Full!

Previous selections:

• 2 years experience summary, tell you how to do well project management
• Java full set of learning materials (14W words), took half a year to sort out
• Message queuing: From model selection to principle, this article takes you through everything
• I spent three months writing the GO core manual for you
• RPC framework: From principle to selection, this article will guide you to understand RPC
• More…

Hello everyone, I am Lou Zai! Microservices are very popular in recent years, and there are many technological ecosystems around microservices, such as microservice gateway, Docker, Kubernetes, etc.

I contact micro service gateway was started in 2019, at that time, and the company’s development with a colleague, due to technical ability is limited, I only responsible for gateway background, subsequent iteration of micro service gateway, I actually didn’t participate, but later took time off to see the micro service gateway at the front desk of the code, so the principle of the micro service gateway is the basic control.

Recently, WHEN I was writing an article about technology stack, I just wrote about microservice gateway, so I simply summarized the knowledge I had learned before, and also sorted out the commonly used microservice gateway in the market, on the one hand, it was convenient for the selection of subsequent technology, on the other hand, it also gave myself an explanation. Here is the table of contents:

API Gateway Basics

What is an API gateway

The API gateway is a server that is a unique entry point to the system. From an object-oriented design perspective, it is similar to the appearance pattern.

The API gateway encapsulates the internal architecture of the system and provides a custom API for each client. It may also have other responsibilities, such as authentication, monitoring, load balancing, caching, protocol transformation, current-limiting fuses, and static response processing.

The core point of the API gateway approach is that all clients and consumers access microservices through a unified gateway and handle all non-business functions at the gateway layer. Typically, gateways also provide REST/HTTP access apis.

Main functions of the gateway

As a unified entrance of micro-service back-end services, micro-service gateway can manage back-end services as a whole, which is mainly divided into data plane and control plane:

• The data plane is used to aggregate HTTP requests and microservices of access users after they are split. The microservice gateway is used to expose the APIS and contracts of back-end services. Routing and filtering functions are the core capability modules of the gateway. In addition, microservices gateway can implement interception mechanism and focus on cross-cutting functions, including protocol conversion, security authentication, fusing limiting, gray publishing, log management, traffic monitoring, etc.
• The control plane centrally manages and configures back-end services. For example, you can control the elastic scaling of gateways; You can deliver configurations in a unified manner. You can tag gateway services; The Swagger function can be configured in the micro service gateway to uniformly expose the API contracts of back-end services to users, complete document services, improve work efficiency and reduce communication costs.

• Routing: Routing is the core capability of the microservices gateway. The routing function microservice gateway can forward requests to the target microservice. In the microservice architecture, gateway can combine the dynamic service discovery of the registry to realize the discovery of back-end services, and the caller only needs to know the service API exposed by the gateway to access the back-end microservices transparently.
• Load balancing: API gateway combines load balancing technology and uses service discovery tools such as Eureka and Consul to implement load balancing for downstream services through polling, specified weights, and IP address hash.
• Unified authentication: in general, no matter the network or the network interface to do the user identity authentication, the user authentication in the bigger system will adopt a unified Single sign-on (Single Sign On) system, if every micro service docking Single sign-on system, then obviously is a waste of resources and low efficiency of the development. API gateway is an excellent place for unified security management, which can extract the authentication part to the gateway layer. Microservice system does not need to pay attention to the authentication logic, but only to its own business.
• Protocol transformation: ONE of the main functions of API gateway is to build heterogeneous systems. As a single entrance, API gateway integrates micro-services based on different styles and implementation technologies such as REST, AMQP and Dubbo through protocol transformation, and provides unified services for specific clients such as Web Mobile and open platform.
• Indicator monitoring: The gateway can count the number of requests to back-end services and update the current traffic health status in real time. It can delay statistics on services with URL granularity. It can also use the Hystrix Dashboard to check the traffic status of back-end services and whether a circuit break has happened.
• Current limiting fuses: In some scenarios, the number and frequency of client access needs to be controlled. In some high-concurrency systems, the number of client access needs to be limited. A threshold can be configured on the gateway to return an error when the number of requests exceeds the threshold. When traffic peaks occur or back-end services are delayed or faulty, the gateway proactively switches the switch to protect back-end services and ensure good front-end user experience.
• Blacklist and whitelist: The microservice gateway uses the system blacklist to filter HTTP request characteristics and block requests from abnormal clients, such as DDoS attacks that erode bandwidth or force service interruption. The microservice gateway can block and filter requests at the gateway level. A common interception policy is to add a blacklist based on IP addresses. In a route service that has authentication management, you can configure the whitelist to directly access the backend service resources without authentication management.
• Grayscale publishing: The microservice gateway can control the traffic according to the special marks in the HTTP request and the metadata identification of the back-end service list, so as to complete grayscale publishing without the awareness of users.
• Traffic dyeing: Similar to the principle of gray publishing, the gateway can dye the request according to the Host, Head, Agent and other identification of HTTP request. With the traffic dyeing function of the gateway, we can track the subsequent invocation link of the service and further analyze the service delay and service running status.
• The gateway, combined with Swagger, can expose back-end microservices to the gateway. As a unified entrance, the gateway provides interface users with THE API specification for viewing back-end services without knowing the Swagger address of each back-end microservice. In this way, the gateway has the effect of aggregating back-end APIS.
• Log audit: The microservices gateway can act as a unified log logger and collector, intercepting log request information and response information of service URL granularity.

API Gateway selection

Common API Gateways

Let’s take a quick look at the common API gateways on the market:

Nginx

Nginx is a high-performance HTTP and reverse proxy server. Nginx on the one hand can do reverse proxy, on the other hand can do static resource server, interface using Lua dynamic language can complete flexible customization functions.

After Nginx is started, there will be a Master process and multiple Worker processes. Master process and Worker process interact with each other through interprocess communication, as shown in the figure. Worker The choke point of a Worker process is at I/O multiplexing function calls such as SELECT (), epoll_wait(), etc., waiting for a data read/write event to occur. Nginx handles requests in an asynchronous, non-blocking manner, which means that Nginx can handle thousands of requests simultaneously.

Zuul

Zuul is Netflix’s open source API gateway component that works with Eureka, the Ribbon, Hystrix, and more. The community is active and integrated into the complete ecology of SpringCloud, which is one of the best models to build the front gateway service of micro-service system.

At the heart of Zuul is a series of filters that do the following:

• Unified authentication + Dynamic routing + Load balancing + pressure test
• Review and monitor: Track meaningful data and statistical results with edge locations, resulting in an accurate production view.
• Multi-region elasticity: Request routing across AWS Regions is designed to diversify the use of The Elastic Load Balancing (ELB) and bring the edges of the system closer to users of the system.

Zuul is currently available in two major versions: Zuul1 and Zuul2

Zuul1 is built on a Servlet framework, as shown in the figure, using a blocking and multithreading approach, in which one thread processes a connection request, which can lead to an increase in live connections and threads in cases of high internal latency and device failures.

Netflix has released a major update to Zuul2, which runs on an asynchronous and non-blocking framework, one thread per CPU core, and handles all requests and responses. The lifecycle of requests and responses is handled through events and callbacks, which reduces the number of threads and therefore costs less.

Spring Cloud GetWay

Spring Cloud Gateway is a new API Gateway project for Spring Cloud designed to replace Zuul1, It is based on Spring5.0 + SpringBoot2.0 + WebFlux (Reactor model based on high performance responsive communication framework Netty, asynchronous non-blocking model) and other technology development, performance is higher than Zuul, official test, Spring Cloud GateWay is 1.6 times larger than Zuul and aims to provide a simple and effective unified API route management approach for microservices architectures.

The Spring Cloud Gateway can be used together with Spring Cloud Discovery Client (such as Eureka), Ribbon, and Hystrix to implement route forwarding, load balancing, fusing, authentication, path rewriting, and log monitoring. Gateway also has a built-in flow limiting filter to achieve the function of flow limiting.

Kong

Kong is a highly available, easily extensible API Gateway project written by Mashape based on OpenResty (Nginx + Lua module).Kong is built on NGINX and Apache Cassandra or PostgreSQL, can provide easy to use RESTful API to operate and configure the API management system, so it can horizontally scale multiple Kong servers, through the pre-load balancing configuration to evenly distribute requests to each Server, to cope with a large number of network requests.

Kong has three main components:

• Kong Server: Nginx-based Server used to receive API requests.
• Apache Cassandra/PostgreSQL: Used to store operational data.
• Kong Dashboard: An official recommended UI management tool. You can also use restfull to manage the Admin API.

Kong uses a plug-in mechanism for functionality customization, where a set of plug-ins (which can be 0 or N) is executed during the lifecycle of the API request response cycle. The plug-in is written using Lua and has several basic functions: HTTP basic authentication, key authentication, CROSS-origin Resource Sharing (CORS), TCP, UDP, file logging, API request flow limiting, request forwarding and Nginx monitoring.

The Kong gateway has the following features:

• Scalability: Horizontal scaling is easy by simply adding more servers, which means your platform can handle any request with a lower load;
• Modularity: Can be extended by adding new plug-ins that can be easily configured through the RESTful Admin API;
• Run on any infrastructure: The Kong Gateway can run anywhere. You can deploy Kong in a cloud or internal network environment, including single or multiple data center Settings, as well as public, private, or invite-only APIs.

Traefik

Træfɪk is a modern HTTP reverse proxy, load balancing tool designed to make deploying microservices easier. It supports multiple backends (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Rest API, File…) To automate and dynamically apply its configuration file Settings.

Important features:

• It is fast, requires no additional dependencies, and is a single executable written in the Go language;
• Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd;
• Supports Rest apis, Websockets, HTTP/2, and Docker images.
• Listen for changes in the background to automate the application of new profile Settings;
• The configuration file is hot updated without process restart.
• Back-end circuit breakers, load balancing, fault tolerance mechanism;
• Clean front page for monitoring service indicators.

For more information on Traefik, check out traefik.cn

API Gateway Comparison

Here is a screenshot of the gateway comparison, so let’s focus on Kong, Traefik and Zuul:

• In terms of open source community activity, Kong and Traefik are definitely the best.
• In terms of maturity, Kong, Tyk and Traefik are better.
• In terms of performance, Kong is a bit ahead of the others;
• In terms of the scalability of the architecture advantages, Kong and Tyk have rich plug-ins, Ambassador also has plug-ins but not many, while Zuul needs to be completely developed by itself. However, Zuul is also highly used due to its deep integration with Spring Cloud. In recent years, Istio service grid has become popular. Ambassador’s seamless integration with Istio is also a big advantage.

Here are the thinking conclusions of other netizens for reference:

• Performance: the Nginx+Lua form is necessarily better than the Java language implementation of the gateway, the Java technology stack Zuul1.0 is servlet-based implementation, the rest is based on webflux implementation, performance is better than the servlet-based implementation. In terms of performance, I think the choice of gateway is probably not that important, just a few more machines can do it.
• Maintainability and extensibility: the combination of Nginx+Lua is not well understood by many people. If the team has a big god, it is more important for the average team to choose the language that their team is good at. Zuul and Spring Cloud Getway require more or less integration and configuration pages to maintain, but for Soul I have no brain to look at the article, which needs to move which good, especially can be brain-free connection Dubbo Beautiful. In addition, Soul2.0 can get rid of ZK, there is no problem in my heart, I love mindless operation.
• High availability: The gateway high availability is basically the same policy is to use the multi-machine deployment mode, the front of the load, for the external need to use some components you pay attention to.

Traefik’s own microservices gateway

This is the micro-service gateway developed by our company, which is developed based on Traefik. The following explains the technology selection, gateway framework, gateway background and protocol conversion, which is absolutely dry!

Technology stack selection

• Traefik: An open source reverse proxy and load balancing tool that integrates directly with common microservices and enables automated dynamic configuration. The Traefik is lightweight, easy to use and set up, and performs well enough to be used in production environments worldwide.
• Etcd: A distributed, highly available, consistent key and value storage system written in Go to provide reliable distributed key and value storage, configuration sharing, and service discovery. (More on ETCD from Raft Principles to Practice)
• Go: Strong concurrency, performance comparable to C, processing power is 4 times of PHP, high efficiency, simple syntax, easy to use, development efficiency close to PHP.

The gateway framework

The whole gateway framework is divided into three parts:

• Gateway backend (HAL-FE and HAL-admin) : used to configure applications, services, and plug-ins, and then publish the configuration information to ETCD;
• Traefik: reads the ETCD configuration and distributes routes based on the configuration information. If authentication is required, the HAL-Agent module implements unified authentication. After authentication, Http requests are directly sent to downstream services. If Grpc or Thrift protocol is used, protocol conversion is performed through the HAL-Proxy module.
• Protocol conversion module: Reads ETCD configuration, performs Grpc and Thrift protocol conversion on Traefik distributed requests (see article RPC Framework for more information: From principle to selection, this article will take you through RPC), and through the service discovery mechanism, obtain the downstream service machine, and through load balancing, the converted data to the downstream service machine.

The gateway background

It is mainly composed of three modules:

• Application: mainly includes application name, domain name, path prefix, group, status, etc., such as Indian overseas mall, Indian community;
• Service: includes service name, registration mode, protocol type, owning group, and status, such as comment service, address service, and search service.
• Plug-in: includes the plug-in name, plug-in type, and plug-in attribute configuration, such as path prefix replacement plug-in and authentication plug-in.

An application can bind only one service, but can bind multiple plug-ins. After configuring the gateway on the background, generate the Config file and publish it to ETCD. The Config file must comply with the strict data format. For example, Traefix configuration must comply with the official file configuration format to be identified by Traefik.

Protocol conversion module

Hal-proxy module is the most complex and technical module in the whole microservice gateway, so I will explain it to you in detail.

Problem is introduced into

Before we talk about this module, let’s look at the following questions:

• When a request comes in from upstream TrAFIK, you need to know the IP and port of the machine accessing the downstream in order to send the request downstream. How do these machines get it?
• With the machine, we need to establish a connection with the downstream machine, if the connection is used once directly released, it will certainly cause great pressure on the service, which requires the introduction of Client cache pool, then how to achieve the Client cache pool?
• Finally, the protocol needs to be transformed. Because different downstream services support different protocol types, how does the gateway dynamically support this?

Realize the principle of

Hal-proxy: Resolver: Hal-Proxy: Resolver: Hal-Proxy: Resolver: Resolver Here I make a brief introduction, at present, the company internal through service access to the machine list many ways, such as MIS platform, service, tree, etc., also is to have a plenty of through the platform configuration, have a plenty of directly hung in service under the tree, either way, we are all through the service name, in a certain way, find the service below all of the host.

Therefore, the function of the Resolver module is to find the IP addresses and service ports of all machines under the service based on the service name, and then store them in the memory persistently and update them periodically.

The protocol module supports different protocol conversions. The conversions of each protocol type need to be independently implemented. The protocol conversions are nothing more than initializing the Client through the machine IP address and port, and then converting the data and sending it directly to the downstream machine.

Finally is the connection pool, we also use the go actually bring the pool before to do, but when to pool data update, need to lock, so performance has been not to, later changed to circular queue, and then by atomic operation approach, all to the operation of the data had been achieved on the lock operation, greatly improve the concurrency performance. The ring queue code, also arranged for you, can be directly read this article Go language core manual -10. Atomic operation.

Implementation logic

This is the logical implementation diagram of HAL-Proxy, which has been drawn for two days, including all the interaction modes of core objects. I won’t go into details here, but you can understand how much you can master by yourself. If you have any questions (or can’t see the picture clearly), please follow my official account and add my wechat to communicate.

It is better to have no books than to have no books. Because of my limited ability, it is hard to avoid omissions and mistakes. If you find bugs or have better suggestions, you are welcome to criticize and correct.