When you decide to use your application as a set of microservices, you need to decide how the application client will interact with the microservices. In a singleton application, there is usually only a set of redundant or load balanced service providers. In a microservice architecture, each microservice exposes a fine-grained set of service delivery points. In this article, we look at how it affects client-to-server communication and propose an API Gateway approach.
1, the introduction
Suppose you are developing a native mobile client for an online shopping application. You need to implement a product final page to display product information.
For example, the figure below shows what you see when you swipe through the final page of a product on the Amazon Android client.
Although this is a smartphone app, the final page of the product displays a lot of information. For example, not only do you have basic product information (name, description, and price), but you also have the following:
- The number of items in the shopping cart
- Order history
- User comments
- Low inventory warning
- Delivery options
- A variety of recommendations include what is often bought with the item, what is bought by other customers who buy the item, and what other products are viewed by customers who buy the item.
- Optional shopping options
When using a single type application architecture, a mobile client will request (GET api.company.com/productdeta through a REST… To get this data. A load balancer distributes requests to one of multiple application instances. The application queries various databases and returns requests to the client.
In contrast, with a microservice architecture, the data on the final page is distributed across different microservices. Here are some of the microservices that might be relevant to product final page data:
- Shopping cart service – The number of items in a shopping cart
- Order service — Order history
- Classification services – basic product information, such as name, picture, and price
- Review service – User reviews
- Inventory service – Low inventory alert
- Delivery services – Delivery options, deadlines, costing from different delivery apis
- Recommend services — Recommend products
1. Direct communication between clients and micro services
In theory, a client can directly make a request to any one of multiple microservices. Every micro service has a foreign server (serviceName.api.com pany. Name). This URL may map to the load balancer of the microservice, which in turn forwards requests to specific nodes. In order to search for product details, the mobile terminal needs to send requests to these microservices one by one.
Unfortunately, this scheme has many difficulties and limitations. One of the problems is the mismatch between client demand and the number of fine-grained apis exposed by each microservice. In the figure, the client requires seven separate requests. In more complex scenarios, more requests may be required. For example, Amazon’s final product page requests hundreds of microservices. Although a client can initiate many requests over the LAN, this is inefficient on the public network, especially on the mobile Internet. This scenario also leads to very complex client code.
Another problem is that the protocol for clients to request microservices directly may not be Web-friendly. One service may use the Thrift RPC protocol, while another may use the AMQP message protocol. None of them are browser-friendly or firewall-friendly, and are best used internally. The application should use a protocol like HTTP or WEBSocket outside the firewall.
Another disadvantage of this solution is that it is difficult to refactor microservices. Over time, we may need to change the current shard scheme for system microservices. For example, we might need to merge two services or split one service into multiple services. However, this refactoring can be difficult to implement if the client interacts directly with the microservice.
Because of these three problems, direct communication between the client and the server is rarely used in practice.
2. Use an API Gateway
In general, a better solution is to adopt the API Gateway approach. The API Gateway is a server, or the only node that enters the system. This is much like the Facade pattern in the object-oriented design pattern. The API Gateway encapsulates the architecture of the internal system and provides apis to individual clients. It may also have other capabilities such as authorization, monitoring, load balancing, caching, request sharding and management, static response processing, and so on. The following figure shows an API Gateway that fits the current architecture.
The API Gateway is responsible for request forwarding, composition, and protocol transformation. All requests from clients are routed through the API Gateway and then routed to the corresponding microservice. The API Gateway will often process a request and aggregate the results of multiple services by invoking multiple microservices. It can convert between Web protocols and non-Web-friendly protocols used internally, such as HTTP and WebSocket.
The API Gateway provides a custom API to the client. It exposes a coarse-grained API to mobile clients. Take the product final page usage scenario as an example. The API Gateway provides a point of service provision (/ productDetails? Productid = XXX) enables mobile clients to retrieve all data from the final page of the product in one request. The API Gateway processes this request by invoking multiple services and returning results, including product information, recommendations, reviews, and so on.
A good example of an API Gateway is the Netfix API Gateway. The Netflix streaming service offers hundreds of different microservices, including TVS, set-top boxes, smartphones, gaming systems, tablets and more. Initially, the Netflix view provided an API for all scenarios. However, they found this format difficult to use because of the variety of devices involved and their unique requirements. Now, they use an API Gateway to provide a fault-tolerant API with code for different types of devices. In fact, an average of six to eight back-end services are invoked by an adapter to process a request. The Netflix API Gateway handles billions of requests every day.
2. APIGateway
1. Unified external interface:
When users need to integrate functions between different products or services, they invoke the capabilities provided by different services. APIGateway allows users to assemble services with a unified interface without being aware of service edges. For different services within the company, the interface provided may have some differences in style. APIGateway can unify these differences. When internal services are modified, APIGateway can be used to adapt, without requiring the caller to adjust and reduce the exposure of services to increase system security.
2. Unified authentication:
APIGateway is used to authenticate access in a unified manner. Each application does not need to authenticate callers separately, and applications can focus on services.
3. Service Registration and Authorization:
You can control what services callers can and cannot use.
4. Service flow limiting:
APIGateway allows callers to call each interface daily and total number of calls
5. Full link tracking:
The unique request Id provided by APIGateway monitors the flow of the call and the response time of the call.
3. Advantages and disadvantages of API Gateway
As you might expect, there are pros and cons to adopting API Gateway. One of the biggest benefits of the API Gateway is that it encapsulates the application internals. It is easier for the client to interact directly with Gatway than to invoke the specified service. The API Gateway provides each client with a specific API, which reduces the amount of client-server communication and simplifies the client code.
The API Gateway also has some disadvantages. It is a highly available component that must be developed, deployed, and managed. Another problem is that it can be a bottleneck in development. Developers must update the API Gateway to provide new service delivery points to support newly exposed microservices. Updating the API Gateway must be as lightweight as possible. Otherwise, developers will queue up for the Gateway update. However, despite these drawbacks, the API Gateway approach works for most applications.
Open source API gateway framework
1, spring zuul:
Zuul is a Netflix load balancer based on JVM routing and server side. Its main features are: authentication, stress testing, Canary testing, dynamic routing, load reduction, security, static response processing, and active/active switch management. Spring Zuul is a spring Cloud component that can be used with various Spring Cloud components.
The overall components of SpringCloud include: Zuul, Ribbon, EureKa, Fein, Hystrix, etc. Zuul is an apigateway-like architecture, Ribbon is a proxy server similar to Nginx, Eureka is used for registration and discovery services, and Hystrix can be used as an architecture disconnect service for service degradation. Fein can serve as a PROVIDER of Rest services that can be invoked internally between services.
2, kong
Kong is an off-the-shelf API Gateway solution developed on Nginx.
API Gateway can be implemented in many ways, such as using NIO based frameworks such as Netty, Vertx, Spring Reactor, JOSS Undertow on the JVM. One non-JVM-based comparison of processes right now is NodeJs. Others include Nginx Plus.
Implement an API Gateway
Now that we know the motivations and pros and cons of adopting the API Gateway, let’s look at what you need to consider when designing it.
1. Performance and scalability
Only a handful of companies have to handle the scale of Netflix, which handles billions of requests a day. However, the performance and scalability of the API Gateway are also important for most applications. Therefore, it makes sense to create an API Gateway that supports synchronous, non-blocking I/O. There are different techniques that can be used to implement an extensible API Gateway. On the JVM, use a NIO based framework such as Netty, Vertx, Spring Reactor, or JBoss Undertow. Node.js is a popular non-JVM platform built on top of Chrome’s JavaScript engine. One alternative is NGINX Plus. NGINX Plus provides a mature, scalable, high-performance Web server and reverse proxy that are easy to deploy, configure, and redevelop. NGINX Plus manages authorization, permission control, load balancing, caching, and application health checking and monitoring.
2. Adopt reactive programming model
Some requests can be handled by the API Gateway by routing the request directly to the corresponding back-end service. For other requests, it calls multiple back-end services and merges the results to process them. For some requests, such as product end page requests, the requests to back-end services are independent of each other. To minimize response time, the API Gateway should process independent requests concurrently. However, sometimes there are dependencies between requests. The API Gateway may need to validate the request through an authorization service before routing it to a back-end service. Similarly, to get a customer’s product wish list, you need to get the user’s profile and then return information about the products on the list. One such API component is the Netflix Video Grid.
Code that uses traditional synchronous callback methods to implement API merges can land you in a callback nightmare. This code would be very difficult and difficult to maintain. An elegant solution is to adopt reactive programming patterns. Similar reactive abstraction implementations are Scala’s Future, Java8’s CompletableFuture, and JavaScript’s Promise. On the Microsoft.NET platform are Reactive Extensions(Rx). Netflix created RxJava for the JVM environment to use their API Gateway. Similarly, JavaScript platforms have RxJS, which runs in browsers and node.js platforms. Taking a reactive programming approach can help you quickly implement an efficient API Gateway code.
3. Service invocation
A microservices-based application is a distributed system and must adopt the mechanism of interthread communication. There are two ways to communicate between threads. One is an asynchronous, message-based approach. Such implementations are JMS and AMQP. Others, such as Zeromq, belong to direct communication between services. There are also synchronization mechanisms for inter-thread communication, such as Thrift and HTTP. In fact, a system can use both synchronous and asynchronous mechanisms. Because it can be implemented in many ways, the API Gateway needs to support multiple modes of communication.
4. Service discovery
The API Gateway needs to know the IP and port of each microservice. In a traditional application, you might hardcode these addresses, but in today’s cloud-based microservices applications, this would be a simple problem. The underlying service typically takes a static address, which can be specified using operating system environment variables. However, detecting the address of the application service is not so easy. Application services typically assign addresses and ports dynamically. Similarly, service instances can change dynamically due to extensions or upgrades. Therefore, the API Gateway needs to adopt the system’s service discovery mechanism, either server-side or client-side discovery. A follow-up article will cover this section in more detail. If the client discovery service is used, the API Gateway must query the service registry, which is a database of microservice instance addresses.
5. Deal with some failures
Another issue to consider when implementing the API Gateway is partial failure. This problem occurs in distributed systems when one service calls another that times out or becomes unavailable. The API Gateway should not be blocked and wait indefinitely for downstream services. However, how this failure is handled depends on the specific scenario and the specific service. For example, if it is the recommendation service module on the product details page that is not responding, then the API Gateway should return the rest of the information to the user because it is also useful. The recommended section can be returned empty or a fixed top 10 can be returned to the user. However, if the product information service is not responding, then the API Gateway should return an error to the client.
The API Gateway should be able to return the cache when it is valid. For example, because the product price changes infrequently, the API Gateway should return the value in the cache when the price service is not available. This data can be cached by the API Gateway itself, or it can be implemented by an external cache such as Redis or Memcached. By returning cached or default data, the API Gateway ensures that system errors do not affect the user experience.
Netflix Hystrix is a very useful library for implementing remote service invocation code. Hystrix records calls that exceed preset limits. It implements a circuit break mode that stops the client from waiting endlessly for unresponsive services. If a service’s error rate exceeds a preset value, Hystrix interrupts the service and all requests become invalid immediately for a period of time. Hystrix can define a fallback operation for request failures, such as reading cache or returning default values. If you are using the JVM, you should consider using Hystrix. If you are working in a non-JVM environment, you should consider using libraries with similar functionality.