Recently, Li Bai, the technical expert of Geturen server, was invited to participate in the SegmentFault D-Day online technology live broadcast, and explored the “evolution path of back-end architecture” with back-end technical experts from the head Internet enterprises. With the theme of “EVOLUTION of API Gateway”, Li Bai shared the practical experience and deep thinking of GOLang-based API gateway construction.
★ The following is li Bai’s speech dry goods arrangement:
The origin of API gateways
API gateway is an architectural pattern that arose with the concept of “microservices”. In the process of microservice separation, the huge single application and business system is divided into many microservice systems for independent maintenance and deployment, resulting in the doubling of API scale and increasing difficulty of API governance. At the same time, when subsystems provide external capabilities through API, there will be the problem of repeated construction of common capabilities. As a result, it is becoming an architectural trend to use API gateways to publish and manage apis uniformly.
In Getuan, the company’s core business systems such as message push and financial risk control are all based on micro-service architecture, and some systems also have self-built gateway modules. With the increasing dependence between different systems, the need for efficient interface governance becomes more and more urgent. Therefore, Getuan introduced a unified API gateway early on to solve a series of problems such as permission control, flow control, service degradation, gray distribution, version management and so on.
1. Push the early API gateway
In 2015, the birth of SpringCloud greatly promoted the development and popularity of microservices architecture. It was also during this period that Geturen built its own true API gateway, relying on the SpringCloud Gateway.
SpringCloud is ecologically friendly and has many monitoring and fault tolerance components that can be used out of the box. However, because it does not support traffic and security governance, it cannot meet the requirements of subsequent companies for API governance. Especially after the construction of data center, there is an urgent need for a unified API gateway to undertake the data center entrance flow. In addition, in terms of development language, SpringCloud only supports Java, so its applicability is limited. Moreover, the performance is not ideal and the operation and maintenance is difficult.
Therefore, in order to achieve more powerful API governance capabilities, simplify access and operation and maintenance methods, and better adapt to the company’s data construction requirements, We choose our own API gateway.
2. Self-developed API gateway
2.1 Self-research objectives
The key goals for building an API gateway are:
Strategies 1) Follow uniform governance for the full LIFE cycle of an API
For example, API design should be unified and standardized, which stipulates that API must have owning services and labels, so as to achieve isolation in API design. After the completion of the design to be able to direct debugging, automatic generation of test code; Release, the API traffic to achieve fine control, support the grayscale release of services; In the process of API operation, we should be able to monitor and alarm the API call in an all-round way, and timely isolate the faulty services. Resources can be recycled in time after offline.
Strategies 2) Need to have sound functional components to process the repeat request
In the whole request link, we have designed and implemented a series of core functions such as link tracing, log, authentication, current limiting, fusing and plug-in.
Strategies for success 3) Ensure simple user migration and access while ensuring three heights
This means that the API gateway should be easy to operate and use, provide various metric detection capabilities, and support automatic fault tolerance and elastic expansion.
2.2 Technology selection
Based on the above objective design and technical research, we choose Golang as the main development language of our own API gateway. Golang was chosen for several reasons:
Strategies **1) ** PUSH Gproxy-CODIS and Gproxy-ES were designed during the process of making multiple machine room DISASTER recovery construction and data split migration, and a proxy cluster was built to route different users. These proxies have worked well so far, with QPS of more than 6W per unit in some large clusters. In the development process of these projects, we have accumulated a lot of development experience and basic components, many of the wheels are directly reusable. Therefore, it was much easier to develop a Gproxy-HTTP suite using Golang and building on these existing components.
Strategies for Development 2) The Golang language naturally supports high concurrency, rapid development, and most importantly machine cost savings.
Strategies for Development 3) Golang is the most used language for cloud native framework, technological precipitation on Golang is also paving the way for cloud native construction.
2.3 Design and implementation
After identifying goals and technology selection, some specific designs and implementations follow.
Strategies 1) Overall Structure
The overall architecture design of GEtui API gateway is shown in the figure below:
Firstly, it is a Web management platform, on which API creation, publication and subsequent management can be configured. After the configuration is complete, the configuration center sends a notification to the gateway, and the gateway periodically pulls the full API configuration. Then there are the core components of the gateway, such as the plug-in engine, which is the plug-in that performs the configuration. Forwarding engine is also the core module of API gateway. The forwarding engine of Getui supports HTTP, GRPC, and the GCF protocol developed by Getui. In the business scenario of data center, it also supports the data push capability of Kafka.
Strategies for Success 2) Plug-in service
As you can see from the figure, there is an independent plug-in service within the overall architecture of the Push API gateway. The core reason for this design is that Golang is a C-like language, which is packaged as an executable file. Golang’s native plug-ins are directly compiled and cannot be updated or uninstalled, so they cannot be added or updated directly on the interface. For this reason, we developed a plug-in service in Java that leverages Java’s dynamic language features to flexibly support the addition, update, and uninstall of plug-ins.
The gateway communicates with the plug-in service through GRPC, and there is a certain loss in performance. In order to minimize the performance loss, we use a native golang plug-in to implement encryption and serialization related plug-ins, and Java plug-in services are recommended for some components with strong business customization.
Strategies 3) Seclusion of resources
Resource isolation is a common means to achieve high availability of the system. The isolation design mainly includes cluster and thread pool isolation.
API gateways mainly support isolation of service clusters, through which cluster-level isolation can support multi-tenancy at the upper level and, if more thorough, isolation of gateway clusters at the LB level. In addition, grayscale publishing of services is also achieved through gateway cluster isolation. During the upgrade, you can configure traffic forwarding rules and clusters on the UI. Through traffic playback, some test traffic can be imported to the gray cluster, or real online traffic can be forwarded to the gray cluster in proportion to ensure that there is no problem before full release.
Thread isolation is mainly reflected in data services. The main function of data API is to provide data of MySQL, ES, and Hbase through API through simple configuration on the interface. It is very convenient for developers to write CRUD or client code. At present, most of the traffic in the data center business scenario is the request data service, so we designed three types of thread pools: common thread pool, slow thread pool and custom configured thread pool. When the request duration exceeds the slow threshold, the interface is allocated to the slow thread pool to prevent the slow request from dragging down the entire service.
Strategies 4) Program services
Service choreography is also a common requirement of API gateways. It is mainly to aggregate multiple apis into calls to greatly reduce call latency. This part of the functionality was previously on the gateway, and the service choreography supported at the gateway level is relatively basic and can support concurrent aggregate calls to apis, but difficult to handle complex business combinations, especially choreography scenarios involving transactions. Therefore, we decided to extract these functions as independent services, and the subsequent link gateway will directly access the service choreography module, which also ensures the overall lightweight of the gateway.
Strategies for Performance optimization
For API gateway performance, we also made a series of compression and optimization, such as using open source functions to replace or rewrite the internal use of serialization, encryption and decryption functions; Extensive use of sync.pool reuse objects, pure asynchronous processing of their internal logic; Developed GNET, replaced the original NET framework, optimized the network model and so on. From the actual online operation results, according to the current data, the daily call volume of TAichung API platform exceeds 1 billion times, the peak value of single QPS is around 2W, and the overall performance loss is 10%+, which exceeds expectations.
Strategies for Ease of design
Through the plug-in mechanism, isolation, and performance optimizations described above, we ensured that the API gateway platform as a whole was highly available and easily extensible. After the platform is completed, it should be convenient for users to access, use, and operation and maintenance.
Therefore, to improve ease of use, we adopted a pure Web interface design and built in multiple API templates. Users can create an API through simple configuration, such as interface authorization validity period, QPS, quota, etc. through the visual interface. Once the API is created, users can also debug directly on the interface.
In addition, when apis are provided externally, users can export API documents of a service in batches, which is very convenient. The individual push API platform also realizes monitoring and statistics functions, such as providing data such as the trend of API being called, the amount of API calling and error statistics under the whole service, which is friendly to operators and r&d personnel.
conclusion
To sum up, Getuian API gateway is based on golang’s independent research and development and fully Web configuration, realizing the standardization and visualization of all API interfaces. In addition to basic gateway requirements, it also supports plug-in hot update, multi-protocol conversion, data push, cluster level resource isolation and other advanced requirements. In addition to being integrated into the system’s microservice architecture, getuan API gateway also acts as a traffic entry point for the company’s data center, carrying billions of daily visits. At present, the new version of GEtui API Gateway cluster has been running stably for more than one year, and it continues to evolve while ensuring the stable operation of the system. In the future, we will explore more possibilities around cloud native to provide stable and efficient basic platform for business.