Background:

With the growth of the business volume, the capacity of a single data center cannot support all the traffic. Meanwhile, the defect of the traditional cold backup mode in the computer room prompted the emergence of Ele. me remote live. The API-Router, as an important component of multiple activities, plays an important role in routing the incoming traffic from the client to the back-end equipment room. When an equipment room fails or even breaks down, the equipment room can be switched over quickly.


Design objectives:

1. Entrance traffic can be routed to the corresponding machine room (ezone, to be exact) according to Sharding key rules;

2, with the machine room switching rules, can realize the flow transfer;

3, as far as possible to achieve high performance, high availability, high concurrency, protection of back-end services, peak filling;

The first two points are the main functions of the API-Router. Thirdly, the problems to be solved in the face of the challenges brought by the exponential increase in business volume of Ele. me; Therefore, apI-Router is positioned as an HTTP reverse proxy and load balancer on which routes are distributed.


I. Implementation of traffic routing

Api-router is deployed on the cloud in clusters based on services. Referring to the overall architecture of Ele. me, the physical architecture of apI-Router is as follows:


1. Api-router obtains the mapping between ezone of each domain name maintained and data center address from Router console;

2. Api-router will subscribe to GZS routing table from SHard to machine room Ezone;

3. Api-router In memory, the Sharding key (geographic location, merchant ID, order ID) in the request header is calculated according to a set of Sharding rules. The most common regular algorithm is to calculate the corresponding polygon based on the user’s latitude and longitude through the ground fence algorithm, and this polygon is allocated a shard in advance;

3. According to the previous 3 steps, the apI-router has sharding key > Shard >ezone > data center mapping table, and finally forwards the request to the corresponding data center.


Implementation of traffic diversion:

When the back-end equipment room is faulty, the API-Router will actively push the message about the mapping change between Shard and EZone according to GZS to switch traffic to eZone, thus ensuring high availability of services.


Ii. Technical Implementation

Api-router adopts the netty-based asynchronous non-blocking model of the whole back end to minimize the network loss of requests at this layer. Epoll enables it to support a higher number of connections. At the same time, some measures are taken to protect the back end services to ensure the stable operation of the whole service link.


1. Peak cutting and valley filling:

As can be seen from the architecture diagram, apI-Router is at the entrance of the website, so peak-cutting and valley filling of traffic at this layer can greatly protect back-end services from instantaneous traffic impact.

When each HTTP request comes, it will first calculate the back-end upstream according to routing rules, and each upstream corresponds to a scheduler. Then, the scheduler limits concurrency through semaphores and accumulates requests through queuing, thus achieving traffic shaping, as shown in the diagram below:


2. Connection multiplexing

With the increase of ele. me users, the number of connections between the client and the router machine is increasing. In view of the impact of TCP connection on the performance, the connection pool reuse technology is adopted between the router and the back end, so the number of connections to the back end is reduced (take a certain machine as an example, the number of connections between the front end and the router is about 10W, Reduced to 140+ after connection multiplexing to the back end), as shown in the diagram below:


3. Hot load filter to process request & Response

Api-router, as a Proxy at the front of the website link, has many requirements to process requests and responses. To this end, we implemented a chain of interceptor functions, each interceptor called Filter, which can intercept request and response, and then do some processing on them like flow limiting, fusing, anti-crawler, whitelist, cross domain, etc. Each filter is delivered dynamically from the console, modified according to a custom ClassLoader implementation and then loaded dynamically into the JVM, reducing the number of releases. The schematic diagram of Filter is as follows:



4. Multi-protocol support

Api-router currently supports including

  • http/https
  • websocket
  • grpc
  • http2
  • TCP

5 or more

In order to further improve the traffic per second carried by a single node and reduce resource loss, we also achieve non-blocking and improve performance from the following aspects.

  • Asynchronous log
  • Asynchronous heartbeat
  • Asynchronous dot
  • Asynchronous Filter
  • Asynchronous link creation

Looking ahead:

1. Realized minute-level dynamic capacity expansion and reduction of server resources to improve resource utilization based on peak and trough fluctuations of Ele. me takeout orders;

2, to further provide high system availability, anti-climb, anti-brush, anti-network attack;


The authors introduce

Luo Hui, who joined Ele. me in 2015, is now a senior engineer in ele. me Framework Tools Department, responsible for the ApI-Router project of Ele. me.