1. The background

In a distributed environment, there are more and more cluster deployments, and the call links between services are getting longer and longer. Each service has its own error code. If a service is abnormal, it is difficult to quickly and clearly locate the fault through the error code. Besides, error codes are scattered among various services and cannot be centrally managed. If each service handles them by itself, the coupling degree is too high. Based on this background, we made the enhanced gateway BetterGateway.

Summary of 2.

In view of the above disadvantages, combined with the current distributed popular design, we decided to do specific transformation at the gateway. Every request must pass through the gateway to reach the business cluster. After being processed by the business cluster, the request is returned to the caller through the gateway again. In order to achieve centralized management of configuration files, we adopt Nacos as the configuration center in view of the popular configuration center. The error code is stored in Nacos, and the dynamic refresh function of Nacos is used to realize the dynamic change of error code without stopping. At the same time, we can also support the configuration file stored in the file system, using NIO to listen for changes in the file, to achieve dynamic refresh, as shown in Figure 1.1.



Figure 1.1

In order to accurately and quickly locate the exception through the error code, we made the following analysis: To quickly locate the exception through the error code, it is necessary to specify a special error code assembly rules, from the error code, it can be seen at a moment which service provider under which function which interface exception. To do this, we specified the format of the error code as shown in Figure 1.2.

Figure 1.2

, moreover also guarantee the uniqueness of error code, the error code is the error code after assembly (please readers to distinguish the error code of each service, and assembly after the difference between the error code and error codes below are used to replace each service error code, if there is any place need to use the assembly after the error code, the author will write clear). Error codes may overlap under each service provider. To ensure uniqueness, we have different rules for different network calls:

Network invocations between services are complex and fall into two broad categories.

  • 1. Internal service invocation. Service name + error code.
  • 2. Three-party interface (external network) calls. Request URL + error code

For an internal service invocation, the internal service defines a bunch of error codes. For example, 1000, 1001, each internal service is a service provider. For example, service A calls the Submit interface of service B, and if the interface is abnormal, the error code returned is 1000. That is, service PROVIDER B returns 1000. Then the service is a service provider, and the service name + error code is the unique identification. For a three-party interface (external service) call, we use the request path + the status code returned by the request as the unique identifier. For example, service B calls the query interface of company XXX. This interface returns 200 on success, 1000 on failure (invalid parameters), and 1001 (authentication failure). In this case, XXX company is a service provider, and the query interface + return status code is the only identification.

Function of 3.

The serial number function Current support
1 Error code conversion Square root
2 Heat configuration (Nacos) Square root
3 Hot configuration (file system) Square root
4 Dynamic authentication Square root
5 forwarding x
6 distribution x
7 retry x
8 The compensation x
9 Dynamic return value x

Design of 4.

4.1 Overall Design

BetterGateway is based on Spring Cloud Alibaba and Spring Cloud design. The configuration file can also be stored in the local file system. The examples in this article are based on Spring Cloud Alibaba.

BetterGateway overall design

The request goes first to the gateway, which forwards it to the downstream system, as shown in the figure above. Gateway calls system A, system A calls system B. If system B is abnormal, system B assembles error codes (mainly service name and error code). At this point, there are two options for system B:

  • An exception is thrown, handled by the global exception handling class, and returned to the upstream system.
  • Handle the error by yourself, and then return the error code to the upstream system.

System A receives the result returned by the interface and determines whether it fails. If it fails, system A also has two options at this time:

  • An exception is thrown, handled by the global exception handling class, and returned to the upstream system.
  • Handle the error by yourself, and then return the error code to the upstream system.

Do not change the error code returned by system B in system A. Do not change the unique identifier (service name + error code or three-party interface path +code). After being processed by system A, the request comes to the gateway, and the gateway determines whether the return value is successful. If it fails, the error code is assembled from the configuration file and returned to the upstream system.

4.2 Function Components

4.2.1. Configuration Files
  • Total configuration of the service provider
    [{"code":"01"."name":"order-service"."dataId":"order-service.json"."domainName":"order-service"."authMethod":"token"."authConfig": {"ignorePath": ["/order/submit"."/order/cancel"]."ignorePathPrefix": [
           "/common"]},"type": "inner"."version": "5"}]Copy the code
    • Code Code of the service provider.
    • Name Name of the service provider.
    • DataId Indicates the ID of the service provider.
    • DomainName Indicates the domain of the service provider.
    • AuthMethod Default authentication method.
    • authConfigAdditional authentication mode.
      • IgnorePath Unauthenticated path (full path).
      • IgnorePathPrefix Indicates the unauthenticated path (prefix).
    • Type Indicates the type of the service provider.
    • Version Indicates the version number and the service provider error code.
[{"url": "order-service"."featCode": "001"."errorCodeList": [{"errorCode": "5106"."code": "001"."type": "P"."tips": "Current user does not have permission to operate this node"."handleStrategy": ""."handleParam": ""}]}, {"url": "order-service"."featCode": "002"."errorCodeList": [{"errorCode": "5012"."code": "001"."type": "P"."tips": "Change order has been canceled."."handleStrategy": "forward"."handleParam": "/viewCancelOrder"
      },
      {
        "errorCode": "5013"."code": "001"."type": "P"."tips": "Order being processed"."handleStrategy": "forward"."handleParam": "/viewDoingOrder"}]}, {"url": "/system/institution/dealer/search"."featCode": ""."errorCodeList": [{"errorCode": ""."code": ""."type": ""."tips": ""."handleStrategy": "distribute"."handleParam": "http://10.98.14.80/system/institution/dealer/search"}}]]Copy the code
  • Url Indicates the interface address (internal service is the service name).
  • FeatCode Specifies the feature domain number.
  • errorCodeListError code list.
    • ErrorCode errorCode (internal).
    • Code Error code.
    • Type Indicates the error type.
    • Tips Error message.
    • HandleStrategy Processing strategy.
    • HandleParam handles policy parameters.

By default, the error message overrides the MSG property in the original response. This can be extended with placeholders. Currently supported placeholders are:

  • # APPCODE#. For use inmsgShows the converted error code.
  • # MSG#. For use inmsgDisplays the original response information.

The distribute policy addresses inbound requests and is independent of error code mapping. Therefore, when configuring the distribution policy, the URL is set to the inbound request path and the other error code properties are set to empty.

4.2.2 Processing policy configuration

Supported Policies

  • Distribute distribute
    • HandleStrategy distribute.
    • HandleParam Indicates the destination address of a distribution. Multiple addresses can be separated by commas (,).
  • Forward Forward (currently unavailable)
    • handleStrategy forward.
    • handleParamDestination address for forwarding.
  • Retry (not available)
    • handleStrategy retry.
    • handleParamRetry times, retry interval.

4.2 Related modifications

Through the above introduction, the error code has been able to quickly and accurately identify which service exception. If a service is abnormal, the error code is transmitted through to the gateway. The call link is displayed as shown in Figure 1.3.

Figure 1.3

As the saying goes, a single wire does not make a line, and a single tree does not make a forest. The practical application of BetterGateway requires system modification and coordination, so we suggest using the following structure. As a uniform return value between service invocations, a custom exception (ApiException), and a matching global exception handler (APIExceptionHandler).

2 structure

@Data
public class ApiResult<T> implements Serializable {
  private static final long serialVersionUID = 0xc93480e15321b2c5L;

  private static final Logger logger = LoggerFactory.getLogger(ApiResult.class);

  /** * Status code */
  private Integer code;

  /** * Description */
  private String msg;

  /** * error code */
  private String appCode;

  /** * Service name /url */
  private String path;

  /** * Return data */
  private T data;

  public ApiResult(a) {
    this.path = PropertyUtils.getServiceName();
  }

  public ApiResult(ResultCode code) {
    this.code = code.code();
    this.msg = code.message();
    this.appCode = String.valueOf(code.code());
    this.path = PropertyUtils.getServiceName();
  }

  public ApiResult(Integer code, String msg) {
    this.code = code;
    this.msg = msg;
    this.appCode = String.valueOf(code);
    this.path = PropertyUtils.getServiceName();
  }

  public ApiResult(ResultCode resultCode, T data) {
    this.code = resultCode.getCode();
    this.msg = resultCode.getMsg();
    this.appCode = String.valueOf(resultCode.getCode());
    this.path = PropertyUtils.getServiceName();
    this.data = data;
  }

  public static <T> ApiResult<T> failure(Integer code, String path) {
    ApiResult<T> result = failure();
    result.setCode(code);
    result.setAppCode(String.valueOf(code));
    result.setPath(path);
    return result;
  }

  public static <T> ApiResult<T> failure(Integer code, String path, T data) {
    ApiResult<T> result = failure();
    result.setCode(code);
    result.setAppCode(String.valueOf(code));
    result.setPath(path);
    result.setData(data);
    return result;
  }

  public static ApiResult<String> failure(ResultCode code, String data, boolean isCustomErrorMessage) {
    ApiResult<String> result = failure(code, data);
    if (isCustomErrorMessage && Toolkit.isValid(data)) {
      result.setMsg(data);
    }
    return result;
  }

  public static <T> ApiResult<T> failure(APIException e) {
    ApiResult<T> result = failure();
    result.setAppCode(e.getAppCode());
    result.setPath(e.getPath());
    result.setMsg(e.getMsg());
    result.setData((T) e.getData());
    return result;
  }

  public static <T> ApiResult<T> success(a) {
    return new ApiResult<T>(ResultCode.SUCCESS);
  }

  public static <T> ApiResult<T> success(T data) {
    ApiResult<T> result = new ApiResult<T>(ResultCode.SUCCESS);
    result.setData(data);
    return result;
  }

  public static <T> ApiResult<T> success(T data, String msg) {
    ApiResult<T> result = new ApiResult<T>(ResultCode.SUCCESS.code(), msg);
    result.setData(data);
    return result;
  }

  public static <T> ApiResult<T> failure(ResultCode rc) {
    StackTraceElement se = Thread.currentThread().getStackTrace()[2];
    logger.error("failure result code: {}, msg: {}", rc.getCode(), rc.getMsg());
    logger.error("failure info: {} {} {}", se.getClassName(), se.getMethodName(), se.getLineNumber());
    return new ApiResult<T>(rc.getCode(), rc.getMsg());
  }

  public static <T> ApiResult<T> failure(a) {
    return failure(ResultCode.FAIL);
  }

  public static <T> ApiResult<T> failure(ResultCode code, T data) {
    ApiResult<T> result = failure(code);
    result.setData(data);
    returnresult; }}Copy the code

You can also use a custom ApiResult as required, but the following attributes are required:

  • Code: Integer Specifies the status code of the type. It is the original form and does not affect the original function.
  • AppCode: String status code, which can store richer information and non-int status codes of third-party interfaces. The converted status code overrides this value.
  • MSG: prompt message. The prompt message will overwrite this value after conversion.
  • path: Abnormal path. It is generally agreed that the third-party interface error is the interface URL, and the internal service invocation is the service name. You can also use other names in the configuration file.

    The difference between code and appCode does not affect the original interface. If the code of the ApiResult is a String, you can use it directly without appCode. Status code mapping is performed only when code is not successful. The successful status is configured in the ResultCode enumeration class in the common package. The default value is 200.

4.2.2 Custom Exceptions (ApiException)

@Data
public class APIException extends RuntimeException {

  private String appCode;
  private String path;
  private String msg;
  private Object data;

  public APIException(a) {}public APIException(String appCode, String path) {
    this.appCode = appCode;
    this.path = path;
  }

  public APIException(String appCode, String path, Object data) {
    this.appCode = appCode;
    this.path = path;
    this.data = data;
  }

  public APIException(String appCode, String path, String msg, Object data) {
    super(msg);
    this.appCode = appCode;
    this.path = path;
    this.msg = msg;
    this.data = data;
  }

  public APIException(ApiResult apiResult) {
    this.appCode = apiResult.getAppCode();
    this.path = apiResult.getPath();
    this.msg = apiResult.getMsg();
    this.data = apiResult.getData(); }}Copy the code

4.2.3 Global Exception Handling (APIExceptionHandler)

@RestControllerAdvice
@Order(Ordered.HIGHEST_PRECEDENCE)
public class APIExceptionHandler {

  private static final Logger logger = LoggerFactory.getLogger(APIExceptionHandler.class);

  @ExceptionHandler(value = APIException.class)
  public ApiResult<Object> handleBizException(APIException e) {
    logger.error("An interface call exception occurred because :", e);
    ApiResult<Object> result = new ApiResult<>(ResultCode.FAIL);
    result.setAppCode(e.getAppCode());
    result.setPath(e.getPath());
    result.setMsg(e.getMsg());
    result.setData(e.getData());
    returnresult; }}Copy the code

5. Important classes

5.1 the filter

  • CacheRequestBodyFilter Cache request body.
  • 【RemoveCachedBodyFilter】 Releases the cached request body.
  • GlobalExceptionHandler Exception handling.
    • Non-gateway service is abnormal. For this kind of exception, the gateway needs to perform error code conversion.
    • Gateway exception. The gateway is also a Java process and may have exceptions itself.
  • ErrorCodeFilter Error code conversion.
  • ErrorCodeSpringFilter. Unlike the above, this class takes advantage of the approach provided by the Spring-cloud-Gateway.
  • [AuthFilter] The policy mode implements dynamic authentication for different service providers.

5.2 Changes to the Configuration File

  • To store the latest ConfigDataChangeContainer 】 【 configuration file.
  • Operating ConfigDataChangeContainer CoreContainerService 】 【 class for contrast data, pull the new data.
  • 【NacosClient】 NacOS client, pull the new configuration file from nacOS.
  • [FileClient] File reading client. Pull the configuration file from the file system. If you want to start local file reading, dataId is the full path.
  • [NacosConfChangeHandler] Nacos listening handler class, used to accept files configured in the configuration center.
  • [FileConfigChangeHandler] This class is used to monitor File changes and read files.

5.3 Several special interfaces

  • DataFormatConversion this interface is used to notify configuration file changes, sending the latest configuration file to the class that implements this interface.
  • CompensationService This interface is used to process the CompensationService if the request fails.
  • [AuthService] Authentication interface.

Test 6.

In the test case, there are four roles: front-end system, gateway, sys-service (system service), API-service (external service interface service), and third-party data interface. As shown in figure 1.4.

Figure 1.4

The following examples show exceptions between different services and tripartite interfaces, and error codes after gateway translation, in different cases.

6.1: The SYS service is normal

http://localhost:81/sys/sysNormal front directly request sys – service, sys – service without exception.

6.2: THE API service is normal

http://localhost:81/sys/apiNormal front directly request sys – service, sys – service invocation API – service, API – service without exception.

6.3 The SYS service is Abnormal

http://localhost:81/sys/sysException front directly request sys – service, sys – service an exception occurs, the gateway conversion error code.

6.4 API Service Is Abnormal

http://localhost:81/sys/apiException front directly request sys – service, sys – service invocation API – service, API – service an exception occurs, the gateway conversion error code.

6.5 An API Call to an External Interface Fails

http://localhost:81/sys/apiCallBaiduException front directly request sys – service, sys – service invocation API – service, API – service invocation tripartite external interface, external interface is unusual, Gateway translation error code.

6.6 An API Call to an External Interface Fails because No Error code is configured

http://localhost:81/sys/apiCallBaiduException front directly request sys – service, sys – service invocation API – service, API – service invocation tripartite external interface, external interface is unusual, If error code conversion is not configured for the interface, the original error code is displayed.


Nanjing S300 Cloud Information Technology Co., LTD. (CH300) was founded on March 27, 2014. It is a mobile Internet enterprise rooted in Nanjing, currently located in Nanjing and Beijing. After 7 years of accumulation, the cumulative valuation has reached 5.2 billion times, and has been favored by many high-quality investment institutions at home and abroad, such as Sequoia Capital, SAIC Industrial Fund, etc. S300 Cloud is an excellent domestic independent third-party SaaS service provider of auto transaction and finance, which is based on artificial intelligence and takes the standardization of auto transaction pricing and auto financial risk control as the core product.

Welcome to join s300 Cloud and witness the booming development of the automobile industry together. Look forward to walking hand in hand with you! Company official website: www.sanbaiyun.com/ Resume: [email protected], please indicate from gold nuggets 😁