Communication standard

  1. Determine the protocol Http/Https
  2. Json /x-www-form-urlencoded/…
  3. Data types of specific interface parameters (weak types are easy to pit when interfacing with strongly typed languages)
  4. Determine the authentication mode
    • token
    • Parameter signature: single Key, exchange public Key
    • Certificate of calibration
  5. Determine the response structure (uniform success response code, etc.)

Exception handling

In general, anomaly structures should be divided into three categories:

  1. RpcException: call exception
  2. BusinessException: BusinessException
  3. Exception: other program exceptions

For different third-party services, exceptions may be classified according to different criteria. Generally, the following rules are used:

  1. If Http StatusCode == 500, RpcException is displayed
  2. Business response code! = Unified success response code is BusinessException
  3. Other anomalies are not classified

consistency

Consistency issues need to be considered when dealing with an interface that has a single order concept, such as payment and refund, or when it comes to state issues. Generally, there are the following requirements (third-party services are also called upstream) :

  1. If you have data upstream, you must have it locally
  2. The upstream state is the same as the state of each piece of data locally (or the state can be mapped relatively one-to-one)

Since third-party services are generally not controlled, consistency here is usually only final consistency

Transaction initiated

  1. Determine the unique identifier of an interface field, usually the requestNo. The value of this field maps the upstream data to the local data. The requestNo that exists upstream must exist locally

  2. Partition state:

  • PENDING: Local data is created but no interface request is initiated
  • UNCONFIRMED: Local data has been created and the interface request is not confirmed
  • PROCESSING: Indicates that an interface request is initiated and the upstream interface responds to the request
  • SUCCESS: Indicates that the service is successful
  • FAIL: indicates that the service fails
  • DEAD: indicates the final state. Local data has been created. The interface cannot be queried alive or DEAD, and upstream data cannot be queried

The process here can be summarized as follows:

  1. Before the upstream interface is initiated, a globally unique requestNo is generated with the status PENDING and is committed to the library.
  2. Request upstream interface without exception:
    1. If possible, process the state synchronously and update the state to the repository.
    2. Otherwise, the request is directly changed to PROCESSING, indicating that the request is successfully upstream and waiting for further confirmation.
  3. If the request upstream interface encounters RpcException and changes to PROCESSING, the request upstream is successful and the status is waiting for further confirmation.
  4. If the request upstream interface encounters BusinessException and changes to FAIL, it indicates that the request upstream succeeds and waits for further confirmation. (In this case, it may be processed as PROCESSING according to different business return codes, waiting for further confirmation)
  5. Other exceptions are encountered during the processing and are updated to UNCONFIRMED. It is not confirmed whether the upstream request has been made and waiting for further confirmation. In this case, it can be summarized as the local transaction processing failure, that is, the saving to the local database fails.

If you have low confidence in upstream, you can merge the PROCESSING state with UNCONFIRMED

Here is a pseudocode to describe the flow of the interface call:

// Start local transactions
startTrans();
Order order = new Order();
// Unique request number
String requestNo = UUID();
order.setRequestNo(requestNo);
order.setState(OrderState.PENDING);
order.save();
 // Commit a local transaction
commit();

try{ startTrans(); RpcResponse rpcRes = rpcService.requestToRpcCreateOrder(...) ; order.setState(OrderState.PROCESSING);// If your interface can synchronously return business status
    if(rpcRes.getState() == 'SUCCESS') {
        order.setState(OrderState.SUCCESS); 
    }
    if(rpcRes.getState() == 'FAIL') {
        order.setState(OrderState.FAIL); 
    }
    
    order.save();
    commit();
} catch(RpcException e) {
    startTrans();
    // It is considered to be processing and is waiting for follow-up check
    order.setState(OrderState.PROCESSING);
    order.save();
    commit();
} catch(BusinessException e) {
    startTrans();
    // Consider it a failure
    order.setState(OrderState.FAIL);
    // In case of failure, it is recommended to record RPC response parameters
    order.setRpcResponseCode(e.getCode());
    order.setRpcResponseMsg(e.getMsg());
    order.save();
    commit();
} catch(Exception e) {
    startTrans();
    // Consider it to be confirmed and wait for follow-up check
    order.setState(OrderState.UNCONFIRMED);
    order.save();
    commit();
}
Copy the code

Transaction rollback (retry)

After the above process, the data will remain in two states: UNCONFIRMED and PROCESSING. Therefore, further confirm these two states to ensure that the data reaches the final state.

Transaction backlookup can be implemented in several ways:

  1. Use timers to scan data in the UNCONFIRMED or PROCESSING state
    • Ensure that the database has indexes
    • If the requestNo field also has an index, you can use the overwrite index mechanism to shorten the query time. Generally, only the requestNo is required to query the upstream data state
  2. The UNCONFIRMED or processed data requestNo is stored in Redis and processed by a timer
  3. Using queues, plug UNCONFIRMED and PROCESSING into the check back queue

In fact, if your RPC requests do not need to be returned synchronously, it is recommended to use message queues with transaction mechanisms, otherwise there is an increase in complexity to consider using queues

Then how to deal with UNCONFIRMED and PROCESSING respectively

  • In contrast to PROCESSING, the PROCESSING method is simple, because the upstream of the status must return the corresponding status (in fact, some upstream does not necessarily), as long as the status is updated to SUCCESS or FAIL

  • Corresponding to UNCONFIRMED, we need to distinguish the case where upstream data does not exist, that is, in the above transaction initiation process, the upstream does not receive our request, so we need to deal with it according to the business situation:

    • Re-initiate the request (make sure the upstream interface is idempotent, otherwise handle it yourself)
    • Update to DEAD, discard this request

    If the upstream record exists, it is treated as PROCESSING

If your upstream provides asynchronous processing notifications, you can go through this phase of transaction lookback along the same lines

Summary and Reflection

This paper briefly summarizes several issues that need to be considered when connecting the third-party Service interface: communication standard, exception handling and consistency. The actual processing is usually divided into RPC layer and Service layer to deal with. RPC encapsulates communication standard and exception handling, and Service layer to deal with consistency. One last issue left undiscussed is that idempotent issues need to be considered before warehousing, before reissuing requests, and when asynchronous notifications are made. Let’s share some solutions for idempotent issues in the next article.