There are several solutions for distributed transactions:

  • Global news
  • Distributed transactions based on reliable messaging services
  • TCC
  • Best effort notice

Scenario 1: Global Transactions (DTP model)

Global transactions are implemented based on the DTP model. DTP is a Distributed Transaction Processing Reference Model proposed by X/Open organization. It specifies that three roles are required to implement distributed transactions:

The public,

  • The only net from 0 to help Java developers to do big data domain public number ~

  • Http://import_bigdata/import_bigdata/import_bigdata/import_bigdata/import_bigdata/import_bigdata/import_bigdata

  • It is the business system we develop, in the process of our development, we can use the transaction interface provided by the resource manager to achieve distributed transactions.

  • TM: Transaction Manager Transaction Manager

    • The implementation of distributed transactions is accomplished by the transaction manager, which provides the operation interface of distributed transactions for our business system to call. These interfaces are called TX interfaces.
    • The transaction manager also manages all resource managers, scheduling them together through the XA interface they provide to implement distributed transactions.
    • DTP is only a set of specifications for the realization of distributed transactions, and does not define how to realize distributed transactions in detail. TM can use 2PC, 3PC, Paxos and other protocols to realize distributed transactions.

  • RM: Resource Manager Indicates the Resource Manager

    • The objects that can provide data services can be resource managers, such as databases, messaging middleware, caches, and so on. In most scenarios, the database is the resource manager in a distributed transaction.
    • Resource managers can provide the transaction capability of a single database. They provide the commit, rollback and other capabilities of the database to the transaction manager through XA interface, so as to help the transaction manager to achieve distributed transaction management.
    • XA is the interface defined by the DTP model to provide the transaction manager with the commit, rollback, and other capabilities of the resource manager (the database).
    • DTP is only a set of specifications to implement distributed transactions, and the specific implementation of RM is done by database vendors.

Actual scenario: two-phase commit based on XA protocol

XA is a distributed transaction protocol proposed by Tuxedo. XA is roughly divided into two parts: the transaction manager and the local resource manager. Among them, the local resource manager is often implemented by the database, such as Oracle, DB2 and other commercial databases have implemented XA interface, and the transaction manager as the global scheduler, responsible for the submission and rollback of each local resource. XA implements distributed transactions as follows:

In general, the XA protocol is relatively simple, and once commercial databases implement it, the cost of using distributed transactions is low. However, XA also has a fatal disadvantage, that is, the performance is not ideal, especially in the transaction of single link, often high concurrency, XA can not meet the high concurrency scenario. Currently, XA is well supported by commercial databases, but not so well supported by mysql databases. The XA implementation of mysql does not record logs in the prepare phase, causing data inconsistency between the active and standby databases due to the switchover between the active and standby databases. Many NoSQL also don’t support XA, which makes for a very narrow application scenario for XA.

Scenario 2: Distributed Transactions Based on Reliable Messaging Services (Transactional messaging Middleware)

This way of implementing distributed transactions needs to be realized by message-oriented middleware. Suppose you have two systems, A and B, that can handle task A and task B, respectively. In this case, A business process exists in system A, and task A and task B need to be processed in the same transaction. The following describes the implementation of this distributed transaction based on message-oriented middleware.

  • Before system A processes task A, it sends A message to the messaging middleware
  • Message-oriented middleware persists the message after receiving it, but does not deliver it. Downstream system B is still unaware of the message.
  • After successful persistence, the message-oriented middleware returns an acknowledgement reply to system A.
  • After receiving the acknowledgement, system A can process task A.
  • After task A completes processing, send A Commit request to the messaging middleware. After the request is sent, the transaction is complete for system A and it can now work on other tasks. However, the COMMIT message may be lost in transit and the messaging middleware will not deliver the message to system B, resulting in inconsistencies in the system. This problem is handled by the transaction backcheck mechanism of message-oriented middleware, as described below.
  • After receiving the Commit command, the message-oriented middleware delivers the message to system B, thus triggering the execution of task B.
  • When task B completes, system B returns an acknowledgement reply to the messaging middleware, telling the messaging middleware that the message has been successfully consumed, and the distributed transaction is complete.

The above process can draw the following conclusions:

  • Message-oriented middleware plays the role of distributed transaction coordinator.
  • There is A time difference between system A completing task A and task B completing task B. In this time difference, the whole system is in a state of data inconsistency, but this temporary inconsistency is acceptable, because after a short time, the system can maintain data consistency, to meet the BASE theory.

In the preceding process, if task A fails to be processed, the rollback process is required, as shown in the following figure:

  • If system A fails to process task A, A Rollback request is sent to the messaging middleware. Just like sending A Commit request, system A can assume that the rollback is complete after sending it, and it can do something else.
  • After receiving the rollback request, the messaging middleware directly discards the message instead of delivering it to system B. In this way, task B of system B is not triggered.

The system is in A consistent state again because task A and task B are not executed.

The Commit and Rollback described above are ideal, but in a real world system, both the Commit and Rollback instructions can be lost in transit. How does messaging middleware ensure data consistency when this happens? The answer is the time-out query mechanism.

In addition to realizing normal business processes, system A also needs to provide an interface for transaction inquiry, which can be invoked by message middleware. When the messaging middleware receives A transactional message, it will start timing. If it does not receive Commit or Rollback instructions from system A within the timeout period, it will proactively call the transaction query interface provided by system A to inquire about the current status of the system. This interface returns three results:

  • Commit If the status obtained is Commit, the message is delivered to system B.
  • Rollback If the obtained status is Rollback, the message is discarded.
  • Processing If the obtained state is Processing, wait.

The timeout query mechanism of message-middleware can prevent system inconsistency caused by the loss of Commit/Rollback instructions in the transmission process, and reduce the blocking time of the upstream system. After issuing Commit/Rollback instructions, the upstream system can process other tasks without waiting for confirmation. The timeout query mechanism is used to compensate for the loss of Commit/Rollback instructions, which greatly reduces the blocking time of the upstream system and improves the system concurrency.

Let’s talk about the reliability of the message delivery process. When the upstream system completes the task and submits the Commit directive to the message-oriented middleware, it is ready to process other tasks. At this point, it can consider the transaction completed and the message-oriented middleware will ensure that the message is successfully consumed by the downstream system! ** So how does this work? This is guaranteed by the messaging middleware delivery process.

After the message is delivered to the downstream system, the message-oriented middleware enters the blocking waiting state, and the downstream system immediately processes the task. After the task is processed, the message-oriented middleware returns the reply. The message-oriented middleware receives an acknowledgement and considers the transaction complete!

If a message is lost during delivery, or an acknowledgement reply for a message is lost on the way back, the messaging middleware redelivers after waiting for the acknowledgement reply to time out until the downstream consumer returns a successful consumption response. Of course, the general message middleware can set the number and interval of message retries. For example, if the first delivery fails, the message will be retried every five minutes for a total of three times. If the delivery fails after three retries, the message requires human intervention.

Some students may ask: why don’t you roll back a failed message instead of trying to redeliver it?

This involves the implementation cost of the whole distributed transaction system. We know that after system A will send A Commit instruction to the messaging middleware, it will do something else. If the message delivery fails and rollback is needed, system A needs to provide the rollback interface in advance, which undoubtedly increases the additional development cost and the complexity of the business system. The design goal of a service system is to minimize the complexity of the system while ensuring performance, thus reducing the operation and maintenance costs of the system.

I don’t know if you have noticed that the upstream system A submits Commit/Rollback messages to the message-oriented middleware in an asynchronous manner, that is, after the upstream system submits the messages, it can do other things, then the submission and Rollback are completely entrusted to the message-oriented middleware, and it completely trusts the message-oriented middleware. Assume that it must complete the commit or rollback of the transaction correctly. However, the messaging middleware delivers messages to downstream systems synchronously. That is, after the message middleware delivers the message to the downstream system, it will block the wait, and the downstream system will cancel the blocking wait after successfully processing the task and returning the acknowledgement reply. Why are the two inconsistent in design?

First, asynchronous communication between upstream systems and messaging middleware is used to improve system concurrency. Business systems deal directly with users, and user experience is particularly important. Therefore, this asynchronous communication mode can greatly reduce user waiting time. In addition, compared with synchronous communication, asynchronous communication has no long blocking wait, so the concurrency of the system is greatly increased. However, asynchronous communication can cause the problem of Commit/Rollback instruction loss, which is compensated by the timeout query mechanism of the messaging middleware.

So why synchronous communication between message-oriented middleware and downstream systems?

Asynchrony can improve system performance but increase system complexity. Synchronization reduces system concurrency, but the implementation cost is low. Therefore, synchronization can be used to reduce the complexity of the system when the requirements for concurrency are not very high, or when the server resources are abundant. We know that the message middleware is an independent of the business system of the third party middleware, coupling it with any business system to produce directly, it also have direct link to the user, it usually deployed on the server cluster of independent and has good expansibility, so don’t too worry about its performance, if the processing speed can’t meet our requirements, It can be solved by adding machines. Moreover, even some delay in the speed of message-oriented middleware processing is acceptable, because the BASE theory introduced earlier tells us that we are looking for final consistency, not real-time consistency, so it is acceptable for message-oriented delay to cause transient inconsistencies in transactions.

Option 3: Best effort notifications (periodic proofreading) also known as local message sheets

Maximum effort notification is also known as periodic proofreading, which is already included in Plan 2 and introduced here separately, mainly for the integrity of the knowledge system. This solution also requires the participation of message-oriented middleware as follows:

  • After completing the task, the upstream system synchronously sends a message to the message middleware to ensure that the message is successfully persisted, and then the upstream system can do other things;
  • After receiving the message, the message-oriented middleware delivers the message synchronously to the corresponding downstream system and triggers the task execution of the downstream system.
  • When the downstream system processes successfully, it sends an acknowledgement reply back to the message-oriented middleware, which then deletes the message and completes the transaction.

The above is an idealized process, but in a real scenario, the following unexpected situations often occur:

  • The messaging middleware failed to deliver a message to the downstream system
  • The upstream system failed to send a message to the messaging middleware

For the first case, the message middleware has a retry mechanism, we can set the message in the message middleware retries and retry interval, for network instability caused by the message delivery failed, often try again after a few messages can be delivered successfully, if more than the retry limit still delivery fails, then the message middleware can no longer deliver the message, Instead, it is recorded in the failure message table. The message middleware needs to provide the failure message query interface, and the downstream system will periodically query the failure message and consume it, which is called “periodic proofreading”.

If repeated delivery and regular proofreading fail to solve the problem, it is often because something is seriously wrong with the downstream system, requiring human intervention.

In the second case, a message retransmission mechanism needs to be established in the upstream system. You can set up a local message table on the upstream system and put the task processing and inserting messages into the local message table in a local transaction. If a message fails to be inserted into the local message table, a rollback is triggered and the previous task processing result is cancelled. If all these steps are executed successfully, the local transaction is complete. A dedicated message sender then keeps sending messages from the local message table, and if it fails, it returns and tries again. Of course, a maximum retry limit should also be set on the sender. Generally, failure to reach the retry limit means that there is a serious problem with the messaging middleware that can only be resolved by human intervention.

For message-oriented middleware that does not support transactional messaging, this approach can be used if distributed transactions are to be implemented. It can be implemented through the retry mechanism + check regularly distributed transactions, but compared to the second kind of plan, it with a long cycle to achieve data consistency, but also need to retry in upstream system realize news release mechanism, to ensure that the message was released to message middleware, it certainly increase the cost of the development of the business system and make the business system is not pure, And this additional business logic will undoubtedly take up hardware resources of the business system, thus affecting performance.

Therefore, use messaging middleware that supports transactional messaging to implement distributed transactions, such as RocketMQ.

Scheme 4: TCC (two-stage, compensation type)

Atomicity requirements for business operations across applications are actually quite common. In third party payment in the scene combination, for example, users in the e-commerce sites after shopping, want to use balance and bonus payment of orders at the same time, the balance system and a red envelope is different application system, payment system when the two systems of the call, just need to keep the balance deductions and red packet using either success at the same time, or fail at the same time.

The emergence of TCC transactions is to solve the atomicity of cross-application business operations caused by application splitting. Of course, because of the poor performance of regular XA transactions (2PC, 2 Phase Commit, two-phase Commit), there are also scenarios where TCC transactions are used to resolve database splitting (such as accounting splitting), as discussed later in this article.

Therefore, from the point of view of the whole system architecture, the different schemes of distributed transactions exist hierarchical structure.

The mechanism of TCC

It is obvious that TCC is an acronym for three English words. Yes, TCC corresponds to the Try, Confirm, and Cancel operations respectively. The three operations have the following meanings:

Try: reserve service resources Confirm: Confirm the service operation Cancel: Cancel the service operation

Take a look at the three operations of relational database transactions: DML, Commit, and Rollback. In a cross-application service operation, the Try operation reserves and locks service resources in multiple applications to lay a foundation for subsequent confirmation. Similarly, the DML operation locks database rows to hold database resources. The Confirm operation is performed after all applications involved in the Try operation are successful. The reserved service resources are used, similar to the Commit operation. Cancel: If all applications involved in the Try operation are not successful, the successful applications need to be cancelled (Rollback). Confirm and Cancel are a pair of reverse business operations.

In short, TCC is the 2PC of the application layer (2 Phase Commit), if you think of your application as a resource manager. In detail, each TCC operation needs to do the following:

1. Try: Attempts to execute services. Complete all service checks (consistency) Reserve required service resources (quasi-isolation) 2. Confirm: Confirm the service. 3. Cancel: Cancels the service execution and releases the service resources reserved during the Try phase

A complete TCC transaction participant consists of three parts:

Master business service: Master business service is the initiator of the whole business activity. For example, in the combination payment scenario mentioned above, payment system is the master business service. Slave business service: The slave business service is responsible for providing TCC business operations and is the operator of the overall business activities. The secondary business service must implement the Try, Confirm, and Cancel interfaces to be invoked by the primary business service. Because the Confirm and Cancel operations can be called repeatedly, the Confirm and Cancel interfaces must be idempotent. The balance system and red envelope system in the previous combined payment scenario are slave business services. Business manager: business manager management control of the business activities, including record maintenance TCC global transaction state of affairs and each from the state of affairs, business service and confirm when business activity to submit all the TCC confirm the operation of the operation, in the business activities to cancel all TCC calls when cancel operation type operation. As you can see, the entire TCC transaction is transparent to the main business service, where the business activity manager and the slave business service each do part of the work.

Advantages and limitations of TCC

The advantages of TCC transactions are as follows: they solve the atomicity problem of business operations across applications and are very useful in scenarios such as combined payments and accounting split. TCC actually refers to the application layer to achieve the two-stage submission of the database layer, which is a one-stage submission for the database, avoiding the low 2PC performance problem of the database layer.

The disadvantages of TCC transactions are mainly one: the Try, Confirm and Cancel operation functions of TCC need to be provided by services, and the development cost is high. Of course, whether this shortcoming of TCC transactions is a weakness is a matter of opinion.

A Case study

TCC To be honest, TCC’s theory is a little confusing. Therefore, the following will take accounting split as an example to describe the process of TCC transactions, hoping to be helpful to understand TCC. The business scenario of account split is as follows: accounts A, B and C in three different branches, A and B together transfer A total of 80 yuan to C: TCC transaction

1. Try: Attempts to execute services. Complete all service checks (consistency) : Check whether the status of accounts A, B, and C is normal, whether the balance of account A is not less than 30 YUAN, and the balance of account B is not less than 50 yuan. Reserve required business resources (quasi-isolation) : increase the frozen amount of account A by $30 and increase the frozen amount of account B by $50 to ensure that the balance of the two accounts will not be reduced by other concurrent processes resulting in insufficient available balance of accounts A and B during subsequent actual transfer operations.

2. Confirm: Confirm the service. Perform services: If the status of accounts A, B, and C is normal and the balance of accounts A and B is sufficient, transfer RMB 30 from account A to account C and RMB 50 from account B to account C. No service check: No service check is required and the service check is completed in the Try phase. Use only the service resources reserved during the Try phase: Use only the frozen amount of accounts A and B during the Try phase.

Release the reserved service resources during the Try phase. If the Try phase is partially successful, for example, the balance of account A is sufficient and the freeze is successful, but the balance of account B is insufficient and the freeze fails, perform the Cancel operation for account A to unfreeze the frozen amount of account A.