It is well known that databases can implement local transactions, which means that within the same database, you can allow a set of operations to either all execute correctly or none at all. There is a special emphasis on local transactions, meaning that the current database can only support transactions within the same database. However, today’s systems tend to use microservices architecture, and business systems have independent databases, so there is a need for transactions across multiple databases, which is called “distributed transactions”. So how can we implement distributed transactions when databases currently do not support cross-library transactions? This paper will first sort out the basic concepts and theoretical basis of distributed transaction, and then introduce several commonly used distributed transaction solutions. Without further ado, let’s begin

What is a transaction?

A transaction consists of a set of operations that we want to execute correctly, and if any step in this set of operations fails, we need to roll back previously completed operations. That is, all operations in the same transaction either all execute correctly or none at all.

The four properties of transactions are ACID

When it comes to transactions, there are four notable features of transactions.

  • Atomicity Atomicity requires that a transaction is an indivisible unit of execution in which all operations are either executed at all or not at all.

  • Consistency Consistency requires that the integrity constraints of the database are not broken before and after a transaction.

  • Isolated transactions execute independently of each other, they do not interfere with each other, and one transaction does not see the data of another ongoing transaction.

  • Persistence Persistence requires that after a transaction is completed, the execution results of the transaction must be persisted. Even if a database crash occurs, the results of the transaction commit will not be lost after the database is recovered.

Note: transactions can only ensure the high reliability of the database, that is, the data can still be recovered after the transaction is submitted even if the database itself has problems. If it is not a failure of the database itself, such as a corrupted hard disk, the data committed by the transaction may be lost. This falls under the category of “high availability”. Therefore, transactions can only ensure the “high reliability” of the database, and the “high availability” needs the cooperation of the whole system to achieve.

The isolation level of the transaction

This extends to a more detailed explanation of transaction isolation.

In ACID, the required isolation is in the strict sense that multiple transactions are executed sequentially without any interference from one another. This can be a complete guarantee of data security, but in real business systems, this approach is not high performance. Therefore, the database defines four isolation levels, which are inversely proportional to database performance. The lower the isolation level, the higher the database performance, and the higher the isolation level, the worse database performance.

Problems that occur when transactions are executed concurrently

Let’s take a look at some of the problems that can occur with databases at different isolation levels:

  1. Update loss When two concurrent transactions update the same row, it is possible for one transaction to overwrite the update from the other. Occurs when no lock has been added to the database.

  2. Dirty read A transaction reads data from another uncommitted transaction. The data may be rolled back and become invalid. An error occurs if the first transaction is processed with invalid data.

  3. Non-repeatable Read Non-repeatable: a transaction reads the same row twice and gets different results. It can be divided into the following two situations:

    • Virtual read: When transaction 1 reads the same record twice, transaction 2 modifies the record so that transaction 1 reads a different record a second time.
    • Phantom read: Transaction 1 inserts and deletes the table in the process of two queries, so the result of the second query of transaction 1 changes.

What is the difference between unrepeatable and dirty reads? Dirty reads are uncommitted data, while unrepeatable reads are committed data that has been modified by another transaction in the process.

Four isolation levels for a database

There are four isolation levels for databases:

  1. Read Uncommitted At this level, when a transaction changes a row, another transaction is not allowed to modify the row, but another transaction is allowed to Read the row. Therefore, at this level, updates will not be lost, but dirty and unrepeatable reads will occur.

  2. Read COMMITTED At this level, an uncommitted write transaction does not allow other transactions to access the row, so dirty reads do not occur. However, the transaction reading the data allows other transactions to access the data in that row, so non-repeatable reads can occur.

  3. Repeatable Read At this level, the read transaction forbids the write transaction, but allows the read transaction, so the same transaction never reads different data twice (non-repeatable read), and the write transaction forbids all other transactions.

  4. Serializable This level requires that all transactions be executed serially, thus avoiding all concurrency problems but being inefficient.

A higher isolation level ensures data integrity and consistency, but has a greater impact on concurrency performance. For most applications, setting the isolation level of the database system to Read Committed is a priority. It can avoid dirty reads and has good concurrency performance. Although it can lead to concurrent problems such as unrepeatable reads, phantom reads, and type ii missing updates, in the rare cases where such problems may occur, it can be controlled by the application using pessimistic or optimistic locking.

What are distributed transactions?

So far, the transactions described have been local transactions based on a single database. Current databases only support single library transactions, not cross-library transactions. With the popularity of microservice architecture, a large business system is often composed of several subsystems, which have their own independent databases. Often a business process needs to be done by multiple subsystems, and these operations may need to be done in a single transaction. These business scenarios are common in microservice systems. At this point, we need to implement cross-database transaction support through some means on the database, which is often referred to as “distributed transaction”.

A typical example of a distributed transaction is the user order process. When our system adopts the micro-service architecture, an e-commerce system is often divided into the following subsystems: commodity system, order system, payment system, points system, etc. The whole ordering process is as follows:

  1. The user browses through the product system, sees a particular product, and clicks to place an order
  2. The order system generates an order
  3. After the order is successfully created, the payment system provides the payment function
  4. When the payment is completed, the points system will add points for the user

Steps 2, 3, and 4 above need to be done in one transaction. Implementing A transaction for A traditional singleton application is as simple as putting these three steps into A method A that is identified by Spring’s @Transactional annotation. Spring, through the transaction support of the database, ensures that these steps are either all performed or none performed. But in this microservice architecture, these three steps involve three systems, involve three databases, and we have to implement distributed transaction support between the database and the application system through some dark technology.

Theory of CAP

CAP theory states that in A distributed system, only two requirements of C, A and P can be satisfied at most.

CAP meaning:

  • C: Consistency Consistency Whether multiple copies of the same data are the same in real time.
  • Availability: A certain amount of time & when the system returns A clear result the system is said to be available.
  • Fault tolerance distributes the same service across multiple systems, ensuring that when one system goes down, other systems still provide the same service.

CAP theory tells us that in distributed systems, we can choose at most two of the three conditions C, A, and P. So the question is, which two conditions are more appropriate to choose?

Availability and fault tolerance of partitions are two conditions that must be met for a business system, and they complement each other. Business systems use distributed systems for two main reasons:

  • Improve the overall performance When the business volume is soaring, a single server can no longer meet our business needs, we need to use a distributed system, using multiple nodes to provide the same function, so as to improve the overall performance of the system, this is the first reason to use a distributed system.

  • Zone fault tolerance If a single node or multiple nodes are on the same network, the service system may break down in the event of a power failure in the equipment room or a natural disaster occurs in the region. To prevent this problem, a distributed system is used to distribute multiple subsystems in different equipment rooms in different regions to ensure high availability of the system.

This shows that partition fault tolerance is the root of distributed system, if the partition fault tolerance can not meet, then the use of distributed system will be meaningless.

In addition, availability is particularly important for business systems. In today’s talk about the user experience, often appear abnormal “system” if the business system, response time is too long, and so on and so forth, this allows the user to liking system, in the Internet industry competitive today, not very enumeration, competitors in the field of the same system of intermittent unavailable immediately cause customers to competitors. Therefore, we can only sacrifice consistency for system availability and partition fault tolerance. So that’s the BASE theory.

The BASE theory of

CAP theory tells us the sad but inevitable truth that we can only choose between C, A and P. For business systems, consistency is often sacrificed in exchange for system availability and partition fault tolerance. It should be noted, however, that the so-called “sacrificing consistency” does not mean abandoning data consistency completely, but rather sacrificing strong consistency for weak consistency. So let’s talk about BASE theory.

  • BA: Basic Available
    • The whole system can still be “available” in the event of some force majeure, that is, it can still return a clear result within a certain time. The difference between “basic availability” and “high availability” is:
      • When a major promotion is held, the response time can be extended appropriately
      • Return a degraded page to some users directly return a degraded page to some users, thus relieving the server pressure. Note, however, that returning to the degraded page still returns explicit results.
  • S: Soft State: Soft State Indicates the State of different copies of the same data, which need not be consistent in real time.
  • E: Eventual Consisstency: Final consistency State of different copies of the same data. It does not need to be consistent in real time, but it must be consistent after a certain period of time.

Acid-base balance

ACID ensures strong transaction consistency, that is, data is consistent in real time. This is not a problem in local transactions. In distributed transactions, strong consistency will greatly affect the performance of the distributed system, so the distributed system can follow the BASE theory. However, different business scenarios of distributed systems have different requirements for consistency. For example, strong consistency is required in transaction scenarios, and ACID theory should be followed. However, real-time consistency is not required in scenarios such as sending SMS verification codes after successful registration, so BASE theory should be followed. Therefore, there is a balance between ACID and BASE depending on your business scenario.

Distributed transaction protocol

Several protocols for implementing distributed transactions are described below.

Two-phase commit protocol 2PC

One of the difficulties of distributed systems is how to ensure the consistency of transactional operations between multiple nodes in the architecture. To achieve this, the two-phase submission algorithm is based on the following assumptions:

  • In this distributed system, one node serves as a Coordinator and the other nodes serve as Cohorts. The nodes can communicate with each other on the network.
  • All nodes use write-ahead logs, and the logs are stored on reliable storage devices after being written, even if the node is damaged.
  • All nodes are not permanently damaged and can be recovered even after damage.

1. Phase I (voting phase) :

  1. The coordinator node asks all the participant nodes if they can vote and waits for the response from each participant node.
  2. The participant node performs all transactions until the query is initiated and writes Undo and Redo information to the log. (Note: if successful, each participant has already performed the transaction.)
  3. Each participant node responds to the query initiated by the coordinator node. If the transaction of the participant node actually succeeds, it returns a “agree” message; If the transaction operation of the participant node actually fails, it returns an abort message.

2. Phase II (submission for implementation) :

When the coordinator node gets the corresponding message “agree” from all the participant nodes:

  1. The coordinator node issues a “commit” request to all the participant nodes.
  2. The participant node formally completes the operation and releases the resources that were held during the entire transaction.
  3. The actor node sends a “done” message to the coordinator node.
  4. The coordinator node completes the transaction after receiving a “complete” message from all the participant nodes.

If the response message returned by any of the participant nodes in the first phase is “abort”, or the coordinator node cannot obtain the response message from all the participant nodes before the first phase query times out:

  1. The coordinator node issues a “rollback” request to all the participant nodes.
  2. The participant node performs a rollback with the previously written Undo information and frees the resources occupied during the entire transaction.
  3. The participant node sends a rollback complete message to the coordinator node.
  4. The coordinator node cancels the transaction after receiving a “rollback complete” message from all the participant nodes.

The second phase ends the current transaction regardless of the final outcome.

Two-phase commits do seem to provide atomic operations, but unfortunately, two-phase commits have several drawbacks:

  1. During execution, all participating nodes are transaction blocking. When a participant occupies a common resource, other third party nodes have to block access to the common resource.
  2. The participant is faulty. Procedure The coordinator needs to specify an additional timeout mechanism for each participant, after which the entire transaction fails. (Not much fault tolerance)
  3. The coordinator has failed. The participants will keep blocking. Additional standby machines are required for fault tolerance. (This can rely on the Paxos protocol to implement HA.)
  4. The problem that phase 2 failed to resolve: The coordinator went down after issuing a COMMIT message, and the only participant who received the message went down at the same time. So even if the coordinator creates a new coordinator by election agreement, the status of the transaction is uncertain, and no one knows whether the transaction has been committed.

For this reason, Dale Skeen and Michael Stonebraker proposed three-stage commit protocol (3PC) in “A Formal Model of Crash Recovery in A Distributed System”.

Three phase commit protocol 3PC

Unlike the two-phase commit, the three-phase commit has two change points.

  • Introduce timeouts. Introduce timeouts for both the coordinator and the participant.
  • Insert a preparation phase between phases 1 and 2. The state of each participating node is consistent before the final submission stage.

That is, in addition to introducing a timeout mechanism, 3PC splits the preparation phase of 2PC in two again, so that there are three phases of CanCommit, PreCommit, and DoCommit.

1. CanCommit phase

The CanCommit phase for 3PC is actually very similar to the preparation phase for 2PC. The coordinator sends a COMMIT request to the participant, who returns a Yes response if he can commit, or a No response otherwise.

  1. The transaction asks the coordinator to send a CanCommit request to the participant. Asks if a transaction commit operation can be performed. It then waits for the participant’s response.
  2. After receiving a CanCommit request, the participant normally returns a Yes response and goes into the preparatory state if it thinks it can execute the transaction successfully. Otherwise feedback No

2. PreCommit stage

The coordinator determines whether a transaction’s PreCommit operation can be remembered based on the response of the participant. Depending on the response, there are two possibilities. If the coordinator receives a Yes response from all participants, the transaction is pre-executed.

  1. Send a PreCommit request The coordinator sends a PreCommit request to the participant and enters the Prepared phase.

  2. Upon receipt of a PreCommit request, a transaction is performed and undo and redo information is recorded in the transaction log.

  3. Response Feedback If the participant successfully executes the transaction, an ACK response is returned and the participant waits for the final instruction.

If either participant sends a No response to the coordinator, or if the coordinator does not receive a response from the participant after a timeout, the transaction is interrupted.

  1. Sending interrupt Requests The coordinator sends abort requests to all participants.

  2. An interrupt transaction participant performs an interrupt of the transaction after receiving an ABORT request from the coordinator (or after a timeout and still no request from the coordinator).

3. DoCommit Phase The doCommit phase can be divided into the following two scenarios.

The actual transaction commit at this stage can also be divided into the following two scenarios.

3.1 Performing the Submission

  1. The send Submit request coordinator receives the ACK response sent by the participant, and he goes from the pre-commit state to the commit state. DoCommit requests are sent to all participants.
  2. After receiving the doCommit request, the transaction commit participant performs the formal transaction commit. All transaction resources are released after the transaction commits.
  3. Response Feedback After the transaction commits, an Ack response is sent to the coordinator.
  4. Completion the transaction coordinator completes the transaction after receiving ack responses from all participants.

3.2 Interrupt Transaction If the coordinator does not receive the ACK response sent by the participant (either the recipient sent a non-ACK response, or the response timed out), the interrupt transaction will be executed.

  1. Sending interrupt Requests The coordinator sends abort requests to all participants

  2. Transaction rollback participants receive abort requests, use the undo information they record in phase two to roll back the transaction and release all transaction resources after the rollback.

  3. Feedback Results After the participant completes the transaction rollback, it sends an ACK message to the coordinator

  4. The interrupt transaction coordinator performs the interrupt of the transaction after receiving the ACK message from the participant.

Distributed transaction solutions

There are several solutions for distributed transactions:

  • Global news
  • Distributed transactions based on reliable messaging services
  • TCC
  • Best effort notice

Scenario 1: Global Transactions (DTP model)

Global transactions are implemented based on the DTP model. DTP is a Distributed Transaction Processing Reference Model proposed by X/Open organization. It specifies that three roles are required to implement distributed transactions:

  • It is the business system we develop, in the process of our development, we can use the transaction interface provided by the resource manager to achieve distributed transactions.

  • TM: Transaction Manager Transaction Manager

    • The implementation of distributed transactions is accomplished by the transaction manager, which provides the operation interface of distributed transactions for our business system to call. These interfaces are called TX interfaces.
    • The transaction manager also manages all resource managers, scheduling them together through the XA interface they provide to implement distributed transactions.
    • DTP is only a set of specifications for the realization of distributed transactions, and does not define how to realize distributed transactions in detail. TM can use 2PC, 3PC, Paxos and other protocols to realize distributed transactions.

  • RM: Resource Manager Indicates the Resource Manager

    • The objects that can provide data services can be resource managers, such as databases, messaging middleware, caches, and so on. In most scenarios, the database is the resource manager in a distributed transaction.
    • Resource managers can provide the transaction capability of a single database. They provide the commit, rollback and other capabilities of the database to the transaction manager through XA interface, so as to help the transaction manager to achieve distributed transaction management.
    • XA is the interface defined by the DTP model to provide the transaction manager with the commit, rollback, and other capabilities of the resource manager (the database).
    • DTP is only a set of specifications to implement distributed transactions, and the specific implementation of RM is done by database vendors.
  1. Is there distributed transaction middleware based on DTP model?
  1. What are the pros and cons of the DTP model?

Scheme 2: Distributed transaction based on reliable message service

This way of implementing distributed transactions needs to be realized by message-oriented middleware. Suppose you have two systems, A and B, that can handle task A and task B, respectively. In this case, A business process exists in system A, and task A and task B need to be processed in the same transaction. The following describes the implementation of this distributed transaction based on message-oriented middleware.

  • Before system A processes task A, it sends A message to the messaging middleware
  • Message-oriented middleware persists the message after receiving it, but does not deliver it. Downstream system B is still unaware of the message.
  • After successful persistence, the message-oriented middleware returns an acknowledgement reply to system A.
  • After receiving the acknowledgement, system A can process task A.
  • After task A completes processing, send A Commit request to the messaging middleware. After the request is sent, the transaction is complete for system A and it can now work on other tasks. However, the COMMIT message may be lost in transit and the messaging middleware will not deliver the message to system B, resulting in inconsistencies in the system. This problem is handled by the transaction backcheck mechanism of message-oriented middleware, as described below.
  • After receiving the Commit command, the message-oriented middleware delivers the message to system B, thus triggering the execution of task B.
  • When task B completes, system B returns an acknowledgement reply to the messaging middleware, telling the messaging middleware that the message has been successfully consumed, and the distributed transaction is complete.

The above process can draw the following conclusions:

  1. Message-oriented middleware plays the role of distributed transaction coordinator.
  2. There is A time difference between system A completing task A and task B completing task B. In this time difference, the whole system is in a state of data inconsistency, but this temporary inconsistency is acceptable, because after a short time, the system can maintain data consistency, to meet the BASE theory.

In the preceding process, if task A fails to be processed, the rollback process is required, as shown in the following figure:

  • If system A fails to process task A, A Rollback request is sent to the messaging middleware. Just like sending A Commit request, system A can assume that the rollback is complete after sending it, and it can do something else.
  • After receiving the rollback request, the messaging middleware directly discards the message instead of delivering it to system B. In this way, task B of system B is not triggered.

The system is in A consistent state again because task A and task B are not executed.

The Commit and Rollback described above are ideal, but in a real world system, both the Commit and Rollback instructions can be lost in transit. How does messaging middleware ensure data consistency when this happens? The answer is the time-out query mechanism.

In addition to realizing normal business processes, system A also needs to provide an interface for transaction inquiry, which can be invoked by message middleware. When the messaging middleware receives A transactional message, it will start timing. If it does not receive Commit or Rollback instructions from system A within the timeout period, it will proactively call the transaction query interface provided by system A to inquire about the current status of the system. This interface returns three results:

  • Commit If the status obtained is Commit, the message is delivered to system B.
  • Rollback If the obtained status is Rollback, the message is discarded.
  • Processing If the obtained state is Processing, wait.

The timeout query mechanism of message-middleware can prevent system inconsistency caused by the loss of Commit/Rollback instructions in the transmission process, and reduce the blocking time of the upstream system. After issuing Commit/Rollback instructions, the upstream system can process other tasks without waiting for confirmation. The timeout query mechanism is used to compensate for the loss of Commit/Rollback instructions, which greatly reduces the blocking time of the upstream system and improves the system concurrency.

Let’s talk about the reliability of the message delivery process. When the upstream system completes the task and submits the Commit directive to the message-oriented middleware, it is ready to process other tasks. At this point, it can consider the transaction completed and the message-oriented middleware will ensure that the message is successfully consumed by the downstream system! ** So how does this work? This is guaranteed by the messaging middleware delivery process.

After the message is delivered to the downstream system, the message-oriented middleware enters the blocking waiting state, and the downstream system immediately processes the task. After the task is processed, the message-oriented middleware returns the reply. The message-oriented middleware receives an acknowledgement and considers the transaction complete!

If a message is lost during delivery, or an acknowledgement reply for a message is lost on the way back, the messaging middleware redelivers after waiting for the acknowledgement reply to time out until the downstream consumer returns a successful consumption response. Of course, the general message middleware can set the number and interval of message retries. For example, if the first delivery fails, the message will be retried every five minutes for a total of three times. If the delivery fails after three retries, the message requires human intervention.

Some students may ask: why don’t you roll back a failed message instead of trying to redeliver it?

This involves the implementation cost of the whole distributed transaction system. We know that after system A will send A Commit instruction to the messaging middleware, it will do something else. If the message delivery fails and rollback is needed, system A needs to provide the rollback interface in advance, which undoubtedly increases the additional development cost and the complexity of the business system. The design goal of a service system is to minimize the complexity of the system while ensuring performance, thus reducing the operation and maintenance costs of the system.

I don’t know if you have noticed that the upstream system A submits Commit/Rollback messages to the message-oriented middleware in an asynchronous manner, that is, after the upstream system submits the messages, it can do other things, then the submission and Rollback are completely entrusted to the message-oriented middleware, and it completely trusts the message-oriented middleware. Assume that it must complete the commit or rollback of the transaction correctly. However, the messaging middleware delivers messages to downstream systems synchronously. That is, after the message middleware delivers the message to the downstream system, it will block the wait, and the downstream system will cancel the blocking wait after successfully processing the task and returning the acknowledgement reply. Why are the two inconsistent in design?

First, asynchronous communication between upstream systems and messaging middleware is used to improve system concurrency. Business systems deal directly with users, and user experience is particularly important. Therefore, this asynchronous communication mode can greatly reduce user waiting time. In addition, compared with synchronous communication, asynchronous communication has no long blocking wait, so the concurrency of the system is greatly increased. However, asynchronous communication can cause the problem of Commit/Rollback instruction loss, which is compensated by the timeout query mechanism of the messaging middleware.

So why synchronous communication between message-oriented middleware and downstream systems?

Asynchrony can improve system performance but increase system complexity. Synchronization reduces system concurrency, but the implementation cost is low. Therefore, synchronization can be used to reduce the complexity of the system when the requirements for concurrency are not very high, or when the server resources are abundant. We know that the message middleware is an independent of the business system of the third party middleware, coupling it with any business system to produce directly, it also have direct link to the user, it usually deployed on the server cluster of independent and has good expansibility, so don’t too worry about its performance, if the processing speed can’t meet our requirements, It can be solved by adding machines. Moreover, even some delay in the speed of message-oriented middleware processing is acceptable, because the BASE theory introduced earlier tells us that we are looking for final consistency, not real-time consistency, so it is acceptable for message-oriented delay to cause transient inconsistencies in transactions.

Option 3: Best effort notification (periodic proofreading)

Maximum effort notification is also known as periodic proofreading, which is already included in Plan 2 and introduced here separately, mainly for the integrity of the knowledge system. This solution also requires the participation of message-oriented middleware as follows:

  • After completing the task, the upstream system synchronously sends a message to the message middleware to ensure that the message is successfully persisted, and then the upstream system can do other things;
  • After receiving the message, the message-oriented middleware delivers the message synchronously to the corresponding downstream system and triggers the task execution of the downstream system.
  • When the downstream system processes successfully, it sends an acknowledgement reply back to the message-oriented middleware, which then deletes the message and completes the transaction.

The above is an idealized process, but in a real scenario, the following unexpected situations often occur:

  1. The messaging middleware failed to deliver a message to the downstream system
  2. The upstream system failed to send a message to the messaging middleware

For the first case, the message middleware has a retry mechanism, we can set the message in the message middleware retries and retry interval, for network instability caused by the message delivery failed, often try again after a few messages can be delivered successfully, if more than the retry limit still delivery fails, then the message middleware can no longer deliver the message, Instead, it is recorded in the failure message table. The message middleware needs to provide the failure message query interface, and the downstream system will periodically query the failure message and consume it, which is called “periodic proofreading”.

If repeated delivery and regular proofreading fail to solve the problem, it is often because something is seriously wrong with the downstream system, requiring human intervention.

In the second case, a message retransmission mechanism needs to be established in the upstream system. You can set up a local message table on the upstream system and put the task processing and inserting messages into the local message table in a local transaction. If a message fails to be inserted into the local message table, a rollback is triggered and the previous task processing result is cancelled. If all these steps are executed successfully, the local transaction is complete. A dedicated message sender then keeps sending messages from the local message table, and if it fails, it returns and tries again. Of course, a maximum retry limit should also be set on the sender. Generally, failure to reach the retry limit means that there is a serious problem with the messaging middleware that can only be resolved by human intervention.

For message-oriented middleware that does not support transactional messaging, this approach can be used if distributed transactions are to be implemented. It can be implemented through the retry mechanism + check regularly distributed transactions, but compared to the second kind of plan, it with a long cycle to achieve data consistency, but also need to retry in upstream system realize news release mechanism, to ensure that the message was released to message middleware, it certainly increase the cost of the development of the business system and make the business system is not pure, And this additional business logic will undoubtedly take up hardware resources of the business system, thus affecting performance.

Therefore, use messaging middleware that supports transactional messaging to implement distributed transactions, such as RocketMQ.

Scheme 4: TCC (two-stage, compensation type)

TCC stands for Try Confirm Cancel, which is a compensated distributed transaction. As the name suggests, TCC implements distributed transactions in three steps:

  • Try: Tries services to be executed
    • This process does not execute the business, but just completes the consistency check for all the business and reserves all the resources required for execution
  • Confirm: Perform the service
    • This process actually starts executing the business, and because the consistency check has been done in the Try phase, this process is executed directly without any checks. In addition, service resources reserved during the Try phase are used.
  • Cancel: Cancels the service
    • If the service fails to be executed, the Cancel phase releases all occupied service resources and rolls back the operations performed in the Confirm phase.

The following uses a transfer example to explain how TCC implements distributed transactions.

Suppose user A uses his account balance to send user B A red envelope of 100 yuan, and the balance system and the red envelope system are two separate systems.

  • Try

    • Create a flow and set the state of the flow to transaction
    • Deduct 100 yuan (reserved service resources) from user A’s account
    • After the Try succeeds, the Confirm phase is entered
    • If any exception occurs during the Try process, it enters the Cancel phase

  • Confirm

    • Add 100 yuan to user B’s red envelope account
    • Sets the flow state to transaction completed
    • If any exception occurs during the Confirm process, the Cancel phase is entered
    • If the Confirm process is successful, the transaction ends

  • Cancel

    • Add $100 to user A’s account
    • Set the flow state to failed transaction

In the traditional transaction mechanism, the execution of business logic and transaction processing are completed by different components at different stages: the business logic part accesses resources to realize data storage, and its processing is responsible for by the business system; The transaction processing part implements transaction management by coordinating resource managers, whose processing is handled by the transaction manager. There is not much interaction between the two, so the transaction logic of a traditional transaction manager only needs to focus on the commit/ ROLLBACK phase, not the business execution phase.

TCC global transactions must be based on RM local transactions to implement global transactions

The TCC service consists of the Try, Confirm, and Cancel services. When the Try, Confirm, and Cancel services are executed, the TCC service accesses the Resource Manager (RM) to access data. These access operations must participate in the RM local transaction to make the data they change either commit or rollback.

This isn’t hard to understand. Consider the following scenario:

Assuming that service B in the figure is not based on RM local transactions (for example, RDBS, which can be simulated by setting auto-commit to true), if [B:Try] fails midway and the TCC transaction framework later decides to roll back the global transaction, The [B:Cancel] operation needs to determine which operations in [B:Try] have been written to the DB and which operations have not been written to the DB. Assume that the [B:Try] service has five write operations to the database. The [B:Cancel] service needs to determine whether the five operations take effect one by one and reverse the effective operations.

Unfortunately, since [B:Cancel] also has n (0<=n<=5) reverse write operations, if [B:Cancel] also fails, the subsequent [B:Cancel] execution will be more onerous. Since the first [B:Cancel] operation requires subsequent [B:Cancel] operations to determine which of the n (0<=n<=5) write libraries of the previous [B:Cancel] operation have been executed and which have not yet been executed, the idempotent problem is involved. The guarantee of idempotency may also involve additional write library operations, which have similar problems due to the lack of RM local transaction support… As you can imagine, the TCC transaction framework cannot effectively manage TCC global transactions without being based on RM local transactions.

On the other hand, TCC transactions based on RM local transaction can be easily handled: [B:Try] operation fails in the middle, TCC transaction framework can participate in RM local transaction rollback. Later, when the TCC transaction framework decides to rollback the global transaction, it does not need to perform the [B:Cancel] operation when it knows that the RM local transaction involved in the [B:Try] operation has rolled back.

In other words, when implementing the TCC transaction framework based on RM local transactions, the Cancel business of a TCC service either executes or does not execute, regardless of the partial execution.

The TCC transaction framework should provide idempotency guarantees for the Confirm/Cancel service

The idempotent nature of a service is generally considered to be a pointer to multiple (n>1) requests to the same service and a single (n=1) request to it, both of which have the same side effects.

In the TCC transaction model, the Confirm/Cancel business may be called repeatedly for a number of reasons. For example, the Confirm/Cancel business logic of each TCC service is invoked when a global transaction commits/rolls back. When performing these Confirm/Cancel services, a failure such as a network outage may occur that prevents the global transaction from completing. Therefore, the failure recovery mechanism will still recommit/roll back these outstanding global transactions, so that the Confirm/Cancel business logic of each TCC service participating in the global transaction will be invoked again.

Since the Confirm/Cancel service may be called multiple times, it needs to be idempotent. So, should the TCC transaction framework provide idempotent security? Or should business systems ensure idempotency themselves? Personally, IT is the TCC transaction framework that provides idempotent security. If only a few services have this problem, then it is ok for the business system to take responsibility; However, this is a common class of problems, and there is no doubt that all TCC services have idempotency problems with Confirm/Cancel services. The common issues of TCC services should be addressed by the TCC transaction framework; Moreover, if you consider the need for business systems to be responsible for idempotence, there is no doubt that this increases the complexity of business systems.

reference

  • Distributed transaction processing in large-scale SOA systems
  • Life beyond Distributed Transactions: an Apostate’s Opinion
  • Some thoughts on how to implement a TCC distributed transaction framework
  • How can a requestor ensure a consistent outcome across multiple, independent providers
  • About distributed transactions, two-phase commit protocol, three-stage commit protocol
  • Three-phase commit protocol