Wechat official account: Moon chat technology

This article is about 5300 words, and it will take you about 13 minutes to read it completely.

What is a distributed transaction

A distributed transaction means that the participants of the transaction, the server supporting the transaction, the resource server and the transaction manager are located on different nodes of different distributed systems.

A large operation is performed by N small operations. These small operations are distributed across different services. For these operations, either they all succeed or they all don’t.

Why are there distributed transactions?

Here’s an example:

Transfer is the most classic distributed transaction scenario. Suppose that user A uses the bank app to initiate an inter-bank transfer to user B. The banking system first deducts user A’s money and then increases the balance in user B’s account.

If one of these steps fails, there are two possible exceptions:

  • 1. User A’s account is successfully deducted, but user B’s account balance fails to increase
  • 2. User A’s account fails to be deducted, but user B’s account balance is successfully increased.

For a banking system, neither of the above is allowed and a transaction is needed to ensure the success of the transfer operation.

In a single application, we only need to post the @Transactional annotation to start a transaction to keep the entire operation atomic.

However, the seemingly simple operation above cannot be a single service in the actual application architecture, so we will hand over this series of operations to N services to complete, that is, split into a distributed microservice architecture.

For example, the ordering service, the inventory service, etc., must ensure the consistency of the results of different service states, hence the distributed transaction.

Distributed theory

The CAP theorem

In a distributed system, you can’t have your cake and eat it

Consistency (C) :

Whether all data backups in a distributed system have the same value at the same time. (Equivalent to all nodes accessing the same up-to-date copy of data)

Availability (A) :

Whether the cluster as a whole can respond to read and write requests from clients after some nodes in the cluster fail. (High availability for data updates)

Partition fault tolerance (P) :

Even if a single component becomes unavailable, the operation can still be completed.

Specifically, in a distributed system, a Web application can only support both of these attributes at most in any database design. Obviously, any scale-out strategy depends on data partitioning. Therefore, designers must choose between consistency and usability.

The BASE theory of

In distributed systems, we tend to pursue availability, which is more important than consistency, so how to achieve high availability?

Another theory, the BASE theory, has been put forward by our predecessors to further expand the CAP theorem. BASE theory refers to:

  • Basically Available
  • Soft state
  • Eventually consistent

BASE theory is the result of a tradeoff between consistency and availability in CAP. The core idea of the theory is that we can’t achieve strong consistency, but every application can achieve Eventual consistency in appropriate ways according to its own business characteristics.

Distributed transaction solutions

Two-phase commit (2PC)

Those of you who are familiar with mysql are familiar with two-phase commit, and mysql transactions do two-phase commit through the logging system.

The two-phase protocol can be used for single-machine centralized systems where multiple resource managers are coordinated by a transaction manager; It can also be used in distributed systems where a global transaction manager coordinates the local transaction managers of each subsystem for two-phase commit.

This protocol has two roles,

Node A is the transaction coordinator, and B and C are the transaction participants.

The commit of a transaction is divided into two phases

The first stage is the voting stage

  • 1. The coordinator first writes the command to the log
  • 2 Send a prepare command to B and C
  • 3. After receiving the message, B and C decide whether they can submit the message according to their actual situation
  • 4. Record the processing result in the log system
  • 5. Return the results to the coordinator

The second stage is the decision stage

After node A receives all the acknowledgement messages from participants B and C

  • judgeAll coordinatorsWhether all can be submitted
    • If so, write to the log and issue the COMMIT command
    • If one does not, write to the log and issue abort
  • The participant receives the command from the coordinator and executes the command
  • Writes the executed command and the result to the log
  • Returns the result to the coordinator

What problems might exist?

  • Single point of failure: Once the transaction manager fails, the entire system becomes unavailable

  • Data inconsistency: In Phase 2, if the transaction manager only sends part of the COMMIT message and the network is abnormal, then only part of the participants receive the COMMIT message, that is, only part of the participants commit the transaction, making the system data inconsistent.

  • Long response time: The entire message link is serial and waits for the response, which is not suitable for high concurrency scenarios

  • Uncertainty: After the co-transaction manager has sent a COMMIT and only one participant has received a COMMIT, the re-elected transaction manager cannot determine whether the message was committed when the participant and the transaction manager are down at the same time.

Phase 3 Submission (3PC)

Three-phase commit, also known as 3PC, adds the CanCommit phase and timeout mechanism to 2PC. If no commit request is received from the coordinator within a period of time, the coordinator automatically performs the commit, which solves the problem of the single point of failure of 2PC.

However, performance issues and inconsistencies have not been fundamentally resolved. So let’s look at what the three-stage process looks like.

  • Phase 1: The CanCommit phase

What happens at this stage is simply that the coordinator asks the transaction participant if you have the ability to complete the transaction.

  • If both return yes, the second phase is entered

  • If one returns no or waits for a response timeout, the transaction is broken and an abort request is sent to all participants

  • Phase two: The PreCommit phase

The coordinator sends a PreCommit request to all participants. After receiving the PreCommit request, the participants perform the transaction and record the Undo and Redo information in the transaction log. After the participant completes the transaction (which is the state of an uncommitted transaction), he/she reports back to the coordinator with an “Ack” indicating that I am ready to commit and waiting for the coordinator’s next instruction.

  • Phase three: DoCommit phase

In Phase 2, if all participant nodes are available for PreCommit, the coordinator will change from “PreCommit state” to “commit state.” Then send “doCommit” request to all participant nodes. After receiving the commit request, participant nodes will execute the transaction commit operation respectively, and feedback “Ack” message to the coordinator node. The coordinator will complete the transaction after receiving the Ack message from all participants. Conversely, if one of the participant nodes does not complete the PreCommit feedback or the feedback times out, the coordinator will send abort requests to all of the participant nodes, thus breaking the transaction.

Compensation Transaction (TCC)

TCC is actually the compensation mechanism adopted. The core idea is that for each operation, a corresponding acknowledgement and compensation (undo) operation should be registered. It is divided into three stages:

Try,Confirm,Cancel

  • The Try phase is basically the rightThe service system detects and reserves resourcesIt is divided into two main stages
    • The Confirm phase is mainly to Confirm and submit to the business system. When the Try phase is successfully executed and the Confirm phase starts, the default Confirm phase is error-free. As long as Try succeeds, Confirm must succeed.
    • In the Cancel phase, services that need to be rolled back are cancelled and reserved resources are released.

For example, subtract one inventory from the next order:

Execution process:

  • Try phase: the order system sets the current order status as in payment, the inventory system checks whether the current remaining inventory quantity is greater than 1, and then sets the available inventory quantity to the remaining inventory quantity -1.
    • If the Try phase is successful, the Confirm phase is executed to change the order status to payment succeeded and the remaining inventory quantity to the available inventory quantity
    • If the Try phase fails, execute the Cancel phase, changing the order status to payment failed and the available inventory quantity to the remaining inventory quantity

The TCC transaction mechanism addresses several disadvantages compared to 2PC described above:

  • 1. The single point of coordinator is solved, and this business activity is initiated and completed by the main business party. The business activity manager has also become multipoint, introducing clusters.
  • 2. Synchronization blocking: A timeout is introduced. After the timeout, the system compensates for the timeout, and the entire resource is not locked.
  • 3. Data consistency is controlled by business activity manager with compensation mechanism

In short, TCC is a two-phase commit that is implemented manually by the code, which is written differently for different business scenarios and greatly increases the complexity of the business code, so this pattern is not well reused.

Local message table

Execution process:

  • The message producer needs to build an additional message table and record the sending status of the message. Message tables and business data are committed in a transaction, which means they are committed in a database. The message is then sent through MQ to the consumer of the message.

    • If the message fails to be sent, it will be sent again.

  • The message consumer needs to process the message and complete its own business logic.

    • At this point, if the local transaction succeeds, the processing is successful
    • If the processing fails, the execution is retried.
      • If the fault is caused by a service failure, you can send a service compensation message to the production side to notify the production side of the rollback.

  • The producers and consumers scan the local message table regularly and send the incomplete or failed messages again.

The message transaction

The principle of message transaction is to asynchronously decouple two transactions through message middleware, similar to the above mentioned local message table, but through the mechanism of message middleware, its essence is’ encapsulate the local message table into the message middleware ‘.

Execution process:

  • The PREPARE message is sent to the message middleware
  • After the send is successful, the local transaction is executed
    • If the transaction is successful, commit, and the messaging middleware delivers the message to the consumer
    • If the transaction fails, it is rolled back and the message middleware deletes the PREPARE message
  • The consumer receives the message and consumes it. If the consumption fails, it tries again

This scheme also achieves the final consistency. Compared with the local message table implementation scheme, there is no need to build a message table and no longer rely on the local database transaction, so this scheme is more suitable for high concurrency scenarios. The only solution currently in the market is Alibaba’s RocketMQ.

Best effort notice

The best effort notification solution is simple to implement and is suitable for businesses with low ultimate conformance requirements.

Execution process:

  • System A sends A message to MQ after the local transaction has completed.
  • There’s going to be a service that’s going to consume MQ, and that service is going to consume MQ and call the interface of system B;
  • Ok if system B executes successfully; If system B fails, the best effort notification service periodically tries to call system B again, N times, and finally gives up.

Sagas transaction model

The Saga transaction model is also known as a long-running transaction

The core idea is to split a long transaction into multiple local short transactions, which are coordinated by the Saga transaction coordinator, and if it ends normally it completes normally, and if a step fails, a compensation operation is invoked once in reverse order.

A distributed transaction in the Seata framework contains three roles:

Transaction Coordinator (TC) : maintains the running status of global transactions, coordinates, and drives the commit or rollback of global transactions. Transaction Manager (TM) : Controls the boundaries of global transactions, is responsible for starting a global Transaction, and ultimately initiates a global commit or global rollback decision. Resource Manager (RM) : Controls branch transactions, is responsible for branch registration, status reporting, and receives instructions from the transaction coordinator to drive the commit and rollback of branch (local) transactions.

The SEATA framework maintains an UNDO_LOG table for each RM, which holds the rollback data for each local transaction.

The process is as follows: 1. The TM applies to the TC to start a global transaction. The global transaction is created successfully and a globally unique XID is generated.

2.XID is propagated in the context of the microservice invocation link.

3.RM starts to execute the branch transaction.RM first parses the SQL statement and generates the corresponding UNDO_LOG record. The UNDO_LOG table contains the branch ID, global transaction ID, and redo and undo data executed by the transaction for two-phase recovery.

5.RM inserts service SQL and UNDO_LOG data in the same local transaction. Before committing the local transaction, RM asks the TC for a global lock on the record.

  • If no, it indicates that other transactions are also working on the record, so it will retry within a period of time. If the retry fails, the local transaction is rolled back and the TC is reported to have failed to execute the local transaction.

6. Before the transaction is committed, RM obtains the global lock of related records, commits the local transaction directly, and reports the success of the local transaction to the TC. The global lock is not released. The release of the global lock depends on whether the two phases commit the command or roll back the command.

7. The TC sends the commit or rollback command to the RM based on the execution results of all branch transactions.

  • If RM receives a TC submission command, it immediately releases the global lock on the relevant record, puts the submission request into an asynchronous task queue, and immediately returns the submission result to the TC. When the commit request in the asynchronous queue actually executes, it simply deletes the corresponding UNDO LOG record.

  • If RM receives the rollback command from the TC, it starts a local transaction and finds the corresponding UNDO LOG records based on the XID and Branch ID. Compare the rearmirror in the UNDO LOG with the current data,

    • If not, the data has been modified by an action outside the current global transaction. This situation needs to be handled according to the configuration policy.
    • If they are the same, the rollback statements are generated and executed according to the pre-image information and business SQL information in the UNDO LOG, and then the local transaction is committed to achieve the rollback purpose. Finally, the global lock of related records is released.

conclusion

This paper introduces some basic theories of distributed transaction and explains some commonly used distributed transaction schemes.

Distributed transaction itself is a technical problem, specific use which plan is still in the business needs to choose different business characteristics, but we also find that, a distributed transaction will greatly increase the complexity of the process, will bring a lot of extra overhead work, code volume up, business complex, performance down.

So, when we do real development, we don’t use distributed transactions if we can.