Distributed transactions fall into two categories, one is XA type and the other is transaction scheme based on message notification. A few days ago I wrote about distributed transactions -2PC and TCC. This time I will talk about Saga and message-based transaction schemes.
Saga
SAGA first appeared in the 1987 paper SAGAS by Hector Garcaa-Molrna & Kenneth Salem. The core idea is to split a long transaction into a number of local short transactions, coordinated by the Saga transaction coordinator, that are completed if they are completed properly, and that compensation operations are invoked once in reverse order if a step fails.
process
Execute successfully
If we want to conduct a business similar to inter-bank transfer, TransOut and TransIn in different microservices respectively, the typical sequence diagram of a SAGA transaction successfully completed is as follows:
On failure
If a forward operation fails, the compensation operation of each branch needs to be called to roll back, and finally the transaction is successfully rolled back. For each transaction branch, the Action and compensate operations are included.
Compensation for failure
According to the Saga protocol, compensation operations are not allowed to fail, and failure is due to temporary faults or program bugs. If the compensation operation fails, the system tries again until it succeeds. In order to avoid the failure of compensation operation caused by program bugs, it is recommended that developers monitor the global transaction table and send an alarm if the transaction has been retried for more than three times, and the operation and maintenance personnel ask the developer to handle it manually.
Design points
Empty compensation
As in TCC mode, when a network problem occurs, transaction participants may only receive requests for compensation operations, so compensation operations require “null compensation”.
The suspension
As in TCC mode, when a network problem occurs, the transaction participant may receive the compensation operation request before the execution operation, so the execution operation requires “suspension proof”.
idempotence
The implementation of the execute operation and compensation operation must be idempotent.
State machine engine
The Saga pattern is typically implemented through a state machine engine, which orchestrates the invocation flow of services and compensates topological relationships of services. This kind of state machine can be based on MQ asynchronous event-driven, which can greatly improve throughput. In fact, distributed transactions based on event notification can be regarded as a special case of Saga mode.
Pseudo code
In the following single process, we need to use coupons and inventory as examples to write the pseudo-code for generating orders using Saga.
-
Generate order number
-
Calculate which coupons to use, which inventory to use, and determine the true amount of the order
-
Register the execution and compensation actions of using coupons, using inventory, generating orders with the transaction manager and set the execution of using coupons and inventory before executing the generating order
-
The transaction manager records the main transaction and six branch transactions (coupon usage, inventory usage, execution of generating point orders, compensation behavior)
-
Transaction manager calls perform use coupons, use inventory, and generate orders
-
The transaction manager returns success if all forward successes, and if there are failures, the transaction manager compensates (the compensating transaction manager retries again and again) and returns failure
-
The main program, according to the return result, shows the order creation success and failure
Saga summary
Because normally each transaction participant commits directly and does not have to wait for the status of other participants, the concurrency performance in Saga mode is very good. The isolation level is read uncommitted, which means that a transaction can read the state of data that has not yet been committed by another transaction.
As for the access cost of business, only one compensation operation needs to be realized for Saga, and the requirement for the realization of compensation operation is not very high, so the access cost of Saga is relatively low.
To sum up, Saga mode is suitable for scenarios with high requirements on concurrency performance and service access cost. However, for scenarios with low requirements on isolation, if there are legacy systems to be connected, Saga mode is also suitable for Saga mode considering its low access cost.
Message-based distributed transactions
Message-based distributed transactions are very different from the above schemes and are suitable for scenarios with long execution cycles and low real-time requirements. Message-based distributed transaction means that when the transaction initiator sends a message after completing the local transaction, the transaction participants (message consumers) will be able to receive the message and successfully process the transaction. This scheme emphasizes that as long as the message is sent to the transaction participants, the final transaction must be consistent. This is typically done using message-oriented middleware.
The transaction initiator (message producer) sends the message to the message middleware, and the transaction participant receives the message from the message middleware. The communication between the transaction initiator and the message middleware, and between the transaction participant (message consumer) and the message middleware is through the network.
The problem
To implement distributed transactions using messages, three problems need to be solved:
- Atomicity issues with local transactions and message sending
- Ensure that database operations are consistent with sending messages, so that one does not succeed and the other does not
- Reliability of messages received by transaction participants
- Transaction participants must be able to receive messages from the message queue and can repeat receiving messages if they fail to receive messages
- The problem of repeated message consumption
- Due to the existence of the network, if a consumption node times out but consumption succeeds, the message middleware will repeatedly deliver this message, which leads to repeated consumption of messages
process
Using RocketMQ transaction messages, you can solve these problems.
Versions of Apache RocketMQ after 4.3 officially support transaction messages, providing convenience support for distributed transaction implementations. RocketMQ transaction message design is mainly designed to solve the atomicity problem of message sending and local transaction execution on the Producer side. RocketMQ’s bi-directional communication capability between the Producer side and the broker makes the broker naturally exist as a transaction coordinator. RocketMQ itself provides a storage mechanism for persistence of transactional messages; RocketMQ’s high availability mechanism and reliable messaging design allow transaction messages to ensure ultimate transaction consistency in the event of a system exception.
-
Producer sends transaction messages
In this case, Producer is the MQ sender.
The Producer sends the transaction message to the MQ Server, which marks the state of the message as Prepared. Note that the message cannot be consumed by the MQ subscriber.
-
The MQ Server responded to the message successfully
If the MQ Server receives a message from Producer, the MQ Server replies that the message is successfully sent.
-
Producer performs local transactions
The Producer end performs business code logic and is controlled by local database transactions.
-
Message delivery
If the Producer’s local transaction is successfully executed, it will automatically send a COMMIT message to the MQServer. After receiving the commit message, the MQServer marks the state as consumable. At this time, the MQ subscriber is the normal consumption message.
If Producer fails to execute the local transaction, a ROLLBACK message is automatically sent to the MQServer. After receiving the rollback message, the MQServer will delete the message.
The MQ subscriber consumes the message and responds to MQ with an ACK on success, otherwise the message will be received repeatedly. In this case, the ACK automatically responds by default, that is, it automatically responds to the ACK if the program is running properly.
-
Transaction to check
If the Producer fails or times out while executing a local transaction, MQ Server will continuously ask the other producers in the same Producer group to obtain the execution status of the transaction. This process is called transaction backcheck. The MQ Server decides whether to deliver the message based on the result of the transaction callback.
RocketMQ has implemented the main process above. On the user side, the user needs to implement the local transaction execution and the local transaction callback method respectively, so only the execution status of the local transaction is concerned.
Message-based distributed transaction summary
Message-based distributed transactions are suitable for scenarios with long execution cycles and low real-time requirements. With the introduction of message mechanism, synchronous transaction operation becomes asynchronous operation based on message execution, avoiding the influence of synchronous blocking operation in distributed transaction, and realizing the decoupling of the two services.
conclusion
The main types of distributed transactions have been covered, each with different characteristics, and you can choose according to your needs. But these are all theories, we have time to achieve a Demo or look at the source code, after all, to really master knowledge.
If the language is Go, you can check DTM, github address is github.com/yedf/dtm, run Go Run app/main. Go qs to check the effect.
In addition, after using transaction manager, the performance will be affected by TM, so unitary solution can be considered.
data
-
TCC Demo code implementation
-
Hmily implements TCC transactions
-
github.com/yedf/dtm
-
dtm.pub/
-
RocketMQ implements the ultimate consistency of reliable messages
The last
If you like my article, you can follow my public account (Programmer Malatang)
My personal blog is shidawuhen.github. IO /
Review of previous articles:
-
Design patterns
-
recruitment
-
thinking
-
storage
-
The algorithm series
-
Reading notes
-
Small tools
-
architecture
-
network
-
The Go