Demand cause

In microservice architecture, with the gradual separation of services, database private has become a consensus, which also leads to the problem of distributed transactions in the process of microservice landing is a very difficult to overcome the obstacle, but there is no complete universal solution.

In fact, not only in microservice architecture, with the gradual increase of user visits, database and even service fragmentation, partition, horizontal split, vertical split have gradually become more commonly used solutions to improve the bottleneck, so more and more atomic operations become cross-library or even cross-service transaction operations. The end result is on the high performance, high scalability, high availability, the pursuit of the path, we begin to relax the pursuit of consistency, but in many situations, especially financial, electricity and other business, inevitably there are consistency issues, make we have to find a mechanism, in order to ensure the consistency of the transaction in distributed environment.

Quote from www.infoq.cn/article/201…

Theoretical foundation

The ACID and BASE

See www.infoq.cn/article/201… See www.txlcn.org/zh-cn/docs/…

2PC

When it comes to distributed transactions, the first thing to talk about is the 2PC (Two Phase Commit) scheme, as shown in the following figure:

2 PC transaction execution is divided into two stages, the first phase, the prepare phase, this phase is actually vote stage, coordinator to confirm whether can jointly submitted to the participants, and got all the participants all answer, coordinator to the all participants commit or rollback together instructions, to ensure the consistency of the transaction. 2PC is the basis of almost all distributed transaction algorithms, and almost all subsequent distributed transaction algorithms are improved from this.

Need the sample

Here we define a top-up requirement, and then we’ll look at how to implement distributed transactions for this requirement in various implementations.

Order and Account are independent services respectively. After recharge, Order should be set as successful and user balance should be increased respectively.

To achieve 1 Seata

Introduction & Framework

Seata (Fescar) is a distributed transaction solution with high performance and ease of use for microservices architecture. Ali is open source and features a transaction manager to manage transactions for each service, essentially an implementation of 2PC (explained later). Seata provides a global transaction manager.

The principle of

Fescar Official this section describes the Fescar global lock

Proxy SQL query, transaction management, similar to middleware.

Realize recharge requirements

Using this scenario to implement requirements, it looks like this:

Both Order and Account plug into Seata to broker transactions.

Code sample

Instead of implementing 2PC yourself, Seata provides a simplified solution, for example, see: Seata Samples

The 2 TCC

introduce

The TRy-confirm-concel (TCC) model is a compensatory transaction, which is mainly divided into three stages: Try: check and reserve resources, Confirm: execute transactions, and Concel: release resources, as shown in the following figure:

The activity manager records the progress status of the global transaction and the execution status of each sub-transaction, and is responsible for pushing each sub-transaction to commit or roll back together. In addition, it is responsible for repeatedly retrying the sub-transaction after the timeout. If the retry fails, the sub-transaction is manually processed to ensure the final consistency of the transaction.

The principle of

Each child node, which implements the TCC interface, can be managed. Advantages: Independent of local transaction, can manage services of non-relational database libraries. Disadvantages: The TCC pattern adds one more state, resulting in increased complexity during business development, increased communication between the coordinator and subtransactions, and more complex state rotation. Moreover, many businesses cannot be compensated, such as bank card recharge.

Implementation framework

tx-lcn LCN distributed transaction framework, compatible with dubbo, spring cloud and Motan framework, supports various relational databases www.txlcn.org

Or Seata MT mode

Code sample

txlcn-demo

Realize recharge requirements

Both oder. done and Account balance + operations need to implement the TCC interface. As you can see, this is really a hassle, and if you can use local transactions, you should try to use local transactions.

Implement 3 transaction messages

introduce

Take the shopping scenario as an example. When Zhang SAN purchases goods and 100 yuan is deducted from his account, 100 points should be added to his account in the downstream member service. Because private database, so lead to in the actual operation process will appear many problems, such as the first to send a message that may result due to buckling failure increase integral without account, if performed before deductions, may be because of service outages, lead to integral cannot be increased, both first message or perform local affairs, and is likely to lead to data inconsistent results.

The essence of transaction message is to solve this kind of problem, solve the atomicity problem of local transaction execution and message sending.

Implementation framework

Apache RocketMQ™ is an open source distributed messaging and streaming data platform.

The principle of

  1. The transaction initiator first sends a prepare message to MQ.
  2. The local transaction is executed after the prepare message is sent successfully.
  3. Return COMMIT or ROLLBACK based on the result of the local transaction.
  4. If the message is ROLLBACK, MQ deletes the prepare message and does not deliver it. If the message is commit, MQ sends the message to the consumer.
  5. If the producer hangs or times out while performing a local transaction, MQ will continuously ask other producers in its group to determine the status.
  6. The successful consumption mechanism on the Consumer side is guaranteed by MQ.

Advantages: Friendly support for asynchronous operations. Disadvantages: The Producer side needs to implement the transaction query interface for RMQ, which increases the complexity in the business development process.

Code sample

// TODO

Realize recharge requirements

With MQ, the Order and Acount operations are guaranteed to either succeed or fail together. Note that if the Account’s balance + fails, there is no way to roll back the Order operation. The Account must ensure that it handles the message correctly.

Implement 4 local message tables

Introduction & Principle

Distributed transaction =A system local transaction + B system local transaction + message notification; Preparation: SYSTEM A maintains A message table log1, whose status is not executed; system B maintains two tables, log2, which is not completed and log3, which is completed. The message middleware uses two topics. Topic1 is A notification from system A that B is going to execute the task, and topic2 is A notification from system B that A has completed the task.

  1. The user picks up the coupon in system A and inserts A record into log1
  2. The scheduled task polls Log1 and sends messages to system B
  3. B After receiving the message, the system checks whether the message has been executed in log3. If not, the system inserts the log2 table and sends an SMS message. After the message is successfully sent, the system deletes the log2 record and inserts the log3
  4. System B sends A message to system A
  5. A The system deletes the message based on the ID

If a network interruption or a system Crash occurs, the transaction needs to be retried to continue. Retry modes:

  1. A scheduled task restores the execution of a transaction,
  2. Messages are delivered using MQ, which ensures that messages are consumed correctly.

Pros: Simple. Disadvantages: The program can be in a halfway state, and retries require each operation to be idempotent.

Note: When implementing idempotent distributed systems, remember to use distributed locks, which are described in the reference article at the end of this article.

Realize recharge requirements

The disconnected transaction is continued through the message table.

conclusion

Let’s start with a summary of these implementations:

Basic principle of implementation advantage Necessary premise
2PC Seata simple Relational database
2PC TCC Do not rely on relational databases Implement the TCC interface
Final consistency Transaction message A high performance Implement the transaction checking interface
Final consistency Local message table decentralized To invade a business, the interface requires idempotency

Each scheme has its own advantages and disadvantages, in the actual use process, we still need to choose different transaction schemes according to the situation to flexibly combine.

For example, service modules A, B, and C exist. Module A is the service of mysql as the data source, module B is the service based on Redis as the data source, and module C is the service based on Mongo as the data source. In order to solve their transaction consistency, it is necessary to adopt different schemes for different nodes and complete the processing of distributed transactions in a unified and coordinated manner.

Solution: It can be solved perfectly by adopting Seata mode for module A and TCC mode for B/C.

Refer to the article

Seata TXLCN Distributed transaction CAP Understanding argumentative Solution Next time someone asks you about distributed locks, throw this article to him

Program little brother introduction

Koala APP system developer Nick focuses on system development and optimization.