Hello, everyone! Welcome to Xiao CAI’s solo school. Knowledge is free and you will absorb it! Pay attention to free, do it!

This article focuses on distributed transactions

Refer to it if necessary

If it is helpful, do not forget the Sunday

Wechat public number has been opened, xiao CAI Liang, did not pay attention to the students remember to pay attention to oh!

Life can be mean to you, but technology can't

I went to the grocery store to buy something, paid for it, the boss turned around for a smoke, and forgot I paid for it? How to do this? It’s not surprising that it happens in everyday life. But you place an order on the net, pay finished money, just want to view an order, prompt you to wait to pay however, in the heart tens of thousands of grass mud horse run also unknown! So to prevent this from happening, distributed transactions become very important.

Some people wonder, pay or not pay has anything to do with distributed transactions, this is not a rogue program? But the rogue behind it is because of distributed transactions at work! If not, you may not understand what a transaction is and what a distributed transaction is

Distributed transaction

define

Transactions provide a mechanism to bring all operations involved in an activity into one indivisible unit of execution. All operations that make up a transaction can only be committed if all operations are executed properly, and failure of any operation will result in a rollback of the entire transaction. In short, do or do not.

It sounds a bit man, but don’t get infatuated with it. Let’s first understand the four characteristics of transactions: ACID

  • A (Atomic) : atomicity. All operations that make up an object are either completed or not performed. There can be no partial success and partial failure.
  • C (Consistency) : Consistency. Before and after a transaction is executed, the Consistency constraint of the database is not damaged. Once all transaction actions are complete, the transaction is committed, and the data and resources are in a consistent state that meets the business rules. So for example, if I pay the store owner, I subtract 100, and the store owner adds 100, that’s called consistency.
  • I (Isolation) : The transactions in a database are generally concurrent. Isolation means that two concurrent transactions do not interfere with each other. One transaction cannot see the intermediate status of other transactions.
  • D (Durability) : Specifies that changes made to data by a transaction are persisted to the database and cannot be rolled back after the transaction completes
Single transaction

In the early days, we used a single structure, like a large family, happily living together and working day and night.

Over time, a variety of problems naturally emerge: high complexity, low deployment frequency, poor reliability, limited scalability… The microservices architecture is easy to develop, expand, understand and maintain, not limited to any technology stack, easy to integrate with third-party applications… There are too many advantages, so that the single system gradually fade out of people’s perspective, it seems that if we do not use microservice architecture to develop projects, we will be disconnected from the society. There are many benefits, but the problems will become more complicated. In this section, we are going to take a look at distributed transactions.

Transactions certainly exist in both singleton and microservice, but how do we typically resolve transactions in singleton architectures? Transactional, this annotation alone enables the Transactional transaction to ensure atomicity of the operation.

Distributed transaction

Microservices architecture, in fact, is the traditional unit divided into multiple services, and then the multiple services cooperate with each other to meet business requirements.

Distributed transaction means that transaction participants, transaction supporting servers, resource servers and transaction managers are located on different nodes of different distributed systems.

Now that we’re talking about distributed transactions, let’s take a look at CAP theory in microservices

  • C (Consistency) : Consistency. Service nodes A, B, and C store user data. Data on the three nodes must be consistent at the same time
  • A (Availability) : Availability. Service node A, B, and C. If one node is down, the whole cluster cannot provide services.
  • Fault Tolerance is to allow systems to work together through the network. Fault Tolerance is to solve problems such as incomplete data and inaccessible data caused by network Partition.

We all know you can’t have your cake and eat it too. CAP currently cannot have both, so the current microservices strategy is either CA, CP, or AP. At this time, another theory emerged, that is the BASE theory. It is used to make some additions to CAP theory, and it deserves to be:

  • BA (Basically Available) : Basically Available
  • S (Soft State) : Soft State
  • E (Eventually Consistent) : Final consistency

The core idea of this theory is that if consistency is to be achieved, then each application should adopt appropriate ways to achieve the ultimate consistency of the system according to its own business characteristics.

A scenario

Let’s go back to distributed transactions. When do distributed transactions occur?

Scenario 1: Although it is a single architecture service, distributed transactions will still occur in the case of repository separation. Therefore, the statement of distributed transactions will not occur in the single service

Scenario 2: In a distributed architecture, two services call each other, even though they use the same database, but distributed transactions still occur. You’re using a distributed architecture

Scenario 3: In a distributed architecture, two services call each other and use different databases. In this case, distributed transactions are a no-brainer!

The solution

There is a solution to the problem, of course not, but here, there is a solution to the distributed transaction problem, if not, small vegetables would not write this article to ask for trouble!

Method one: global transactions

I don’t know whether you are familiar with global transactions or two-phase commit (2PC).

Global transactions are implemented based on the DTP model, which specifies that three roles are required to implement distributed transactions:

  • AP (Application) : Application system (microservices)
  • TM (Transaction Manager) : Transaction Manager (Global Transaction Management)
  • RM: Resource Manager (database)

In addition to the role of AP, we also know the other two students are transaction manager and resource manager, so what role they play, then we have to see, the two-phase commit is which two phases!

Stage 1: Voting stage

All participants pre-commit their transactions and provide information on their success to the coordinator

  1. The transaction manager sends oneprepareThe instructions go to servers A and B
  2. After receiving the message, servers A and B determine whether they can submit transactions based on their own conditions
  3. Record the processing results to the resource manager
  4. Returns the processing results to the transaction manager
Phase 2: Execution phase

Based on feedback from all participants, the coordinator notifies all participants and performs commit or rollback in lockstep

  1. The transaction manager sends commit instructions to both servers A and B
  2. After receiving the instructions, servers A and B commit their own transactions
  3. Log the processing results to the resource manager
  4. Returns the processing results to the transaction manager

This is the general process of two-phase commit, which increases the probability of data consistency and is cheaper to implement. But the downside of this approach is obvious!

  • Single point of failure: If the transaction manager fails, the entire system becomes unavailable
  • Synchronous blocking: Delays commit events and lengthens resource blocking events, not suitable for high concurrency scenarios
  • Data inconsistency: If the second phase is executed and the commit result is still unknown, only some participants received the COMMIT message and some did not, then only some participants committed the transaction, data inconsistency will still occur
Method 2: Phase 3 commit

Since two-phase commit isn’t going to solve the problem, let’s go with three-phase commit. The three-phase commit adds a ConCommit phase and a timeout mechanism compared to the two-phase commit. If server participants do not receive commit execution from the transaction manager within a specified period of time, they commit themselves automatically, thus resolving the single failure in both phases.

Let’s look at the three phases of commit:

  • CanCommit: Preparation phase. The only thing to do in this phase is to ask the participants if they are qualified to accept the transaction. This is not too violent.

  • PreCommit: This phase is when the transaction manager sends a request to each participant to prepare the commit. Each participant receives the request or records the result of the processing in his or her resource manager. If ready, the participant sends back an ACK to the coordinator indicating that I am ready to commit.

  • DoCommit: This is the transition from pre-commit state to commit state. The transaction manager sends a commit request to each participant, who, upon receiving the request, performs the commit of his own transaction. If one participant fails to complete the PreCommit or if the response times out, the coordinator sends abort requests to all participant nodes to break the transaction.

In fact, the three-phase commit looks like changing the commit phase of the two-phase commit into the pre-commit phase and commit phase.

In fact, it can be seen from the above that the three-phase commit only solves the problem of single failure in the two-phase commit, because the timeout mechanism is added, and the timeout mechanism applies to the pre-commit stage and the commit stage. If waiting for a pre-submit request times out, the participant has said nothing and gone straight back to the preparation phase. If the commit request times out, the participant commits the transaction.

So you can see that the three-phase commit does not solve the problem at all, although it is a little bit better than the two-phase commit

Method 3: TCC

TCC (Try Confirm Cancel), which is a compensating distributed transaction. The idea is to register a corresponding acknowledgement and compensation (undo) operation for each operation. TCC implements distributed transactions in three steps:

  • Try: Tries services to be executed

This process does not execute the business, but just completes the consistency check for all the business and reserves all the resources required for execution

  • Confirm: Indicates that the service is executed

Confirm that the service operation is performed without any service check. Only the service resources reserved in the Try phase are used. In general, TCC assumes that the Confirm phase is infallible. If the Try succeeds, Confirm succeeds. If Confirm fails, a retry mechanism or manual processing is required

  • Cancel: Cancels the service to be performed

Cancel service resources reserved for the Try phase. In general, TCC assumes that the Cancel stage is also guaranteed to succeed. If the Cancel stage does go wrong, a retry mechanism or manual processing should also be introduced

TCC is a distributed transaction at the business level that is ultimately consistent and does not hold a lock on a resource all the time. Its advantages and disadvantages are as follows:

Advantages: The database layer of the two phase submission mentioned the application layer to achieve, to avoid the low 2PC performance of the database

Disadvantages: The Try, Confirm, and Cancel operation functions of TCC need to be provided by services, resulting in high development costs. TCC is highly intrusive and tightly coupled to services, so operations need to be designed according to specific scenarios and business logic

Method 5: Reliable message transactions

The principle of message transaction is to decouple two transactions asynchronously through messaging middleware. The scheme based on reliable message service ensures the consistency of upstream and downstream application data operation through message middleware. Suppose there are two services, A and B, and the distribution can handle two tasks, A and B. In this case, there needs to be A business process to process task A and B in the same thing, which can be realized by message-oriented middleware.

The overall process can be divided into two major steps: service A publishes messages to the messaging middleware and message A delivers messages to service B

Step 1: Service A publishes messages to the messaging middleware

  1. Before service A processes task A, it first sends A half-message to the messaging middleware
  2. Message-oriented middleware persists the message after receiving it, but does not deliver it. After successful persistence, A confirmation reply is returned to service A
  3. After service A receives an acknowledgement, it can start processing task A
  4. After task A is completed, service A sends A Commit or Rollback request to the messaging middleware. After the request is sent, service A’s work is complete and the transaction is completed
  5. After the messaging middleware receives the Commit, it delivers a message to service B. If it receives Rollback, it discards the message

If the messaging middleware does not receive Commit or Rollback instructions from service A for A long time in the final process, it will need to rely on timeout queries

Timeout query mechanism:

In addition to implementing normal business processes, service A also needs to provide an interface for transactional inquiries from messaging middleware. In the message middleware for the first time after receiving the message will begin to timing, if in excess of the prescribed time not received subsequent instruction, will take the initiative to call A service provided by the transaction inquiry interface, ask the current state of the service, usually the interface returns three as A result, the middleware need according to the results of the three different make A different treatment:

  • Commit: The message is delivered directly to service B
  • Rollback: The message is discarded directly
  • Processing: Continue to wait and reset the timer

Step 2: The messaging middleware delivers messages to service B

After receiving the Commit instruction from service A, the message-oriented middleware delivers the message to service B and then sets its state to blocked wait. After receiving the message sent by the messaging middleware, the service B begins to process task B and sends a response to the messaging middleware when the processing is complete. But there are also problems when the messaging middleware blocks waiting

  • Normal: After the message-oriented middleware delivers the message, it enters the blocking wait state. After receiving the acknowledgement reply, the transaction is considered complete and the process ends
  • Wait timeout condition: After waiting for an acknowledgement reply to time out, the post is reposted until server B returns a successful consumptionresponse. The number and interval of message retries can be set, and human intervention is required if the delivery fails.

The reliable messaging service solution is the ultimate consistency. Compared with the local message table implementation, there is no need to create a message table. Do not rely on local database transactions, suitable for high concurrency scenarios. RocketMQ does a good job of supporting message transactions.

Method 4: Do your best to inform

Best effort notification also becomes periodic proofreading, which is a further optimization of the reliable messaging service. It introduces a local message table to record error messages, and then adds periodic proofreading of failure messages to further ensure that messages are consumed by downstream services.

Again, like message transactions, there are two steps:

Step 1: Service A sends A message to the messaging middleware

  1. In the same transaction that processes the business, a record is written to the local message table
  2. The message sender continually fetches messages from the local message table and sends them to the message middleware, and retries if sending fails

Step 2: The messaging middleware delivers a message to service B

  1. After receiving the message, the message-oriented middleware delivers the message to downstream service B, and service B executes its own service after receiving the message
  2. When the service B service is successfully processed, a feedback response is returned to the message-oriented middleware. The message-oriented middleware can delete the message and the process ends
  3. If the messaging middleware fails to deliver a message to service B, it tries to retry, and if the retry fails, it adds the message to the failure message table
  4. The message middleware also needs to provide an interface for querying failure messages. Service B periodically queries failure messages and consumes them

The maximum effort notification scheme is simple to implement and is suitable for some businesses with low final consistency requirements.

Seata

Concept of Seata

Since distributed transaction processing is so troublesome, can we make distributed transaction processing as simple as local transaction? Of course that’s our vision. Of course this vision is what all developers want. The Alibaba team did just that, launching the open source project Seata (Simple Extensible Autonomous Transaction Architecture). This is a distributed transaction solution designed to address all aspects of distributed transactions that developers encounter.

Seata is designed to be business-free, so it improves on the traditional two-phase commit (global transaction) by starting with business-free two-phase commit (global transaction). It understands a distributed transaction as a global transaction containing several branch transactions. The responsibility of a global transaction is to coordinate the branch transactions it manages to achieve consistency, either committing together successfully or rolling back together when it fails. That is to say, both rise and fall

An Seata

There are several important roles in Seata:

  • Transaction Coordinator (TC) : Transaction Coordinator. Manages the state of global branch transactions for commit and rollback of global transactions.
  • Transaction Manager (TM) : Transaction Manager. Use to start, commit, or roll back a transaction.
  • Resource Manager (RM) : indicates the Resource Manager. It is used for resource management of branch transactions, registering branch transactions with TC, reporting the status of branch transactions, and receiving TC commands to submit or roll back branch transactions.

This is a very clever design, let’s look at the picture:

The execution process is like this:

  1. When A TM in service A requests A TC to start A global transaction, the TC creates A global transaction and returns A unique XID
  2. The RM in service A registers the branch transaction with the TC and then brings the branch transaction into the global transaction jurisdiction corresponding to the XID
  3. Service A starts executing branch transactions
  4. Service A starts calling service B remotely, at which point the XID is propagated along the call chain
  5. The RM in service B also registers the branch transaction with the TC and then brings the branch transaction into the global transaction jurisdiction corresponding to the XID
  6. Service B starts the branch transaction
  7. After the global transaction invocation process is completed, TM will initiate the commit or rollback of the global transaction to TC based on the error exception
  8. The TC coordinates all branch transactions under its jurisdiction and decides whether to commit or roll back

Use Seata

Now that we’ve seen the composition and execution of Seata, let’s put Seata to practical use.

Example demonstrates

We simply created a microservice project with an order service and an inventory service.

We have used NACOS as the registry and started two services respectively. We can see two registered services in the NACOS console:

Extra: If you are not familiar with NacOS, you can jump to see the explanation of NacOS: The new micro services of NacOS

We then create a database with two tables: C_ORDER and C_Product, where there is one data in the goods table and no data in the order table, which we are going to manipulate!

We now simulate an ordering process:

  1. Request in, through the product PID to check the product information in the database
  2. Create an order for this item
  3. The inventory of the commodity is deducted accordingly
  4. End of the process

Let’s get into the code demo:

Note: ProductService is not a class in the inventory service, but an interface that uses Feign to call the inventory service remotely

Code three steps, normal please certainly is no problem:

The order is generated and the inventory is reduced accordingly. When we feel that our code can go online and get on the right track, we will simulate the abnormality in the inventory. The number of goods in stock is 100 and the order table is cleared:

As we continue to send the order request, we can see that the inventory service has thrown an exception

Normally at this point, the inventory table should not be reduced and the order table should not insert order data, but is this really the case? Let’s look at the data:

The inventory stayed the same, but orders increased. Well, here you have the disastrous consequences of distributed transactions. Next comes the hero!

Seata installation

We first need to download Seata by clicking the download address.

Since we are using NacOS as the service center and configuration center, we need to make some changes after downloading and unpacking

  • Enter theconfThe directory editorregistry.conf 和 file.confTwo files, edited as follows:

  • Due to the newSeataThere is nonacos-conf.sh 和 config.txtTwo files, so we need to download them independently:

Nacos-config. sh Download address

Config.txt Download address

We need to put the config.txt file in the seata directory, not the conf directory, and we need to modify the config.txt content

Config.txt is the various detailed seata configurations, which can be imported into nacos by executing nacos-config.sh, so that we don’t need to put file.conf and registry. Conf in our project, we can read what we need directly from Nacos.

  • Perform the import

Open the git bash window in the conf directory and run the following command:

Sh -h localhost -p 8848 -g SEATA_GROUP -t namespace-id(to be replaced) -u nacos -w nacosCopy the code

After the operation, we can see the configuration list in the NacOS console. If the configuration needs to be modified in the future, we can directly modify it from here without modifying the directory file:

  • Database Configuration

SQL files are still not available in the latest version of 1.4.1, so we still need to download another: SQL download address

Execute this file in seATA data to generate three tables:

Execute the undo_log table in our business database:

CREATE TABLE `undo_log`
(
    `id` BIGINT(20) NOT NULL AUTO_INCREMENT,
    `branch_id` BIGINT(20) NOT NULL,
    `xid` VARCHAR(100) NOT NULL,
    `context` VARCHAR(128) NOT NULL,
    `rollback_info` LONGBLOB NOT NULL,
    `log_status` INT(11) NOT NULL,
    `log_created` DATETIME NOT NULL,
    `log_modified` DATETIME NOT NULL,
    `ext` VARCHAR(100) DEFAULT NULL.PRIMARY KEY (`id`),
    UNIQUE KEY `ux_undo_log` (`xid`, `branch_id`)
) ENGINE = INNODB
AUTO_INCREMENT = 1
DEFAULT CHARSET = utf8;
Copy the code
  • Adding a log file

If we do not have the log output file, starting seATA might give an error, so we need to create the logs folder under the seata directory and the seatA_gC.log file under the logs file

  • Start the

With the above preparations in place, we can start seata by executing the bat script directly in the bin directory. After starting, we can see the Seata service in nacOS:

Seata integration

In the Seata installation step, we completed the Seata server boot installation, followed by the integration of the Seata client into the project

  • Step 1: We need to be inpom.xmlAdd two dependencies to the file:Seata rely on 和 Nacos configuration dependencies
<! --nacos-->
<dependency>
    <groupId>com.alibaba.cloud</groupId>
    <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>

<! --seata-->
<dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-seata</artifactId>
            <exclusions>
                <! -- exclude dependencies on the specified version and server side -->
                <exclusion>
                    <groupId>io.seata</groupId>
                    <artifactId>seata-all</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.seata</groupId>
                    <artifactId>seata-spring-boot-starter</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>io.seata</groupId>
            <artifactId>seata-all</artifactId>
            <version>1.4.1</version>
        </dependency>

        <dependency>
            <groupId>io.seata</groupId>
            <artifactId>seata-spring-boot-starter</artifactId>
            <version>1.4.1</version>
        </dependency>
Copy the code

Note: Here we need to remove the seATA dependencies inherent in Spring-cloud-starter-Alibaba-Seata, and then introduce our own version of SeATA, No available server to connect error may occur if the version is not consistent.

  • Step 2: We need to putrestry.confThe files are copied to the project’s Resource directory

  • Step 3: You need to configure the SEATA proxy data source yourself
@Configuration
public class DataSourceProxyConfig {
    @Bean
    @ConfigurationProperties(prefix = "spring.datasource")
    public DruidDataSource druidDataSource(a) {
        return new DruidDataSource();
    }

    @Primary
    @Bean
    public DataSourceProxy dataSource(DruidDataSource druidDataSource) {
        return newDataSourceProxy(druidDataSource); }}Copy the code

After configuring the data source, we need to exclude the Druid data source dependency on the SpringBootApplication of the startup class. Otherwise, we may get a cyclic dependency error:

@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
Copy the code
  • Step 4: Add the transaction group item for our service to the profile console of NACOS:
service.vgroupMapping + Service name = default
Group as follows: SEATA_GROUP
Copy the code

  • Step 5: Modify the configuration in the project

  • Step 6: Enable global transactions

This is the final step, adding the @GlobalTransactional annotation to the method we need to start the transaction, similar to the @Transactional annotation we added for our single transaction

Seata test

Back in the project, we saw in the example demonstration above that if an exception occurs to the inventory service, the inventory is not reduced and the order is still generated. Would that change if we added Seata to manage global transactions? Our tests are as follows:

The inventory service has been abnormal:

Take a look at the database data:

It looks like our global transaction is in effect and the transaction is under perfect control!

The undo_log table we created also initiates an important role in managing transactions:

After seeing the above operation, we strike while the iron is hot to comb the execution process, so that you are more impressed

After reading this diagram, you will be more familiar with the process of Seata executing transactions.

That’s not the end, but let’s take a look at some of the highlights:

  1. Each RM needs to connect to the database using DataSourceProxy in order to use ConnectionProxy. The purpose of using the data source and data ConnectionProxy is to put undo_log and business data into a local transaction commit in the first phase. This saves that whenever there is a business operation there must be undo_log generated!
  2. Undo_log before and after data changes are stored in phase 1 in preparation for transaction rollback, so phase 1 commits the branch transaction and frees the lock resource!
  3. When TM turns on the global transaction, it puts the XID into the context of the global transaction, and we pass the XID into the downstream service through the Feign call. Each Branch transaction associates its Branch ID with the XID!
  4. In the second stage, if the global transaction is committed normally, then TC will notify each branch participant to commit the branch transaction, and each participant only needs to delete the corresponding undo_log, and can execute asynchronously!
  5. In the second stage, if the global transaction needs to be rolled back, the TC will inform each Branch transaction participant to roll back the Branch transaction, find the corresponding undo_log log by XID and Branch ID, generate the reverse SQL by rollback log and execute it, and complete the state before the transaction submission. If the rollback fails, the rollback operation will be retried!

END

Here, a distributed transaction is finished, we review, from the five solutions of distributed transaction to the use of Seata, xiaokai students are really well-intentioned ~ get down to business, after seeing how much absorbed, move your hands, write code, let knowledge and you closer ~

Today you work harder, tomorrow you will be able to say less words!

I am xiao CAI, a man who studies with you. 💋

Wechat public number has been opened, xiao CAI Liang, did not pay attention to the students remember to pay attention to oh!