preface

When discussing distributed systems, we must be familiar with distributed transactions. In this article, we will discuss the basic theory of distribution. This paper will start from local affairs to CAP and BASE theory, and finally focus on introducing the principles and similarities and differences between 2PC and 3PC.

Local transactions

Before we talk about distributed transactions, let’s look at what a local transaction is. Local transactions are often referred to as database transactions, or traditional transactions. This is implemented via Spring’s @Transaction annotation. It ensures ACID properties. Then we need to think, if the implementation of local transactions is very mature, why do we still have distributed transactions? Because with the emergence of mass data processing and distributed technology, the traditional stand-alone database or stand-alone service has been unable to meet our needs. At this time, some people in the industry put forward the CAP theory and gave relevant demonstration.

Theory of CAP

What is CAP?

Consistency, Availability, and Partition tolerance in a distributed system. The CAP principle is that, at best, these three elements can achieve two, not all at once.

(Source: Baidu Baike)

What I want to talk about here is partition fault tolerance? Take a specific example: the commodity library or service of a mall is deployed in a mobile computer room in Shanghai. Under normal circumstances, it can maintain communication with the outside world and provide services. However, one day, due to road construction, a passer-by accidentally cut the outlet optical fiber of the machine room. As a result, users cannot browse the whole commodity information before the repair is completed. Therefore, to avoid this situation, it is necessary to deploy the goods warehouse or service in Shanghai and Nanjing respectively, so that even if one has a problem, the machine room in the other location can still provide service. This is called partition fault tolerance, that is, through the network partition, to a certain degree of fault tolerance. It is also possible that both machine rooms will fail. In this case, multiple locations can be backed up to improve system availability. However, the more machine rooms backup, the availability is improved, but also introduces another problem, is the problem of data consistency.

For example, if there is only one inventory, after the order is placed in Zhangsan, the quantity of goods in Shanghai machine room will be deducted by 1 immediately, and then the inventory in Nanjing and Beijing will also be cleared. But because the network of Beijing machine room just appears big delay, and at the moment the user Li Si comes to place an order, then there will be oversold situation at this time, the reason is to improve the availability of the system and service, and reduce the consistency of data.

So when we design architecture, we often choose between conformance C and usability A. Zookeeper, for example, retains consistency features, while Eureka retains availability features.

The BASE theory of

Since CAP theory could not satisfy both usability and consistency at the same time, someone put forward BASE theory at this time. Let’s look at the definition

BASE theory is the result of balancing consistency and availability in CAP. The core idea of the theory is: we cannot achieve strong consistency, but each application can adopt appropriate ways to achieve the final consistency of the system according to its own business characteristics. (Source: Baidu Baike)

According to the definition, BASE theory is actually an evolutionary theory that ensures the final consistency on the basis of ensuring the usability of CAP theory. For several types of final consistency, see the figure below.

2PC two-phase commit transaction

After looking at some of the theoretical concepts of distribution, let’s talk about what a distributed transaction is. The so-called distributed transactions are compared to traditional local transactions. In the microservices era, a single request often involves multiple service invocation chains, and these services are deployed on many machines, and it is completely impossible to rely on traditional database transactions to guarantee this. And that’s when 2PC comes into play.

Let’s get familiar with a few terms: 2PC: Two Phase Commitment Protocol, as shown in the figure below. There are Two stages: pre-commitment and final Commitment. Hence the name. TM: Transaction manager, or coordinator. Responsible for the scheduling of transaction flow operations. RM: Resource manager, or participant. Responsible for the execution of transactions.

So with that in mind, let’s look at 2PC. Here is the flow chart of 2PC’s execution:

• A phase:

At this point, the coordinator TM will issue the first request to all participant RMS participating in the transaction, asking all RMS to attempt an SQL operation without committing (just write UNDO and REDO log files to the transaction log) and give feedback. At this point, TM will issue a second commit request to all RMS only after all participants have returned OK. If one RM fails or times out, the system sends a rollback request to rollback.

• Two stage

Unlike phase 1, this phase commits or rolls back SQL. TM will get the “commit” or “rollback” instruction according to the feedback given by each RM in the first phase. If the instruction given by TM is commit, the phase will execute the commit operation to complete the transaction. If TM gives a rollback, the phase is rolled back to complete the transaction interrupt.

At this point, the two-phase transaction commit is complete, isn’t that simple? But here, we need to consider the pros and cons of phase 2.

Advantages: simple principle, convenient implementation

Disadvantages: Synchronous blocking During all phases of a two-phase commit, all nodes are waiting for responses from other nodes and cannot perform other operations. This synchronization blocking greatly limits the performance of distributed systems.

The single point of issue coordinator is important throughout the two-phase commit process, and if the coordinator TM has a problem during the commit phase, for example, the service goes down, then the whole process will not work. More importantly, other participants will remain in a “locked transaction resource” state and will not be able to continue to complete the transaction. 2PC transaction management, only RM participants have timeout mechanism, TM coordinator does not. Therefore, when TM fails to receive feedback from RM, it can also judge and deal with it according to the timeout mechanism. But when TM itself dies, there is no way to tell.

Data inconsistency Suppose that after the coordinator TM sends commit requests to all participants, a local network exception occurs, or the coordinator crashes itself before all commit requests are sent, resulting in only some participants RM receiving commit requests. This can lead to serious data inconsistencies.

Poor fault tolerance If the RM of the participant fails in the submission query stage of the two-phase submission, the coordinator cannot obtain the confirmation information of all the participants all the time, then the coordinator can only rely on its own timeout mechanism to determine whether to interrupt the transaction. Clearly, this strategy is too conservative. In other words, the two-phase commit protocol does not have a well-designed fault tolerance mechanism, and the failure of any node will lead to the failure of the whole transaction.

Three cases of 2PC single point problem

• The TM coordinator is normal, but the RM participant is down

At this point, TM could not receive part or all of the feedback from RM, which would cause the block of the whole node. Therefore, 2PC introduced a timeout processing mechanism. When TM failed to receive feedback from RM in time, TM would initiate a rollback operation to terminate the transaction.

• The RM participant is normal, but the TM coordinator is down

The coordinator goes down, which blocks the entire transaction node. Therefore, we can back up and synchronize operation logs by backing up TM nodes. When the TM breakdown is detected, the TM backup machine is enabled and transaction operations are resumed according to the backup logs.

• Both the TM coordinator and the RM participant went down

There are three cases to analyze

• In phase 1: In phase 1, due to the participant RM, the submission was not made, a new coordinator was re-voted, and phase 1 and Phase 2 could be repeated.
• The transaction is not committed, but phases 1 and 2 can be executed again after both sides recover. It’s actually similar to the first case.
• In the second phase, TM sent part of the COMMIT or rollback to other RMS. That is to say, some participants have finished the COMMIT operation and some hang up without receiving the command. At this point, data inconsistency occurs. This problem 2PC has no solution, which requires the compensation scheme described below.

Compensation scheme for 2PC

It has been discussed previously that 2PC has inconsistent data. How to deal with these problems? There are three commonly used schemes: manual compensation, scheduled task, script compensation and so on.

conclusion

Based on 2PC distributed transaction solution, the principle and implementation are relatively simple, can solve most of the distributed transaction problems. However, because it locks resources during the whole process and causes synchronization blocking, it is not suitable for high concurrency scenarios. And there may be the need for manual or script intervention data inconsistency, of course, the current technology, 2PC such as flexible distributed transactions, it is really not able to ensure the consistency of data like local transactions, can only constantly reduce the probability of error.

3PC three-phase commit transaction

3PC is an improved version of 2PC, which also includes participants and coordinators. Let’s analyze the 3PC process in stages.

• A phase

A diagram of this stage is shown above. The coordinator TM issues the command to obtain the query to all participants RM, and RM will try to obtain the lock of the database after receiving the command. When done, if the lock is available, return Yes and prepare for the preparatory phase of phase 2; If No lock is obtained, feedback No is given. In this phase, resources are not locked, so the performance is greatly improved compared with 2PC.

• Two stage

If Yes is returned in the first phase, the phase is entered. From this stage, resources are locked. This phase is similar to the first phase of 2PC in that SQL is executed but not committed. UNDO and REDO files are written to the transaction log. After completing this phase, each participant RM will give feedback “ACK” for success or “No” for failure to coordinator TM.

• Three phase

The three phases are divided into two scenarios: commit and rollback.

1. submit

There are two cases where RM commits a transaction in three phases, as follows:

• When TM sends ACK to all RM, RM commits the transaction after receiving ACK instruction. When done, the response is given and the locked resource is released
• If TM breaks down or the network times out, RM cannot receive the command sent by TM immediately. In this case, RM triggers the default timeout commit. This is compared to an optimization of 2PC, where a TM failure would block resources. However, in 3PC, RM will commit by default. The designer of 3PC believes that if the first two stages are OK, then the third stage is also likely to be OK, so the mechanism of automatic submission is provided.

2. Roll back:

When TM issues the rollback command, RM will rollback the previous operations, interrupt the transaction, and release resources such as database locks.

• 3 PCS to summarize

3PC made two improvements on the basis of 2PC:

• Added a canCommit phase, which reduces the probability of resources being locked during a block because it does not lock resources.
• Timeout mechanism is introduced in TM and RM: 2PC only sets timeout mechanism in TM (if TM cannot receive feedback from RM in time, it will trigger rollback). The automatic timeout submission mechanism is also configured in RM of 3PC.

We conclude with a flow chart to illustrate the 3PC process

conclusion

This paper introduces the theoretical basis of CAP, BASE, 2PC and 3PC. These theories are the theoretical basis of many distributed transaction frameworks. Understanding them is helpful for our understanding of the framework.