This is my 42nd original article
Today we’ll talk about various solutions for data consistency in distributed environments. It’s a little brainy, but it’s fun.
Consistency issues in monomer environments
In the traditional relational database environment (single environment, or called single machine), you can record your own, the single machine environment does not have the problem of environmental data consistency, but the problem of concurrent operation data consistency.
For a database operation (the technical term is “transaction”), there are four properties: atomicity, consistency, isolation, and durability (ACID). All we need to know is that these properties guarantee the accuracy of the data in the database.
- Atomicity: All operations in a transaction, indivisible, either all succeed or all fail;
- Consistency: Data integrity before and after a transaction must be consistent.
- Isolation: the execution of a transaction cannot be disturbed by other transactions. In case of multiple concurrent transactions, transactions must be isolated from each other.
- Durability: Once a transaction is committed, its changes to data in the database are permanent.
To prevent many people from working together, databases make heavy use of locks. Lock is when you operate this data, add a status bit, others can not operate, similar to when you go to the bathroom to lock the door
In the figure above, there are two business operations that need to modify their level at the same time, but business operation 3 got the lock, so after the operation, Guan Yu’s level will be changed to 35. Operation 2 fails because the record is locked.
Standalone database environment (traditional database), most of the use is pessimistic lock, which means to use a pessimistic attitude towards database modification, assuming that any time to take data may encounter someone will modify:
- When you modify this data, the database will automatically help you to record a lock, others can not change the data when you modify at the same time, this is called line lock;
- When you change the table, the database also locks the table, which is called a table lock;
- Locking when you read data is read lock;
- Locking when you write data is called write locking.
Consistency issues in distributed environments
Distributed environments, however, are different. Lock is faced with concurrent tasks to solve the problem of multiple tasks to grab a resource. This problem is also encountered with distributed locks.
But distribution itself leads to another problem: distribution is a clustered environment, and a clustered environment stores multiple copies. This is where the problem arises. Since one copy of data stores multiple copies, how can a cluster ensure that the data in multiple copies is the same? And how do we make sure that when we modify the data, we read the data, the data that we just modified?
The reason for this problem is that the distributed environment is a cluster composed of many small databases, and the abnormal network communication is the premise of cluster construction. That is, when business operation 1 modifies the guan Yu level, it is possible to have inconsistent data for three replicas. Copies 1 and 2 have been modified, while copies 3 have not been modified due to network delay. When business operation 2 reads guan Yu level, how to ensure that the latest data can be read? This is the problem of data consistency in a distributed environment.
The CAP theorem
In 2000, Eric Brewer proposed the CAP conjecture at an international conference. In 2002, Lynch and others proved the Brewer conjecture. The proved CAP theorem is as follows:
-
C Consistency, that is, the data of all copies is consistent;
-
A High availability, that is, some nodes fail and can respond to the client’s read request \
-
P partition fault tolerance, distributed environment, that is, when the network partition, can also enable the client to get the latest data
-
C\A\P cannot be satisfied at the same time, only two of them can be satisfied at most.
P (partition fault tolerance) is required, that is, CP and AP are the only two choices left. Either abandon data consistency and pursue high availability, or abandon high availability and pursue data consistency. And neither of those outcomes is what we want. How can I do that?
The BASE theory of
There are always heroes in this world. Dan Pritchett, an architect at eBay, offers the BASE theory:
- Basically Availble (A in CAP, high availability)
- Soft-state, Soft state/flexible transaction
- Eventual Consistency (C in CAP, data Consistency)
In all non-financial application scenarios, we follow the BASE theory and provide services that are basically available and ultimately consistent.
Quorum mechanism
It is based on CAP and BASE theory that Quorum mechanism is developed to solve the problem of data consistency in distributed environment. Quorum, also known as NRW, allows for different scenarios in a cluster, as long as new data is guaranteed to be read.
- N= Total number of nodes
- R=read, number of copies read
- W=write, number of copies to be written
- The total number of copies needed to read to get the latest data: R= n-w +1, the larger W is, the smaller the probability of writing, the smaller the pressure of reading; The smaller W is, the higher the write probability is, but the more nodes are read, and the worse the read performance is
The above figure shows an example. Assuming that only two nodes are successfully modified when business operation 1 modifies guan Yu level, then business operation 2 must read R= n-w +1=3-2+1=2 copies to obtain the latest data. You see, we don’t need all the nodes to be consistent to get the data available. As for the node that has not been changed, the data synchronization mechanism of the cluster will be synchronized slowly.
Is there a way to write to all nodes at once? Yes, there are many ways.
Strong consistency solution
Currently, there are four mainstream strong consistency solutions:
- 2PC (2 Phase commit) 2 Phase commit method
- 3PC (3 Phase Commit) 3 phase commit method
- 4PC (laughter) TCC (try-cancel-commit
- Messaging middleware consistency solution
It’s basically the same logic as the TCP/IP three-way handshake. 2PC goes through two phases and finally submits the transaction. 3PC is to go through three stages and finally submit the transaction; TCC is try, cancel, submit; Half MQ is to pit, confirm and submit.
- 2PC
2PC is divided into two stages: voting stage and operation stage. Since it’s a vote, there has to be one person to coordinate, otherwise how to ensure fairness. So 2PC introduces a coordinator. A distributed transaction request comes in, goes to the coordinator, and the coordinator initiates the first stage, the voting stage, and asks all participants: Are you ready? Everyone says: Yes! And then the coordinator initiated the second stage, the submission stage, and said, guys, change. All participants then initiate a local transaction to change the level of about to 34. If someone says, “I’m not ready yet,” then the coordinator initiates a second phase, only it becomes a cancel action.
It’s like a running race. Stage one: the judge says, “Get ready.” Stage 2: The referee shouts “Run!”
2PC consumes more resources, because everyone’s resources are locked, and it can only be submitted/cancelled after everyone responds. In case of node or even coordinator dropping out, it will be troublesome, and all have to wait.
- 3PC
3PC is an improved version of 2PC, adding a “pre-execute” Percommit phase between the two phases of 2PC, so there are three phases: CanCommit, Percommit, and DoCommit.
This is just like a running race. Stage one: the referee shouts “Everyone in position ~~”; Stage 2: the referee shouts “Ready ~~~”; Stage 3: The referee shouts “Run!”
3PC because of the addition of pre-executed operations, resources will not be locked for a long time, and the rule of automatic submission of timeout is added, the efficiency is greatly increased.
But what if some node times out, some node tells the coordinator that it is going to fail, and the coordinator cancels the task? This still leads to data inconsistencies.
- TCC
TCC is actually implemented in the business logic layer by programmers who write the business code. 2PC and 3PC are implemented by the resource layer and encapsulated by the underlying logic. TCC is Try, Cancel, Commit, which is the same as 2PC. Ready – Cancel or execution. So a lot of programmers look at TCC and 2PC and see the similarities. TCC has the advantage of applying its own granularity of database operations, reducing lock conflicts and increasing throughput.
The problem with TCC is that TCC is written once for each business. It’s too complicated. Is there a better solution?
- Messaging middleware consistency solution
Consistency solutions for messaging middleware are divided into regular MQ and Half MQ Half messages (transactional messages).
Now we don’t have to ask all the time, and we don’t have to experiment with business logic all the time. Things that need to be changed are thrown directly into MQ, and other components periodically consume messages in MQ and do as required. Totally decoupled, perfect!
But MQ itself has a few minor issues, such as upstream applications finishing processing and having a problem throwing messages to MQ, which is dead and inconsistent. This is where the semi-message (transaction message) comes in.
Half MQ means splitting the message twice and doing a 2PC-like operation.
This avoids the pitfalls of normal MQ. The producer needs to wait for a half-message acknowledgement to succeed before writing a local transaction. MQ has confirmed that the message is OK, and then delivers it to the consumer, who will consume it.
We found that a simple thing became very complicated in a distributed environment. It’s as hard to make a decision as it is to make a decision as a group. If it’s up to you, it’s up to you. Collective decisions require a wide variety of rules, or everyone will argue with each other. \
Fortunately, computers are so much simpler than people that they do what we say. Computers are much easier to deal with than people.
And there’s always a solution to all kinds of problems, and we’re always approaching perfection, but it doesn’t seem to be perfect, there’s always some little problem. Is it too much for me to stand on the shoulders of giants and lament the imperfections of the world?
Or more ~ ~ ~ \
Highlights from the past
Through the middle dry | the architect’s point of view
The underlying rules of data driven business | theory of “force”
Hot article | system of big data engineer career path whole solution
Help me to click “like” + “watching”, this year’s wages are rising!
