2021.09.14 update

0 foreword

This article was written in 2019, and this introduction was written in 2021 after a two-year break, fixing many of the errors in the article and adding more detailed steps and testing procedures.

This article mainly describes the four features of transactions and dirty read, magic read, unrepeatable read, lost update, there are four isolation levels, and in the four isolation levels of the dirty read, magic read, unrepeatable read were tested, in addition to the MySQL part of the lock (X lock, S lock, optimistic lock, pessimistic lock).

Test environment:

  • Database: used locallyDockerTo create theMySQL
  • Version:8.0.25
  • Engine:InnoDB

1 ACID

The four characteristics of transactions are ACID.

1.1a: Atomicity

Atomicity refers to the fact that a transaction either executes completely or not at all.

1.2C: Consistency

When the transaction completes, the data must be in a consistent state. If an error occurs during the execution of a transaction, it is rolled back to the state before the previous transaction was executed, so that the data remains in a consistent state. If the transaction fails and is not rolled back, some of the changes are written to the database, then the data is in an inconsistent state.

1.3I: Isolation

When multiple transactions are processed simultaneously, the execution of one transaction cannot be interfered with by another, and the internal operations of the transaction are isolated from other concurrent transactions.

1.4 D: Durability

After a transaction commits, changes to the data are permanent.

2 the MySQL lock

MySQL locks can be classified in many ways:

  • Press add lock mechanism cent: can divide for optimistic lock, pessimistic lock
  • According to compatibility, it can be divided into X lock and S lock
  • According to the lock granularity: table lock, row lock, page lock
  • According to the lock mode: it can be divided into record lock, gap lock, next-key lock, intention lock, insert intention lock

Here we mainly discuss S lock, X lock, optimistic lock and pessimistic lock.

2.1 S Lock and X Lock

S locking and X locking are two standard row locking mechanisms implemented by InnoDB engine. View the default engine available

show variables like '%storage_engine%';
Copy the code

The author’s MySQL version is 8.0.17, and the result is as follows:

Set up test table:

create table a
(
id int primary key auto_increment,
money int
);
Copy the code

2.1.1 S lock

S lock is also called shared lock. Read lock. Data can only be read but cannot be modified.

Lock S on table:

lock table a read;
Copy the code

Then you can see that there is a read lock on the change:

You can only read but not write.

2.1.2 X lock

X lock is also called exclusive lock. After a transaction locks a table, other transactions cannot add, delete, or change the table.

Set manual commit, open transaction, lock X:

set autocmmmit=0;
start transaction;
lock table a write;
Copy the code

To start another transaction, use select:

set autocommit=0;
start transaction;
select * from a;
Copy the code

You can see that with the X lock, another transaction will block even reading. This is because the transaction that opened the X lock blocks the select operation of the other transaction.

You can use

unlock table;
Copy the code

Release the lock so that another transaction can see when it is locked:

2.2 Optimistic and pessimistic locks

2.2.1 optimistic locking

Optimistic locking is a kind of loose locking mechanism that always assumes the best case and does not lock the data every time the operation is performed, but determines whether there are other operations to update the data during the update.

MySQL itself does not provide support for optimistic locking, so you need to implement it yourself. Common methods include version control and timestamp control.

  • Versioning: Versioning is adding a tableversionField with which data is readversionAdd 1 to the version number and compare the version number of the submitted data with the version number in the database. If the version number of the submitted data is greater than the version number in the database, the data will be updated
  • Timestamp control: Timestamp control is similar to version controlversionField is changed totimestampfield

For example, suppose at this point:

  • version=1Transaction A performs the operation
  • After transaction A updates the dataversion=2At the same time, transaction B also operates and updates the dataversion=2
  • Transaction A completes the operation first and is the first to put the databaseversionSet to 2
  • After transaction A completes, transaction B commits its ownversionAnd found in the databaseversionAs such, transaction B’s commit is not accepted

There is also a realization method called CAS algorithm, interested in their own search.

2.2.2 pessimistic locks

Pessimistic locking is always assuming the worst-case scenario that the data is locked for the entire data-processing state. Pessimistic locking implementations often rely on the locking mechanism of the database, which locks data every time before it is retrieved.

MySQL is pessimistic about locking when calling statements such as:

select * from a where xxx for update;
Copy the code

Turn off two transactions for auto-commit:

set autocommit=0;
start transaction;
Copy the code

The tests are as follows:

The select statement will block until the locked transaction commits (unlocks the pessimistic lock).

3 dirty read, phantom read, unrepeatable read with two types of lost updates

3.1 dirty read

Dirty read refers to that a transaction reads uncommitted data from another transaction, causing data inconsistency before and after the select.

For example, if transaction A modifies some data but does not commit it, then transaction B reads the data modified by transaction A but does not commit it, then transaction B forms A dirty read. Normal transaction A is followed by A rollback, and transaction B reads A temporary value.

The schematic diagram is as follows:

3.2 phantom read

For example, transaction A reads A table with A total of 10 rows, then transaction B inserts several rows and commits them, then transaction A reads again and finds that several rows have been added, and the number of rows before and after the select changes, just like hallucination.

The schematic diagram is as follows:

3.3 Cannot be read repeatedly

Non-repeatable read refers to a transaction reading the data committed by another transaction, causing data inconsistency before and after the select.

For example, if transaction A modifies some data and commits it, then transaction B reads it, then transaction B becomes unrepeatable read.

The schematic diagram is as follows:

3.4 Type 1 missing updates

The first type of lost update is when two transactions update the same data at the same time. After one transaction completes the update and commits, the other transaction rolls back, causing the committed update to be lost.

The schematic diagram is as follows:

3.5 Type 2 missing updates

The second type of lost update is that two transactions update one data at the same time, and the data submitted by the first updated transaction will be overwritten by the data submitted by the later updated transaction, that is, the data submitted by the first updated transaction is lost.

The schematic diagram is as follows:

4 Lockdown protocol and isolation level

The locking protocol is a set of rules made when using X lock or S lock, such as the duration of the lock, lock time and so on. Different lockdown protocols correspond to different isolation levels. There are four isolation levels of a transaction, which are Read uncommitted, Read committed, Repeatable Read, and Serializable, respectively corresponding to the corresponding blocking protocol levels.

4.1 Lockdown protocols

Level 1 blocking protocols correspond to the Read Uncommitted isolation level, Read Uncommitted, which is the lowest level at which a transaction can Read data that another transaction has not committed. A level 1 lockdown protocol is essentially an X lock placed before a transaction modiates data and is not released until the end of the transaction, which includes normal completion (transaction commit) and abnormal completion (transaction rollback).

The tier 1 lockdown protocol does not cause update loss, but can cause dirty reads, phantom reads, and unrepeatable reads.

To test this, turn off automatic commit and turn on transactions with the isolation level set:

set autocommit=0;
set session transaction isolation level read uncommitted;
start transaction;
Copy the code

4.1.1 Causing dirty Reads

One transaction can read data that another transaction has not committed. Although Read uncommited has an X lock attached to it, the select of other transactions does not require the lock, so uncommitted data can be Read directly, resulting in dirty reads.

Test steps:

  • Transaction A and transaction B are set separatelyread uncommitedIsolation level to turn off automatic commit and start transactions
  • Transaction BselectThe data,money=100
  • Transaction A modificationmoney=300
  • Transaction B againselect.money=300

Test screenshot:

4.1.2 Trigger hallucinations

Test steps:

  • Transaction A and transaction B are set separatelyread uncommitedIsolation level to turn off automatic commit and start transactions
  • Transaction Bselect count(*)Count rows, only one record
  • Transaction A inserts A record and commits
  • Transaction B againselect count(*)Counting the number of rows becomes two records

Test screenshot:

4.1.3 Triggering unrepeatable reads

Test steps:

  • Transaction A and transaction B are set separatelyread uncommitedIsolation level to turn off automatic commit and start transactions
  • Transaction BselectThe data,money=300
  • Transaction A modificationmoney=100And submit
  • Transaction B againselectThe data,money=100

Test screenshot:

4.2 Secondary lockdown protocol

In essence, the second-level lockdown protocol is based on the first-level protocol (X lock is added when data is modified). S lock is added when data is read, and S lock is released immediately after reading. This prevents dirty reads, but non-repeatable reads and phantom reads may occur. The level-2 lockdown protocol corresponds to the Read COMMITTED and Repeatable Read isolation levels.

The following tests also turn off automatic commit and enable transactions after setting the isolation level:

set autocommit=0;
set session transaction isolation level read committed;
#set session transaction isolation level repeatable read;
start transaction;
Copy the code

2 Read committed

Read commits, which avoid dirty reads but can be illusory and unrepeatable reads.

4.2.1.1 Avoiding Dirty Reads

Test steps:

  • Transaction A and transaction B are set separatelyread commitedIsolation level to turn off automatic commit and start transactions
  • Transaction BselectThe data,money=100
  • Transaction A modificationmoney=300
  • Transaction B againselect, the data remains unchanged,money=100

Test screenshot:

Note that dirty reads are actually not allowed at the Read COMMITTED isolation level, but MySQL does not block queries, instead returning a backup of the data before it was modified. This mechanism is called MVCC (Multi-version Concurrency Control).

4.2.1.2 Raising illusions

Test steps:

  • Transaction A and transaction B are set separatelyread commitedIsolation level to turn off automatic commit and start transactions
  • Transaction Bselect count(*)Count rows, only one record
  • Transaction A inserts A record and commits
  • Transaction B againselect count(*)Counting the number of rows becomes two records

Test screenshot:

4.2.2.3 Causing Unrepeatable Reads

Test steps:

  • Transaction A and transaction B are set separatelyread commitedIsolation level to turn off automatic commit and start transactions
  • Transaction BselectThe data,money=100
  • Transaction A modificationmoney=100And submit
  • Transaction B againselectThe data,money=300

Test screenshot:

Holdings Repeatable read

Repeatable reads are a bit stricter than read commits and are the default level of MySQL. The read process is more affected by MVCC to prevent unrepeatable and dirty reads, but phantom reads are still possible.

4.2.3.1 Avoiding Dirty Reads

Test steps:

  • Transaction A and transaction B are set separatelyrepeatable readIsolation level to turn off automatic commit and start transactions
  • Transaction BselectThe data,money=300
  • Transaction A modificationmoney=100
  • Transaction B againselect, the data remains unchanged,money=300

Test screenshot:

4.2.3.2 Avoiding Unrepeatable Reads

Test steps:

  • Transaction A and transaction B are set separatelyrepeatable readIsolation level to turn off automatic commit and start transactions
  • Transaction BselectThe data,money=300
  • Transaction A modificationmoney=100And submit
  • Transaction B againselect, the data remains unchanged,money=300

Test screenshot:

4.2.3.3 Raising illusions

Test steps:

  • Transaction A and transaction B are set separatelyrepeatable readIsolation level to turn off automatic commit and start transactions
  • Transaction Bselect count(*)Count rows, only one record
  • Transaction A inserts A record and commits
  • Transaction B againselect count(*)Count rows, or only one record
  • Transaction B updates the table,update set money=money+20
  • Before transaction B updates,selectThere’s only one row, soupdateShould be an update line, that is, should prompt1 rows affected
  • After transaction B updates, the prompt is2 rows affected
  • Transaction B againselectTwo records were found

Test screenshot:

So why doesn’t transaction B see two rows immediately after transaction A commits? Isn’t that avoiding hallucinations?

Don’t worry. Look at the next section.

4.2.3.4 Phantom Read Problems in MySQL at repeatable Read Level

Repeatable read is the default level of MySQL, different database implementations have different methods of repeatable read, no matter what implementation method, can avoid the problem of unrepeatable read, but some implementation methods can also avoid phantom read and lost update problems.

MySQL: select from repeatable read levels does not have phantom reads.

The answer is: avoided, but not completely avoided.

MySQL uses snapshot read to implement repeatable read level. Before each transaction, snapshot is taken for the previous data. Data read by each transaction:

  • Or the contents of the snapshot, which is before the transaction begins
  • Or the transaction itself is updated

In other words, if the transaction is not updated, the contents of the snapshot are read, so in the above test results, A single piece of data is displayed in transaction B after transaction A is inserted.

But why does the “hidden” line come out after the transaction Bupdate?

SQL > select (); SQL > select (); SQL > select (); I have two numbers.

In PostgreSQL, however, this does not happen because all PostgreSQL statements are read from snapshots, so there is no phantom read in MySQL.

4.3 Tertiary lockdown agreement

The three-level locking protocol, based on the first-level locking protocol (X lock is added for modification), adds an S lock to read data (similar to the two-level locking), but does not release the S lock until the end of the transaction to avoid phantom, dirty and unrepeatable reads. The isolation level corresponding to the level 3 lockdown protocol is Serializable.

4.3.1 Avoid dirty reads

Test steps:

  • Transaction A and transaction B set the isolation level, turn off automatic commit and start transaction respectively
  • Transaction BselectThe data,money=100
  • Transaction A modificationmoney=300Transaction A is blocked
  • Transaction B commits at the same time transaction A commitsupdatesuccessful

Test screenshot:

Because updates block at the serialization level, dirty reads are avoided.

4.3.2 Avoid illusory and unrepeatable reads

As you can see from the dirty read example, both insert and update operations block directly at the serialization level, thus avoiding phantom and unrepeatable reads.

5 Reference Links

  • CSDN – What is a transaction? And transaction four features
  • CSDN — View and modify the transaction isolation level in mysql8.0.16
  • CSDN — Two lock and three lock protocols in mysql
  • CSDN — [database] blockade management and blockade protocol
  • CSDN — Interviewer: Dirty reading, non-repeatable reading, how does phantom reading happen?
  • Blog Park – What is business? The four characteristics of a transaction and the isolation level of the transaction
  • Blog garden — Mysql lock mechanism — Optimistic lock & Pessimistic lock
  • Isolation level of transactions (in detail)
  • SegmentFault — Optimistic lock, Pessimistic lock, this article is enough!
  • SegmentFault — Analysis of MySQL secondary lock
  • SegmentFault — Details, examples, and solutions for mysql phantom reading
  • Chester Chu — Three common misconceptions about MySQL Repeatable Read Isolation
  • Chester Chu — Review the Isolation Level of the database with illustrations of five common Race Conditions
  • MySQL lock
  • Nuggets of optimism and pessimism are essential for job interviews
  • Tencent Cloud — understand MySql transaction isolation mechanism, locks and various lock protocols
  • Open Source China – MySql- two phase locking protocol