Recently, transaction-related problems are frequently encountered in the reconfiguration of old code, such as transaction scope and effectiveness, transaction synchronization, transaction coordination with other middleware, and transaction granularity control. This paper systematically combs the knowledge content related to transaction.

1. Concepts related to transactions

The concept of transactions comes from two separate requirements: concurrent database access and system error recovery. In database operations, a transaction can be viewed as a collection of SQL statements in a single unit of operations.

1.1 Characteristics of Transactions (ACID)

  • A (Atomacity) : A transaction must be an atomic unit of work; All or none of its data modifications are performed.

  • C (consistency consistency) : a transaction changes a database from one consistent state to the next. When a transaction completes, all data must be in a consistent state.

  • I (isolation isolation) : Changes made by concurrent transactions must be isolated from changes made by any other concurrent transactions. The state in which data is viewed by a transaction, either before it was modified by another concurrent transaction or after it was modified by another transaction, and the data in the intermediate state is not viewed by a transaction.

  • D (Durability) : After a transaction completes, its impact on the system is permanent. This modification will persist even in the event of a fatal system failure.

1.2 Transaction-related behaviors

The actions of a transaction include opening a transaction, committing a transaction, and rolling back a transaction. All user SQL execution in MySQL’s InnoDB storage engine is under transaction control.

  • By default, autoCOMMIT is set to true, MySQL automatically commits a transaction after a single SQL execution is successful, or performs a transaction commit or rollback depending on the type of exception if the SQL execution fails.
  • START TRANSACTION or BEGIN can be used to START transactions, COMMIT and ROLLBACK can be used to COMMIT and ROLLBACK transactions.
  • When autoCOMMIT is set to false, subsequent SQL statements will be executed in one transaction and will not be committed or rolled back until a COMMIT or ROLLBACK transaction is performed.

2.MySQL implements transaction principle

In business, MySQL middleware is often used as a relational database. To achieve the effectiveness of transactions, it is necessary to solve the problems of reliability and concurrency of MySQL. Reliability is to ensure the consistency of data operation before and after insert or update operation or database crash. Concurrency To avoid reading dirty data when multiple transactions are executed concurrently, read and write between transactions must be isolated. MySQL uses log files (redo log and undo log), locking, and MVCC to implement transactions.

2.1 Log Files

MySQL, a common database in business, relies on redo and undo to implement transaction redo and recovery.

  • Redo log: Records changes made to successfully committed transactions. To improve performance, every change is not synchronized to disk in real time. Instead, it is stored in the Boffer Pool, and the background thread does the synchronization between the buffer Pool and disk. Therefore, redo logs are introduced to record the changes of successfully committed transactions, and redo logs are persisted to disk. For example, after the system restarts, the redo log is read to restore the latest data.
  • Undo log: Records the information before data modification. In order to roll back the previous operations when an error occurs, the previous operations need to be recorded and can be rolled back only when an error occurs. The undo log does not need to be persisted. After the transaction is committed, the undo log of the transaction will be deleted.

SQL > update user’s account information; SQL > update user’s account information; SQL > update user’s account information; A record of the results after execution. When the transaction commits, the redo log is synchronized to disk and the undo log is deleted.

2.2 the lock and MVCC

No action can be taken when there are multiple requests to read data from a table, but there must be a measure of concurrency control when there are read requests and modify requests. Read/write locks solve the above problem simply by using a combination of the following two locks to control read/write requests.

  • A shared lock is also called a “read lock”. Read locks can be shared, or read data can be shared by multiple read requests without blocking.
  • An exclusive lock, also known as a write lock, blocks all other lock requests until the lock is released.

InnoDB storage engine MVCC is implemented by storing two hidden columns at the end of each row. These two columns, one stores the creation time of the row, the other stores the expiration time of the row, of course, the storage is not the actual time value, but the system version number, the main idea is to achieve read and write separation through multiple versions of data. So as to achieve read without lock and then do read and write parallel. The implementation of MVCC in mysql relies on undo log and Read View

  • Undo log: The undo log records multiple versions of a row of data.
  • Read View: Used to determine the visibility of the current version of data

In summary, MySQL transactions are atomized by undo log, persistent by redo log, and isolated by read/write lock +MVCC.

3. Transaction implementation principle of Spring framework

3.1 Spring-AOP enhancement principles

Spring uses AOP (aspect oriented programming) to implement declarative transactions, which will be explained in more detail later in the implementation of Spring transactions. The concept of AOP can be understood by referring to Spring AOP concepts, which I won’t go into detail here. Talk about dynamic proxies and AOP enhancements.

 

Dynamic proxies are Spring’s default way of implementing AOP, and there are two types: JDK dynamic proxies and CGLIB dynamic proxies. JDK dynamic proxy is interface oriented and generates anonymous implementation classes of the target proxy interface through reflection. The CGLIB dynamic proxy generates proxy subclasses for the target proxy class using bytecode enhancement through inheritance. Spring defaults to using JDK dynamic proxies for interface implementations and CGLIB for concrete classes, as well as configuring CGLIB globally to generate proxy objects. Detailed practices can be found at juejin.cn/post/684790…

We use the @aspect annotation in Aspect configuration, which uses Aspectj’s Aspect expression. Aspectj is an AOP framework implemented in the Java language that uses the static proxy pattern and has full AOP functionality that complements Spring AOP. Spring uses Aspectj’s powerful way of defining section expressions, but it still uses dynamic proxies by default and does not use Aspectj’s compiler and weaver, although it also supports configuring Aspectj static proxies instead of dynamic proxies. Aspectj is more powerful, for example, supporting field and POJO class enhancements, in contrast to Spring’s support for Bean method level enhancements.

Spring enhances methods in five ways:

  • Front enhancement (org. Springframework. Aop. BeforeAdvice) : in the target method enhanced before implementation;
  • Rear enhancement (org. Springframework. Aop. AfterReturningAdvice) : enhanced after the target method execution;
  • Surrounding the enhancement (org. Aopalliance. Intercept. MethodInterceptor) : in the target method perform before and after enhancement;
  • Exception thrown enhancement (org. Springframework. Aop. ThrowsAdvice) : in the target method throws an exception executed after enhancement;
  • Introducing enhancement (org. Springframework. Aop. IntroductionInterceptor) : add new methods and properties for the target class.

Declarative transactions are implemented by means of wrap reinforcement, starting the transaction before the target method executes and committing or rolling back the transaction after the target method executes, as shown in the inheritance diagram of the transaction interceptor:

3.2 Spring transaction facets implementation principle

Spring transactions are implemented in an AOP way, mainly through the TransactionAspectSupport class. You can see the main implementation process of transaction operations in this class as follows:

/ / 1. Get @ the related parameters of Transactional annotation TransactionAttributeSource tas = getTransactionAttributeSource (); Final TransactionAttribute txAttr = (tas! = null ? tas.getTransactionAttribute(method, targetClass) : null); final PlatformTransactionManager tm = determineTransactionManager(txAttr); final String joinpointIdentification = methodIdentification(method, targetClass, txAttr); if (txAttr == null || ! (tm instanceof CallbackPreferringPlatformTransactionManager)) { // 3. Get TransactionInfo, Contains the tm and TransactionStatus TransactionInfo txInfo = createTransactionIfNecessary (tm, txAttr, joinpointIdentification); Object retVal = null; Try {/ / 4. Implement the target method retVal = invocation. ProceedWithInvocation (); } the catch (Throwable ex) {/ / 5. Abnormal, rollback transaction completeTransactionAfterThrowing (txInfo, ex); throw ex; } finally {// 6. CleanupTransactionInfo (txInfo); } / / 7. Commit the transaction commitTransactionAfterReturning (txInfo); return retVal; }Copy the code

TransactionManager stores the current data source connection, provides the transaction commit rollback operation interface to the data source, and implements the method of transaction related operation. The DataSource has the following core methods:

  • Public commit Commits the transaction
  • Public rollback Rollback the transaction
  • Public getTransaction Gets the current transaction status
  • Protected doBegin Starts the transaction, primarily by executing the CON.setautoCommit (false) method of JDBC. Many ThreadLocal variables related to database connections are handled at the same time.
  • Protected doSuspend suspends the transaction
  • Protected doCommit Commits the transaction
  • Protected doRollback Rolls back the transaction
  • Protected doGetTransaction() gets transaction information

3.3 Spring transaction propagation mechanism

Spring transaction level describes the spread of multiple use @ Transactional annotation method calls to each other, the Spring of transaction processing logic, transmission level are:

  • REQUIRED, joins a transaction if the current thread is already in one, creates a new transaction otherwise.
  • SUPPORT, if the current thread is already in a transaction, join the transaction, otherwise no transaction is used.
  • MANDATORY, joining a transaction if the current thread is already in it, otherwise raising an exception.
  • REQUIRES_NEW, which creates a new transaction anyway, suspends the current transaction and creates a new one if the current thread is already in one.
  • NOT_SUPPORTED, suspends the transaction if the current thread is in a transaction.
  • NEVER, an exception is thrown if the current thread is in a transaction.
  • NESTED executes a NESTED transaction, similar to REQUIRED but different from SAVEPOINT in Mysql.

Suspending a transaction means storing the properties of the current transaction, such as transaction name and isolation level, in a variable, and setting all transaction-related ThreadLocal variables in the current thread as if the thread had never been started. Spring maintains the transaction state of a current thread to determine if and what kind of transaction the current thread is in. After the transaction is suspended, the transaction state of the current thread is as if there were no transaction.

4. Pothole avoidance Guide

4.1 Transaction propagation level

The following are some common cases of transaction propagation level in business development, especially when multiple transactions are interwoven. It is a first requirement to be familiar with the specific manifestations of the propagation level. The following tests were performed for different performance at the REQUIRED/REQUIRED_NEW/NESTED propagation levels between transactions.

The REQURED propagation level is tested as follows: if the enclosing method does not open a transaction, the REQUIRED inner method opens its own transaction, and the opened transactions are independent of each other. If the enclosing method opens a transaction, the REQUIRED inner method adds to the enclosing method’s transaction. All internal and peripheral methods decorated with REQUIRED belong to the same transaction, and if one method is rolled back, the entire transaction is rolled back.

@service public class TestServiceAImpl implements TestServiceA {/** * */ public void saveApp1() {serviceb.saveb_required (); serviceC.saveC_required(); throw new RuntimeException("test"); } public void saveApp2() {serviceb.saveb_required ();} public void saveApp2() {serviceb.saveb_required (); serviceC.saveC_required_throw_exception(); } /** * start the transaction outside, B and C are added to the outer transaction, */ @transactional (Propagation = REQUIRED) public void saveApp3() {serviceb.saveb_required (); serviceC.saveC_required(); throw new RuntimeException("test"); } /** * the whole transaction is rolled back when the inner method of C throws an exception. */ @transactional (Propagation = REQUIRED) public void saveApp4() {serviceb.saveb_required (); serviceC.saveC_required_throw_exception(); } /** * the external method starts the transaction, and both B and C join the external transaction. * When the inner method of C throws an exception, it is caught, even if the external method does not sense it. */ @transactional (Propagation = REQUIRED) public void saveApp5() {serviceb.saveb_required (); try { serviceC.saveC_required_throw_exception(); } catch (Exception ex) { } } }Copy the code

The REQUIRED_NEW propagation level was tested as follows: In the case that the external method did not open a transaction, the internal method modified by REQUIRES_NEW opened its own transaction, and the opened transactions were independent of each other. The internal method modified by REQUIRES_NEW still starts a separate transaction and is independent of the external method transaction even if the external method starts a transaction. Internal methods and internal method and external method transactions are independent of each other and do not interfere with each other.

@service public class TestServiceAImpl implements TestServiceA {/** * Public void saveApp6() {serviceb.saveb_new (); public void saveApp6() {serviceb.saveb_new (); serviceC.saveC_new(); throw new RuntimeException("test"); Public void saveApp7() {serviceb.saveb_new (); public void saveApp7() {serviceb.saveb_new (); serviceC.saveC_new_throw_exception(); } /** * B_required is the same as the B_required, B_required is rolled back, B_new and C_new are the two new transactions not affected. */ @transactional (Propagation = REQUIRED) public void saveApp8() {saveb.saveb_required (); serviceB.saveB_new(); serviceC.saveC_new(); throw new RuntimeException("test"); } /** * B_required is the same as the external transaction, * C_new is the new transaction, and the rollback of B_required is not affected.  */ @Transactional(propagation = REQUIRED) public void saveApp9() { serviceB.saveB_required(); serviceB.saveB_new(); serviceC.saveC_new_throw_exception(); } /** * B_required and C_new are inserted properly. B_required and C_new are inserted properly. */ @transactional (Propagation = REQUIRED) public void saveApp10() {serviceb.saveb_required (); */ @Transactional(Propagation = REQUIRED) public void saveApp10() {serviceb.saveb_required (); serviceC.saveC_new(); try { serviceC.saveC_new_throw_exception(); } catch (Exception ex) { } } }Copy the code

The test for the NESTED propagation level is as follows: When the NESTED method does not start a transaction, both the modified internal method and REQUIRED will start its own transaction, and the enabled transaction is independent of each other. The NESTED inner method belongs to a subtransaction of an external transaction when a peripheral method starts a transaction. The peripheral main transaction is rolled back, and the subtransaction must be rolled back. The inner subtransaction can be rolled back independently without affecting the peripheral main transaction and other subtransactions.

@service public class TestServiceAImpl implements TestServiceA {/** * */ public void saveApp11() {serviceb.saveb_nest (); serviceC.saveC_nest(); throw new RuntimeException("test"); Public void saveApp12() {serviceb.saveb_nest ();} public void saveApp12() {serviceb.saveb_nest (); serviceC.saveC_nest_throw_exception(); } /** * the inner transaction is a subtransaction of the outer transaction, */ @transactional (Propagation = REQUIRED) public void saveApp13() {serviceb.saveb_nest (); serviceC.saveC_nest(); throw new RuntimeException("test"); } /** * If the internal transaction throws an exception, the external method will cause the whole transaction to roll back. */ @transactional (Propagation = REQUIRED) public void saveApp14() {serviceb.saveb_nest (); */ @transactional (Propagation = REQUIRED) public void saveApp14() {serviceb.saveb_nest (); serviceC.saveC_nest_throw_exception(); } /** * if (C = 0) {} /** * if (C = 0) {} /** * if (C = 0) {} */ @transactional (Propagation = REQUIRED) public void saveApp15() {serviceb.saveb_nest (); try { serviceC.saveC_nest_throw_exception(); } catch (Exception ex) { } } }Copy the code

In summary, the related features of the three transaction isolation levels of NESTED, REQUIRED, and REQUIRED_NEW are as follows:

  • NESTED and REQUIRED modified internal methods are peripheral method transactions, and both methods’ transactions are rolled back if the peripheral method throws an exception.
  • REQUIRED joins the enclosing method transaction and therefore belongs to the same transaction as the enclosing method transaction, which will be rolled back once the REQUIRED transaction throws an exception.
  • NESTED is a subtransaction of a peripheral method and has a separate savepoint. Therefore, an exception thrown by a NESTED method is rolled back and does not affect the transactions of the peripheral method.
  • Both NESTED and REQUIRES_NEW can roll back internal method transactions without affecting peripheral method transactions. However, because NESTED transactions are NESTED, after the rollback of the peripheral method, the child transactions that are the peripheral method transactions are also rolled back.
  • REQUIRES_NEW is implemented by starting a new transaction. The internal and peripheral transactions are two transactions, and the rollback of the peripheral transaction does not affect the internal transaction.

4.2 Transaction Validity

Transactional failures are common in business development. Consider databases, AOP technologies, Spring’s @Transactional annotation, and your application’s data source configuration. Here are some lessons:

  • Innodb storage engine only supports transactions in MySQL database, MyISAM storage engine does not support transactions;
  • No rollbackFor parameter is specified. The Throwable thrown by default contains both Exception and Error subtypes.
  • When there are multiple data sources, such as database table/read/write separation, it is common to encounter an application with multiple data sources configured and no transactionManager parameter specified. The default transactionManager may not be the same
  • If AOP uses JDK dynamic proxies, object methods that call each other are not intercepted by Spring’s AOP and the @Transactional annotation is not valid.
  • If AOP uses a CGLIB proxy, the Transactional method or class is not public, cannot be accessed by external packages, or is final and cannot be inherited, the @Transactional annotation is not valid.

4.3 Granularity control of transactions

In business development at ordinary times, often encountered in a big way and add the @ Transactional annotation, from the calling method has been open and takes up a connection resources, in this great way to call a lot of external interface and logic, which consumes a lot of time, wait until this is done to perform the last step of warehousing operations, This greatly affects the performance of the database, such as throughput, which is often shown as insufficient database connections. This can be obtained by analyzing the communication packets between the MySQL database server and the application server. The whole transaction scope is too large, and the transaction scope should be appropriately small according to the situation. In this way, it is more reasonable to use the connection resources, and we have learned the blood lesson of the excessive transaction scope in our team.

@transactional public void big_transaction() {List<User> users = findUser(classUid); Map<String, String> classNameMap = findName(classUid); // Check the user list check(users); . SaveUser (users); }Copy the code

4.4 Transaction synchronization with different middleware

In real business development, we often have important operations that need to be performed after the DB transaction commits, such as sending MQ or clearing the cache. For a practical business scenario: updating user information, two interfaces are associated

  • Query user information: Queries user information from the cache. If the user information cannot be found, load the user information from the DB and put it into the cache
  • Update user information: Update db information and delete cache

For example, within A transaction, the code sequence is A-B-C-D, and we specify that B and D are executed after the transaction commits.

  • If the transaction is successfully committed, the execution sequence is a-C – Transaction commit – B-D
  • Transaction rollback is performed in the following order: A-C – transaction rollback

Our team made transaction enhancements to this situation, updating database records, clearing the cache, and sending messages in the same transaction. Without enhanced components, the entire transaction process would be very long and would be affected by caching connection processing and sending messages processing, affecting database throughput. The commit and rollback of the transaction is controlled after the component is used.

@service @invokeaftercommitted public class UserServiceAfterImpl {/** * public void after() {// Clear the cache clearCache(); // sendMq sendMq(); } } @Service public clsss ScheduleServiceImpl { @Autowired privated UserServiceAfterImpl afterInvokerImpl; @transactional @ovveride public void updateSchedule() {update SQL scheduleManager.udpate(); Afterinvokerimpl.after (); }Copy the code

Part of the code that does this is shown below.

public class CustomTransactionManager extends DataSourceTransactionManager { @Override protected DefaultTransactionStatus newTransactionStatus(TransactionDefinition definition, Object transaction, boolean newTransaction, boolean newSynchronization, boolean debug, Object suspendedResources) { DefaultTransactionStatus transactionStatus = super.newTransactionStatus(definition, transaction, newTransaction, newSynchronization, debug, suspendedResources); return DefaultTransactionStatusProxy.proxyTransaction(transactionStatus, transaction, newTransaction, transactionStatus.isNewSynchronization(), definition.isReadOnly(), debug, suspendedResources); } @Override protected void doBegin(Object transaction, TransactionDefinition definition) { super.doBegin(transaction, definition); InvokerHolder.init(); } @Override protected void doResume(Object transaction, Object suspendedResources) { super.doResume(transaction, suspendedResources); Object messageTxHolderObj = TransactionSynchronizationManager.unbindResource(suspendedResources); if (messageTxHolderObj instanceof InvokerHolder) { InvokerHolder.resume((InvokerHolder) messageTxHolderObj); } else { ... } } @Override protected void doCommit(DefaultTransactionStatus status) { super.doCommit(status); List<Invoker> logicList = InvokerHolder.clear(); for (Invoker logic : logicList) { try { logic.doInvoke(); } catch (Exception e) { ... } } } @Override protected void doRollback(DefaultTransactionStatus status) { super.doRollback(status); List<Invoker> logicList = InvokerHolder.clear(); for (Invoker logic : logicList) { ... }}}Copy the code

5. To summarize

This paper mainly focuses on the use of transactions in daily business development. First of all, it introduces the basic knowledge of transactions, and introduces some technical principles of MySQL and Spring to implement transactions, and summarizes some problems found in our team’s daily development and the corresponding practice, hoping to bring you some harvest!

reference

Cloud.tencent.com/developer/a… Mysql transaction implementation principle

www.cnblogs.com/kismetv/p/1…

Blog.csdn.net/weixin\_443…

Segmentfault.com/a/119000001… Spring transaction propagation behavior in detail

Blog.csdn.net/qq\_3852657… Spring propagation behavior

—————————————————————————————–

If you think the article is well written, give it a thumbs up!

Please like 👍 please follow ❤️ please share 👥