preface

Recently, a net friend asked me a question: how to deal with the big transaction problem in the system?

Just some time ago, I dealt with this problem in our company. Due to the limited time at the initial stage of the project, we ignored some performance problems of the system in order to quickly complete business functions. After the project was successfully launched, one iteration was devoted to solving big transaction problems, which has been optimized and successfully launched. Give you a summary of some of the solutions we used at that time, so that you can refer to the same problem when troubled.

Problems caused by big things

Before sharing solutions, take a look at the problems that can arise if a large transaction occurs in the system

As can be seen from the figure above, if there are large transactions in the system, the problem is not small, so we should try to avoid the situation of large transactions in the actual project development. If we have a large transaction problem in our existing system, how do we solve it?

The solution

Don’t use the @Transactional annotation

In real project development, it is common for business methods to use the @Transactional annotation to turn on Transactional functionality. This is called declarative transactions. Part of the code is as follows:

@Transactional(rollbackFor=Exception.class)
   public void save(User user) {
         doSameThing...
   }
Copy the code

However, the first thing I would say is: don’t use the @Transactional annotation. Why is that?

We know that the @Transactional annotation works through Spring’s AOP, but the Transactional functionality can fail if used incorrectly. If you happen to be inexperienced, this kind of problem is not easy to check. As for when a transaction will fail, you can refer to my previous post on the 10 Pitfalls of Spring Transactions that you may not be aware of!! This article. The @Transactional annotation, usually applied to a business method, causes the entire business method to be in the same transaction. This coarse-grained annotation makes it difficult to control transaction scope and is the most common cause of large transaction problems.

So what are we going to do?

You can use programmatic transactions to perform transactions manually in spring projects using objects of the TransactionTemplate class. Part of the code is as follows:

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { transactionTemplate.execute((status) => { doSameThing... return Boolean.TRUE; })}Copy the code

As you can see from the code above, using the programmatic transaction capability of TransactionTemplate to control the scope of the transaction itself is the preferred way to avoid large transaction problems.

When I say you shouldn’t use the Transactional annotation to start transactions, I don’t mean you shouldn’t use it. If you have simple business logic in your project that doesn’t change often, it’s okay to use the Transactional annotation to start transactions because it’s simpler and more efficient. But beware of transaction failures.

Put the query (SELECT) method outside the transaction

If a large transaction occurs, it is also common to place the query (SELECT) method outside of the transaction, since such methods do not normally require a transaction.

For example, the following code appears:

@Transactional(rollbackFor=Exception.class)
   public void save(User user) {
         queryData1();
         queryData2();
         addData1();
         updateData2();
   }
Copy the code

The queryData1 and queryData2 query methods can be placed outside of the transaction and the code that really needs to be executed in the transaction, such as the addData1 and updateData2 methods, can be effectively reduced transaction granularity. This is very easy to change if you use a programmatic transaction with TransactionTemplate.

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { queryData1(); queryData2(); transactionTemplate.execute((status) => { addData1(); updateData2(); return Boolean.TRUE; })}Copy the code

But if you really want to use the @Transactional annotation, how do you split it?

public void save(User user) {
         queryData1();
         queryData2();
         doSave();
    }
   
    @Transactional(rollbackFor=Exception.class)
    public void doSave(User user) {
       addData1();
       updateData2();
    }
Copy the code

This example is a very classic error, this direct method call approach transaction will not work, warning friends in the pit. Because declarative transactions with the @Transactional annotation work through Spring AOP, which generates proxy objects and uses the original object for direct method calls, transactions do not take effect.

Is there a way around this?

1. Add a new Service method

This method is as simple as adding a new Service method, adding the @Transactional annotation to the new Service method, and moving the Transactional code to the new method. The specific code is as follows:

@Servcie publicclass ServiceA { @Autowired prvate ServiceB serviceB; public void save(User user) { queryData1(); queryData2(); serviceB.doSave(user); } } @Servcie publicclass ServiceB { @Transactional(rollbackFor=Exception.class) public void doSave(User user) { addData1(); updateData2(); }}Copy the code

2. Inject itself into the Service class

If you don’t want to add a new Service class, injecting yourself into the Service class is an option. The specific code is as follows:

@Servcie publicclass ServiceA { @Autowired prvate ServiceA serviceA; public void save(User user) { queryData1(); queryData2(); serviceA.doSave(user); } @Transactional(rollbackFor=Exception.class) public void doSave(User user) { addData1(); updateData2(); }}Copy the code

Some people might wonder: Does this approach create a cyclic dependency problem?

In fact, spring IOC’s internal level 3 cache ensures that it does not have cyclic dependencies. If you want to learn more about loop dependencies, check out my previous article “Why Does Spring Solve loop dependencies with Tertiary Caching?” .

3. Use aopContext.currentProxy () in the Service class to obtain the proxy object

Method 2 above does solve the problem, but the code doesn’t look intuitive. You can do the same thing by using AOPProxy in the Service class to get the proxy object. The specific code is as follows:

@Servcie publicclass ServiceA { public void save(User user) { queryData1(); queryData2(); ((ServiceA)AopContext.currentProxy()).doSave(user); } @Transactional(rollbackFor=Exception.class) public void doSave(User user) { addData1(); updateData2(); }}Copy the code

Avoid remote calls in transactions

It is inevitable that we call the interfaces of other systems in the interface. Due to the network instability, the response time of such remote calls may be long. If the code of remote calls is placed in something, it may be a large transaction. Of course, remote calls are not only about calling interfaces, but also about sending MQ messages, or connecting to Redis and mongodb to save data.

@Transactional(rollbackFor=Exception.class)
   public void save(User user) {
         callRemoteApi();
         addData1();
   }
Copy the code

The code for remote calls can be time-consuming, so keep it out of transactions.

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { callRemoteApi(); transactionTemplate.execute((status) => { addData1(); return Boolean.TRUE; })}Copy the code

Some friends may ask, how do you ensure data consistency when the code that is called remotely is not in a transaction? This needs to establish: retry + compensation mechanism, to achieve the final consistency of data.

Avoid processing too much data at once in a transaction

Large transaction problems can also arise if there is too much data to process in a transaction. For example, for ease of operation, you may need to batch update 1000 data at a time, which can lead to a lot of data lock waiting, especially in high-concurrency systems.

The solution is paging, 1000 pieces of data, divided into 50 pages, and processing only 20 pieces of data at a time, which can greatly reduce the occurrence of large transactions.

Nontransactional execution

Before using transactions, we should all think about whether all database operations need to be performed in a transaction.

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { transactionTemplate.execute((status) => { addData(); addLog(); updateCount(); return Boolean.TRUE; })}Copy the code

In the example above, the addLog method to add operation logs and the updateCount method to update the count can not be executed in a transaction because the operation log and count services allow for a small number of data inconsistencies.

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { transactionTemplate.execute((status) => { addData(); return Boolean.TRUE; }) addLog(); updateCount(); }Copy the code

Of course, it is not so easy to identify which methods can be executed non-transactionally in large transactions, and you need to comb through the entire business to find the most reasonable answer.

Asynchronous processing

Also important, do all methods in a transaction need to be executed synchronously? As we all know, method synchronous execution needs to wait for method return. If too many methods are executed synchronously in a transaction, it will inevitably lead to long wait times and large transaction problems.

Consider the following example:

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { transactionTemplate.execute((status) => { order(); delivery(); return Boolean.TRUE; })}Copy the code

The order method is used to place the order, and the delivery method is used to deliver the goods.

The answer is no.

The shipping function can actually follow mq asynchronous processing logic.

@Autowired private TransactionTemplate transactionTemplate; . public void save(final User user) { transactionTemplate.execute((status) => { order(); return Boolean.TRUE; }) sendMq(); }Copy the code

conclusion

Starting from a question from a netizen, I share 6 ways to deal with big affairs in combination with my actual work experience:

  • Don’t use the @Transactional annotation
  • Put the query (SELECT) method outside the transaction
  • Avoid remote calls in transactions
  • Avoid processing too much data at once in a transaction
  • Nontransactional execution
  • Asynchronous processing

One last word (attention, don’t fuck me for nothing)

If this article is helpful to you, or inspired, help scan the QR code to pay attention to it, or like, forward, look at. In the public number reply: interview, code artifact, development manual, time management have super good fan welfare, in addition reply: add group, can communicate with many big factory elder and study.