background

This article mainly shares some problems found during pressure measurement (high concurrency). The previous two articles have covered some summary and optimization of message queues and database connection pools in the case of high concurrency. If you are interested, please browse my public account. Without further ado, let’s get to the point.

Transaction, presumably you the king of the CRUD is no stranger to it, there are basically you need to use multiple write requests, and Spring for the transaction and the use of special simple, you just need to a @ Transactional annotation, is shown in the following example:

    @Transactional
    public int createOrder(Order order){
        orderDbStorage.save(order);
        orderItemDbStorage.save(order.getItems());
        return order.getId();
    }
Copy the code

When we create an order, we usually need to place the order and the order item in the same transaction to make sure it meets ACID, so we just need to write the transaction annotation on the way we create the order.

Fair use of transactions

For the order creation code above, what if you now need to add a requirement to send a message to a message queue or call an RPC after the order is created? The first thing many students will think of is to call directly from a transaction method:

    @Transactional
    public int createOrder(Order order){
        orderDbStorage.save(order);
        orderItemDbStorage.save(order.getItems());
        sendRpc();
        sendMessage();
        return order.getId();
    }
Copy the code

This kind of code will appear in many people write business, transaction nested RPC, nested some non-DB operations, generally there is no problem with this writing, once the non-DB write operation appears relatively slow, or traffic is relatively large, there will be a problem of large transactions. The database connection is held up because the transaction is never committed. At this time you may ask, I can not expand the database connection point on the line, 100 not on 1000, in the last article has said that the database connection pool size will still affect the performance of our database, so, the database connection is not to expand as much as you want to expand.

So how do we optimize it? Here we can think about our non-DB operation, in fact, does not satisfy the ACID of our transaction, so why write it in the transaction, so here we can extract it.

    public int createOrder(Order order){
        createOrderService.createOrder(order);
        sendRpc();
        sendMessage();
    }
Copy the code

In this method, the transaction creation order is called first, and then other non-DB operations are called. If we now want more complex logic, such as creating an order to send a successful RPC request, failure to send a failed RPC request, from the above code we can do the following transformation:

public int createOrder(Order order){ try { createOrderService.createOrder(order); sendSuccessedRpc(); }catch (Exception e){ sendFailedRpc(); throw e; }}Copy the code

Usually we catch the exception, or do some special processing based on the return value. This implementation needs to catch the exception explicitly, and then throw it again. This method is not very elegant.

TransactionSynchronizationManager

There are just a few tools available in Spring transactions to help us with this requirement. In TransactionSynchronizationManager provides let’s transaction register callBack methods:

public static void registerSynchronization(TransactionSynchronization synchronization)
			throws IllegalStateException {

		Assert.notNull(synchronization, "TransactionSynchronization must not be null");
		if(! isSynchronizationActive()) { throw new IllegalStateException("Transaction synchronization is not active");
		}
		synchronizations.get().add(synchronization);
	}
Copy the code

TransactionSynchronization our transaction callBack, provides some extension points for us:

public interface TransactionSynchronization extends Flushable { int STATUS_COMMITTED = 0; int STATUS_ROLLED_BACK = 1; int STATUS_UNKNOWN = 2; /** ** suspends */ voidsuspend(a); /** * suspends a transaction and raises an exception */ void resume(); @Override void flush(); /** * trigger before transaction commit */ void beforeCommit(BooleanreadOnly); /** * triggers before the transaction completes */ void beforeCompletion(); Void afterCommit(); / / void afterCompletion(int status); }Copy the code

We can use the afterComplettion method to implement our business logic above:

    @Transactional
    public int createOrder(Order order){
        orderDbStorage.save(order);
        orderItemDbStorage.save(order.getItems());
        TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
            @Override
            public void afterCompletion(int status) {
                if (status == STATUS_COMMITTED){
                    sendSuccessedRpc();
                }else{ sendFailedRpc(); }}});return order.getId();
    }
Copy the code

Here we implement afterCompletion directly, judging by the transaction’s status which RPC we should send. . Of course, we can further encapsulated TransactionSynchronizationManager registerSynchronization Util to encapsulate it into a transaction, we can make the code more concise.

This way we don’t have to write all the non-DB operations out of the method, so the code is more logically coherent, readable, and elegant.

The pit of afterCompletion

The registration transaction callback code in our in our business logic often appear, such as a transaction after flushing the cache, sending a message queue, send a notification message, etc., in the course of everyday use, we use this basic didn’t also what’s the problem, but in the process of suppression, found a piece of a bottleneck, take special for a long time, Through a series of monitoring, it was found that the waiting time for obtaining a connection from the database connection pool was long. Finally, we located the afterCompeltion action, which did not return the database connection.

In the Spring AbstractPlatformTransactionManager, to commit processing code is as follows:

private void processCommit(DefaultTransactionStatus status) throws TransactionException {
		try {
			boolean beforeCompletionInvoked = false;
			try {
				prepareForCommit(status);
				triggerBeforeCommit(status);
				triggerBeforeCompletion(status);
				beforeCompletionInvoked = true;
				boolean globalRollbackOnly = false;
				if (status.isNewTransaction() || isFailEarlyOnGlobalRollbackOnly()) {
					globalRollbackOnly = status.isGlobalRollbackOnly();
				}
				if (status.hasSavepoint()) {
					if (status.isDebug()) {
						logger.debug("Releasing transaction savepoint");
					}
					status.releaseHeldSavepoint();
				}
				else if (status.isNewTransaction()) {
					if (status.isDebug()) {
						logger.debug("Initiating transaction commit");
					}
					doCommit(status);
				}
				// Throw UnexpectedRollbackException if we have a global rollback-only
				// marker but still didn't get a corresponding exception from commit. if (globalRollbackOnly) { throw new UnexpectedRollbackException( "Transaction silently rolled back because it has been marked as rollback-only"); } } // Trigger afterCommit callbacks, with an exception thrown there // propagated to callers but the transaction still considered as committed. try { triggerAfterCommit(status); } finally { triggerAfterCompletion(status, TransactionSynchronization.STATUS_COMMITTED); } } finally { cleanupAfterCompletion(status); }}Copy the code

Here we only need to focus on the penultimate lines of code, and we can see that our triggerAfterCompletion, which is the penultimate execution logic, will execute our cleanupAfterCompletion when all the code has been executed. Our return database connection is also in this code, which causes us to get the database connection slowly.

How to optimize

How to optimize the above problem? Here are three options to optimize:

• The method of removing non-DB operations from the transaction, which is the original method above, can be extracted for some simple logic, but for some complex logic, such as the nesting of transactions, where afterCompletion is called, this will add a lot of work and cause problems.
• To increase the speed of database connection pool return, do it asynchronously through multiple threads. This is suitable for registering afterCompletion at the end of the transaction and directly transferring the work to other threads. However, if registration of afterCompletion occurs between our transactions, such as nested transactions, it will result in subsequent business logic and transaction parallelism.
• Emulate Spring transaction callback registration to implement new annotations. Both approaches have their drawbacks, so in the end we adopted this approach and implemented a custom annotation@MethodCallBackAnnotate all transactions that use this annotation, and then use similar registration code.

    @Transactional
    @MethodCallBack
    public int createOrder(Order order){
        orderDbStorage.save(order);
        orderItemDbStorage.save(order.getItems());
        MethodCallbackHelper.registerOnSuccess(() -> sendSuccessedRpc());
         MethodCallbackHelper.registerOnThrowable(throwable -> sendFailedRpc());
        return order.getId();
    }
Copy the code

With the third method, we basically just need to replace all the places where we register transaction callbacks to work.

Let’s talk about the big things

After all this talk about big things, what exactly is big things? A simple one is a transaction that runs for a long time, which is a large transaction. Generally speaking, transaction times are long due to the following factors:

• If there is a lot of data manipulation, such as inserting a lot of data into a transaction, the transaction will take a long time to execute.
• Lock contention is high, and when all connections operate on the same data at the same time, there will be queuing and transaction times will naturally become longer.
• In a transaction other than the DB operation, such as some RPC requests, some people say that my RPC soon, do not increase the running time of the transaction, but the RPC requests itself is an unstable factors, influenced by many factors, network fluctuation, the downstream service response is slow, once appear, if these factors, there will be a lot of affairs for a long time, Mysql may hang, causing an avalanche.

The above three conditions, the front two possible is not very common, but the third transaction has a lot of the DB operation, this is we are very common, usually appear some of the reasons for this situation is our own habits when specification, a beginner or some of the less experienced people write code, tend to write a big way, in this method, combined with transaction annotations directly, And then add to that, whatever the logic, a shuttle, like this:

Of course, there are some people who want to do distributed transactions, but do it the wrong way. For distributed transactions, you can pay attention to Seata, and you can also use an annotation to help you do distributed transactions.

The last

And in the end, why does this happen? The common understanding is that the database connection must have been released after it was done, but this is not the case. As a result, there are a lot of apis that we can’t read from the book, and if they don’t have a doc, you should dig deeper into the implementation details.

Of course, the final hope is that before you write code, try not to shuttle, take every line of code seriously.

If you find this article helpful to you, your attention and forwarding will be my biggest support.