The background,

At present, due to historical reasons, many applications in the company have access to multiple databases for insertion and update operations, which may cause data consistency problems. At the same time, applications may also have transaction problems if they are called across services.

Currently, dynamic-datasource-spring-boot-starter is used for multi-data source control. Seata is an open source distributed transaction framework. We learned that the new version of dynamic-datasource-spring-boot-starter already supports distributed transactions based on SEATA, while the examples on the website are mostly standard single-source integrations. In the following, we respectively use the functions of dynamic-datasource- Spring-boot-starter, SEATA and their integration.

Second, the dynamic – the datasource

Dynamic-datasource -spring-boot-starter is a fast integrated multi-data source initiator based on Springboot. The features are listed below. For more information, see Github’s website. We mainly use its support for SEATA data sources and its feature of dynamically switching data sources.

Official website: github.com/baomidou/dy…

  1. Support data source grouping, suitable for a variety of scenarios pure multi-library read and write separation of a master multi-slave mixed mode.

  2. Support database sensitive configuration information encryption ENC().

  3. Support each database to independently initialize the table structure schema and database.

  4. Supports no data source startup and lazy loading of data sources (creating connections when needed).

  5. Support custom annotations, inherit DS(3.2.0+).

  6. Provides and simplifies fast integration with Druid, HikariCp, BeeCp, and Dbcp2.

  7. Provide integration solutions for Mybatis-Plus, Quartz, ShardingJdbc, P6sy, Jndi and other components.

  8. Provide a custom data source source solution (such as full load from the database).

  9. Provides a dynamic solution to add and remove data sources after the project starts.

  10. Provide pure read and write separation scheme in Mybatis environment.

  11. Provides a solution for parsing data sources using SPEL dynamic parameters. Built-in SPEL, session, header, support customization.

  12. Supports nested switching of multiple data sources. (ServiceA >>> ServiceB >>> ServiceC).

  13. Provides a distributed transaction solution based on SEATA.

  14. Provides a local multi-data source transaction scheme.

3. Introduction to SEATA

Seata is an open source distributed transaction solution dedicated to providing high performance and easy to use distributed transaction services. Seata focuses on the AT model. The mechanism of AT mode is as follows:

  1. Phase one: Business data and rollback log records are committed in the same local transaction, freeing local locks and connection resources.

  2. Phase two: Commit asynchronously, done very quickly. Rollback is compensated in reverse by the rollback log of one phase.

Fourth, hands-on practice

In the actual use of the above framework process, will encounter a variety of scenarios, encountered a variety of problems. Practice is the real knowledge, let’s simulate simple business practice to test their functions!

4.1 Service Flow Chart

As shown in the figure above, we simulate an order business, the main business process:

  1. Postman uses HTTP to request an order service

  2. The ordering service accesses the order library to generate order records, access the credit branch library to verify and deduct credit points

  3. Visit the inventory and deduct the inventory

  4. This will eventually return true if the order is successfully placed

4.2 Preparing the Environment

  1. Databases: SeatA_storage, SeatA_Order, SeatA_credit, SEATA

  2. Application: Order-service, storage-Service, SEATA TC (Distributed transaction coordinator)

  3. Configuration center: Nacos

  4. Seata server: The SEATA version for this practice is V1.4.2

4.2.1 SeATA installation

See the official website: seata. IO /zh-cn/docs/…

  1. Download the TC package at github.com/seata/seata…

  2. After the TC package is decompressed, modify register.conf to configure the NACOS address and namespace.

  3. Nacos configuration items can be accessed via the script :github.com/seata/seata…

  4. To create a SEATA database, see github.com/seata/seata…

  5. Start the Seata server using seataserver.sh in the decompressed package. You can view the seata cluster in nacOS

Stomp: If nacOS is using a version with permissions, do not use special characters in the password, otherwise it will always be 403 error at startup, because special characters are escaped from seata server Request to Nacos, resulting in an error.

4.2.2 Service Application Configuration Order Application (Two Databases are connected) :

# # # # # # # # # # service registry spring. Cloud. Nacos. Discovery. The server - addr = localhost: 8848 spring. Cloud. Nacos. Discovery. The username = nacos_test Spring. Cloud. Nacos. Discovery. Password = nacos_test # # # # # # # # # # seata related seata. Enabled = true seata.tx-service-group=my_test_tx_group seata.enable-auto-data-source-proxy=false seata.config.type=nacos seata.config.nacos.data-id=seataServer.properties seata.config.nacos.server-addr=localhost:8848 seata.config.nacos.application=seata-server seata.config.nacos.namespace=7322f40d-74de-4679-ad4c-c3dff499cd98 seata.config.nacos.group=SEATA_GROUP seata.config.nacos.username=seata seata.config.nacos.password=seata seata.registry.type=nacos seata.registry.nacos.server-addr=localhost:8848 seata.registry.nacos.namespace=7322f40d-74de-4679-ad4c-c3dff499cd98 seata.registry.nacos.group=SEATA_GROUP seata.registry.nacos.username=seata seata.registry.nacos.password=seata logging.level.io.seata = debug # # # # # # # # 1 (library) the main data source, order data source spring. The datasource. Dynamic. The primary = master spring. The datasource. Dynamic. Seata = true spring.datasource.dynamic.datasource.master.driver-class-name=com.mysql.jdbc.Driver spring.datasource.dynamic.datasource.master.url=jdbc:mysql://localhost:3306/seata_order?useUnicode=true&characterEncodin g=utf-8&useSSL=false spring.datasource.dynamic.datasource.master.username=seata Spring. The datasource. Dynamic. The datasource. Master. Password = seata # # # # # # # # # 2 (credit depots) data source spring.datasource.dynamic.datasource.credit.driver-class-name=com.mysql.jdbc.Driver spring.datasource.dynamic.datasource.credit.url=jdbc:mysql://localhost:3306/seata_credit?useUnicode=true&characterEncodi ng=utf-8&useSSL=false spring.datasource.dynamic.datasource.credit.username=seata spring.datasource.dynamic.datasource.credit.password=seataCopy the code

Inventory application (a database) :

# # # # # # # # # # service registry spring. Cloud. Nacos. Discovery. The server - addr = localhost: 8848 spring. Cloud. Nacos. Discovery. The username = nacos_test Spring. Cloud. Nacos. Discovery. Password = nacos_test # # # # seata related seata. Enabled = true seata. Tx - service - group = my_test_tx_group seata.enable-auto-data-source-proxy=false seata.config.type=nacos seata.config.nacos.data-id=seataServer.properties seata.config.nacos.server-addr=localhost:8848 seata.config.nacos.application=seata-server seata.config.nacos.namespace=7322f40d-74de-4679-ad4c-c3dff499cd98 seata.config.nacos.group=SEATA_GROUP seata.config.nacos.username=seata seata.config.nacos.password=seata seata.registry.type=nacos seata.registry.nacos.server-addr=localhost:8848 seata.registry.nacos.namespace=7322f40d-74de-4679-ad4c-c3dff499cd98 seata.registry.nacos.group=SEATA_GROUP seata.registry.nacos.username=seata seata.registry.nacos.password=seata Logging. Level. IO. Seata = debug # # # # # # data source spring. The datasource. Dynamic. The primary = master spring. The datasource. Dynamic. Seata = true spring.datasource.dynamic.datasource.master.driver-class-name=com.mysql.jdbc.Driver spring.datasource.dynamic.datasource.master.url=jdbc:mysql://localhost:3306/seata_storage?useUnicode=true&characterEncod ing=utf-8&useSSL=false spring.datasource.dynamic.datasource.master.username=seata spring.datasource.dynamic.datasource.master.password=seataCopy the code

Pom depends on:

<dependency> <artifactId>dynamic-datasource-spring-boot-starter</artifactId> <groupId>com.baomidou</groupId> < version > 3.2.1 < / version > < / dependency > < the dependency > < groupId > com. Alibaba. Cloud < / groupId > <artifactId>spring-cloud-starter-alibaba-seata</artifactId> <exclusions> <exclusion> <groupId>io.seata</groupId> <artifactId>seata-spring-boot-starter</artifactId> </exclusion> </exclusions> </dependency> <dependency> < the groupId > IO. Seata < / groupId > < artifactId > seata - spring - the boot - starter < / artifactId > < version > 1.4.2 < / version > < / dependency >Copy the code

4.3 Starting Verification

Let’s verify that this multi-source transaction is valid in a variety of cases, plus SEATA to control what happens with the transaction. The logic of multi-database and multi-table invocation is as follows: The order service inserts the order record into the order library, then deducts the credit score from the credit score library, and temporarily does not access the inventory service.

@RequestMapping("/orderByCredit") public Boolean orderByCredit(String userId, @Nullable String requestId, Orderservice. onlyOrder(userId, "product-1", requestId, count); / / (2) the library, access to credit check and deduct the credit points. CreditService checkHasCredit (userId, count); return true; }Copy the code

2) Use multi-source @DS annotations to access the credit library to modify the data

@DS("credit") public void checkHasCredit(String userId, Integer val){ Credit credit = creditDAO.selectByPrimaryKey(userId); If (credit == null) {throw new RuntimeException(" no credit, no purchase "); } credit.setCredit(credit.getCredit() - val); creditDAO.updateByPrimaryKey(credit); If (credit. GetCredit () <= 0) {throw new RuntimeException(" credit is not enough, cannot buy "); }}Copy the code

3) Using the postman request, the new order record is successfully added, and the operation of the second database credit score is insufficient, according to the expected situation, the transaction will not take effect, the order library insert data successfully, the interface also reported an error.

4) Use spring’s @Transactional annotation to place an order

@RequestMapping("/orderByCredit") @Transactional public Boolean orderByCredit(String userId, Orderservice. onlyOrder(userId, "product-1", requestId, count); @nullable String requestId, int count) { / / (2) the library, access to credit check and deduct the credit points. CreditService checkHasCredit (userId, count); return true; }Copy the code

The @ds (“credit”) annotation for operation ② is invalid. The credit library is not accessed as expected, but the data inserted by operation ① is rolled back.

Note: The order library order table is successfully inserted, and the order log fails to be inserted will be rolled back.

Transactional transactions in a method that operates on multiple data sources and cannot be Transactional without using @DB annotations to switch data sources will fail. However, transactions that do not Transactional will fail because they have not yet been committed. How to implement transaction control for this multi-source situation by using SEATA5), seATA’s @GlobalTransactional annotation, to make a single request

@RequestMapping("/orderByCredit") @GlobalTransactional public Boolean orderByCredit(String userId, Orderservice. onlyOrder(userId, "product-1", requestId, count); @nullable String requestId, int count) { / / (2) the library, access to credit check and deduct the credit points. CreditService checkHasCredit (userId, count); return true; }Copy the code

With SEATA available, transactions can be rolled back if an error occurs anywhere in steps 1 and 2.

Seata’s @GlobalTransactional annotation and Spring’s @Transactional annotation will result in a switch exception.

4.3.2 Multiple libraries and multiple tables across applications

The call logic is:

The order service inserts the order record into the order library, deducts credit points from the credit branch library, and remotely invokes the inventory service to deduct the inventory.

1) Placing orders, resulting in errors in inventory deduction

Order service

@RequestMapping("/orderByCredit") @GlobalTransactional public Boolean orderByCredit(String userId, @nullable String requestId2, @nullable String requestId2, int count) { Orderservice. onlyOrder(userId, "product-1", requestId, count); / / (2) access to direct credit library, check and deduct the credit points creditService. CheckHasCredit (userId, count); / / (3) call service, inventory deduct inventory storageFeignClient. DeductFlow (" product - 1 ", the count, requestId2); return true; }Copy the code

The inventory service

public void deductFlow(String commodityCode, int count, String requestId) throws InterruptedException {
    Storage storage = this.deduct(commodityCode, count);
    StorageFlow storageFlow = new StorageFlow();
    storageFlow.setStorageId(storage.getId());
    storageFlow.setFlowId(StringUtils.isEmpty(requestId) ? String.valueOf(System.currentTimeMillis()): requestId);
    storageFlowDAO.insert(storageFlow);
}
Copy the code

If an error occurs anywhere in the inventory service, the database operation in the calling chain is rolled back, the @GlobalTransactional annotation propagates the XID requested by SeATA through the Request, and the called service, if connected to SeATA, forms a complete distributed transaction.

4.3.3 Cross-service invocation eat exception

1) As shown in the code below, the ordering service accesses the database of its own application, and Feign remotely calls the inventory service, but catches an exception.

@RequestMapping("/orderByCredit") @GlobalTransactional public Boolean orderByCredit(String userId, @nullable String requestId2, @nullable String requestId2, int count) { Orderservice. onlyOrder(userId, "product-1", requestId, count); / / (2) access to direct credit library, check and deduct the credit points creditService. CheckHasCredit (userId, count); Try {/ / (3) call service, inventory deduct inventory storageFeignClient. DeductFlow (" product - 1 ", the count, requestId2); } catch (Exception e){ e.printStackTrace(); }}Copy the code

2) Make the interface request, step ①, step ② normal request, step ③ internal database primary key abnormal error. The global transaction was committed, indicating that SEATA did not detect an exception.

Five, the summary

  1. A distributed transaction can be implemented using @GlobalTransactional at the method entry where it is required

  2. Multiple data sources cannot put @Transactional around methods. This will cause switching libraries to fail. Transactional effects can be achieved directly using @GlobalTransactional because this is a distributed transaction scenario by nature.

  3. The global transaction final decision is made by the global transaction entry application (TM). Even if an exception occurs on the downstream transaction node, as long as TM does not catch the exception, there is no global rollback. That is, the resolution is issued by TM, not by TC as some documents say.

  4. Analysis: TM resolution is simple and fast. Compared with TC performance, TC pressure is dispersed by each TM

  5. The global transaction ID needs to be passed on across service invocations. Seata already encapsulates the relevant code and relies on related packages such as Spring-cloud-starter-Alibaba-seata