The background,

The account funds of the company’s business system are connected with the third-party depository business, which refers to the customer transaction settlement fund management services provided by banks and securities companies for investors in accordance with relevant laws and regulations. According to the requirements of the bank, after the market closes each trading day, the business system will generate a settlement file of the customer’s fund changes and send it to the bank for fund settlement.

Under micro service architecture, the system exist multiple business subsystem (the same set of account system), so each business subsystem of money change of settlement, we separate a settlement subsystem, settlement subsystem to do is according to the bank for money data statistical rules of each business subsystem customers and generate money changes occurring clearing files, Then settle with the bank.

Ii. Problems and challenges

2.1 Data synchronization and data accuracy assurance

Because each business subsystem and settlement subsystem is not in the same database, so we are faced with the first problem is to get the data synchronization to each business subsystem of the capital change settlement subsystem, is due to a daily settlement time, so the real-time demand of data is not high, as long as change to synchronize the data before clearing. The accuracy of data means that the data of the settlement subsystem must be consistent with the actual data of the business subsystem, neither more nor less, otherwise the settlement will fail.

2.2 Flexible response to changing business scenarios

The development of business is changeable. If several business subsystems are added to the system or cut off, the settlement subsystem is better to meet the access or elimination of new business without any change.

2.3 Fast access of service subsystems

For a business subsystem, getting access to a settlement subsystem is as simple as possible, preferably in a few lines of code.

Iii. Solutions

3.1 Data synchronization Scheme

Common data synchronization schemes include database synchronization, data file synchronization, remote interface acquisition, MQ data synchronization, and manual synchronization. Here’s a brief analysis of the pros and cons of each of the five synchronization methods.

1) Database synchronization

Database synchronization refers to the synchronization of data from database A to database B through some tool, such as ETL tool, MySQL, can consider Canal, Alibaba open source MySQL binlog incremental subscription & consumption component.

  • Advantages: High accuracy of data

  • Disadvantages: the implementation is more complex; Expandability of

2) Data file synchronization

File synchronization means that the business subsystem puts the capital data generation file in a location, such as FTP, and then the FTP file is retrieved and parsed into the repository on the settlement subsystem.

  • Advantages: High data accuracy
  • Disadvantages: Lack of flexibility

3) Remote interface acquisition

Each business subsystem provides an interface for fund data query, and the settlement subsystem obtains data through interface call

  • Advantages: It is suitable for data transmission synchronization in the case of small amount of data
  • Disadvantages: low efficiency; Poor flexibility and scalability

4) MQ data synchronization

When the capital data changes, the business subsystem sends the change data to the settlement subsystem in quasi-real time through MQ, and the settlement subsystem consumes the message and stores it in the database.

  • Advantages: Flexible and extensible, and meet the requirements of rapid access of the service subsystem, only need to send a message, the settlement subsystem only need to pay attention to the consumption message and storage, even if the subsequent access of new service subsystem, the settlement subsystem does not need to change or a small amount of changes
  • Disadvantages: Strong reliance on MQ middleware, data accuracy cannot be guaranteed when MQ messages block or go down

5) Manual synchronization

Relying on manual export && import data is actually not too much work after the relevant scripts are defined

  • Advantages: High accuracy of data, data can be checked
  • Disadvantages: no automation, increased manual workload

Iv. Implementation Plan

Data synchronization scheme

The data synchronization scheme can be MQ data synchronization and manual synchronization. MQ is used for data synchronization. If MQ synchronization fails, manual synchronization is used for disaster recovery. Generally, MQ is fine. Of course, this scheme is mainly considering its flexibility and expansibility, coupled with the implementation of relatively simple. Considering the accuracy of data, data file synchronization is also a good way.

The overall train of thought

The overall idea is as follows: the settlement subsystem defines unified message fields, such as user number, change amount, change time, change type (profit/loss/freeze/unfreeze), business source and other related fields. Each business subsystem sends messages according to the unified message format, and the settlement subsystem consumes the information uniformly. Before initiating settlement, the system shall prepare settlement data for the operation to check, and then initiate settlement with the bank after checking.

Possible problems and solutions

Because there may be news consumption congestion situation, so in the generated settlement documents, in order to guarantee the accuracy of the data, each business subsystem to provide a query data interface of the total amount of the total number of article and as a basis for the data verification, settlement subsystem according to the source of business calls the corresponding business subsystem interface to get verification data, and compare the data of settlement subsystem, again If yes, data on the service subsystem is synchronized. If no, manually synchronize alarms.

Give the article a “like” if it’s helpful to you. If there are any inaccuracies, please point them out. Article first public number: dancing robot, welcome to scan code attention.