Creditease Institute of Technology
The core of Internet financial system is payment and settlement, and the basis of payment and settlement is account system. Mutual gold account system is characterized by large concurrency, fast response, large transaction amount and prominent hot account problems. A qualified account system should not only solve the above problems, but also absolutely guarantee the security of funds. The account system of CreditEase, the payment and settlement center of the Internet finance company, also has the above characteristics.
1. Account system
1. Account structure
Creditease payment and settlement account system is a three-layer structure of customer, user and account. The id number and id type uniquely identify a customer, while the customer number and organization number identify a user. Multiple accounts of different types can be opened under one user. As shown in figure:
2. Account Attributes
The account system is based on the account, all operations are carried out around the account, account contains the following attributes:
• Accounts: Changes in the amount of each account should reflect some accounting attributes for accounting purposes.
• Account category: individual account, enterprise account and platform account.
• Account details: The account details reflect the details of each change in the balance of the account, using double entry bookkeeping method, including the other party’s account number, account information, lending direction, summary, debit and credit amount and balance and other information.
• Account balance: Record the real-time balance of the account.
3. Accounting subjects
Under the account hangs under the lowermost accounting account, the accounting account decides the meaning of the account and the direction of balance change. Some attributes of accounting items are as follows:
• Subject categories: Assets, liabilities, owners’ equity, costs, profits and losses, etc.
• Subject level: accounting subject level, level I, level II, level III, etc. Subordinate subjects belong to superior subjects.
• Balance direction: indicate whether the balance is in debit or credit.
• Ending balance of accounts: the balance of all the lower accounts on the previous accounting day will be summarized after the daily cut, and the balance of the lower accounts will be summarized in the upper accounts.
4. Subject tree
Creditease payment settlement account system adopts the concept of account tree, and each institution will be bound to an account tree. The root node of the subject tree is the first-level subject, and the bottom subject is the account, with the structure as follows:
2. Account system architecture
Creditease Payment settlement account system adopts the distributed micro-service framework developed by the company, exposing HTTP JSON interface externally, and adopting redis message queue communication among internal services.
1. Functional architecture of the account system
Creditease payment and settlement account system is divided into access module, accounting subsystem, account opening subsystem, asynchronous accounting module, query subsystem, timing task subsystem, day end subsystem, and asynchronous log module. The following is the function module diagram of the accounting system:
• Access module: provides public services such as packet parsing, visa verification, parameter verification, and permission authentication. It is a unified entrance to the account system.
• Asynchronous log module: records service system request packets asynchronously.
• Billing subsystem: the core module of the account system, which processes the billing requests of the business system.
• Registration subsystem: processes registration requests of business systems.
• First account opening: To open a customer, user and pre-configured default account for an individual or business.
• Designated account opening: after opening an account for the first time, individuals or enterprises can designate to open an account according to the account number.
• Query subsystem: provides some query functions of accounts and bookkeeping.
• Asynchronous billing module: provides the function of asynchronous recording of account flow.
• Scheduled task subsystem: processes scheduled tasks such as retry failures and hot accounts.
• Daily end subsystem: provide daily cutting and daily end run batch function.
1) Accounting processing
Bookkeeping processing is the core of the account system function, the function of performance requirements is higher, the hotspot account under high concurrency problem is outstanding, the correctness of the funds also must make the pledge that we, and according to the different business, accounting entry is multifarious, appropriate letter payment settlement account system is how to deal with these problems, it focuses on:
• Account system bookkeeping adopts the concept of bookkeeping service. Each bookkeeping service is a template of bookkeeping entry, according to which the business system imports information such as the amount of bookkeeping, account number or user number.
• The account system uses redis distributed lock to prevent the business system from submitting requests repeatedly. Set up an anti-duplicate table for bookkeeping orders, and perform idempotency check for bookkeeping requests according to the request number and organization number.
• Double entry bookkeeping, according to the accounting rules according to the borrowing records, borrowing must be credited.
• When handling bookkeeping, update the account balance and return the result to the business system synchronously, and process the bookkeeping flow asynchronously. At the same time, set the compensation mechanism, retry the failed order of bookkeeping flow processing regularly, and alarm manual intervention after three failed retries.
• Accounting rules processing. Each accounting service can bind some accounting rules. The account system traverses its bound rules according to the accounting service and processes them in sequence.
2) Hot account issues
Hot account problem is the pain point of the account system, but also troubled us for a long time, here to emphasize.
• The charge entry at the time of recharge is,
Debit: Tripartite payment to liquidated account (+)
Credit: Personal Balance Account (+)
When a large number of users recharge, the account to be cleared of tripartite payment is a hot account, frequently increasing the balance.
• The entry at the time of withdrawal is,
Debit: Personal Balance Account (-)
Credit: Tripartite payment asset Account (-)
When a large number of users withdraw cash, the asset account of tripartite payment is a hot account, frequently reducing the balance.
• The entry for service charges is,
Debit: Personal Account (-)
Credit: Merchant service fee Account (+)
When a large amount of service fee is charged to users, the merchant service fee account is a hot account, which will frequently increase the balance.
• The entry for payment of business service charges is,
Debit: Merchant Service fee Account (-)
Credit: Personal Account (+)
When a large amount of service fee balance is used for payment to users, the merchant service fee account is a hot account and will frequently reduce the balance.
In the case of high concurrency, when the above type of hot account occurs, due to the database row-level lock, the operation of updating the balance of the same account is changed from parallel to serial, and the response time of a single request becomes longer, thus dragging down the entire bookkeeping service.
Creditease Pay settlement account system has dealt with the above problems as follows:
According to the change direction of amount, hot accounts are divided into frequency-increasing accounts (frequent increase of balance), frequency-decreasing accounts (frequent deduction of balance) and dual-frequency accounts (frequent deduction of balance increase).
• Frequency-charged account processing: quasi-real-time balance update. The amount change is inserted into the temporary table first, and the scheduled task summarizes the amount according to a certain frequency, updates the account balance, and then deletes the temporary record. When the frequency account to reduce the money balance is insufficient, take the initiative to summarize the amount. We set a Redis lock during the execution of the scheduled task to prevent concurrency. When we actively summarize, we will judge whether the Redis lock exists. If there is, the scheduled task is being executed, and there is no need to actively summarize, because the balance may be really insufficient. Active summary will also set the Redis lock, scheduled tasks will also be judged.
• Frequency reduction account processing: the frequency reduction account will be split into multiple sub-accounts, and the amount alarm will be set for the frequency reduction sub-account. If the balance of a frequency reduction sub-account is insufficient to trigger the alarm, the fund will be collected for the sub-account, and the balance of other sub-accounts will be collected to the sub-account (each sub-account will set the amount limit of collectible). If the balance of this sub-account is found to be insufficient during the transaction, switch to another sub-account for accounting. Due to the split account, balance query needs to summarize the balance of each sub-account return; To record the master account flow, the balance after accounting needs to be calculated asynchronously. When you add money to a frequency-reduced account, you need to evenly distribute the money to the subaccounts that are disconnected.
• Dual-band account processing: Split the dual-band account into multiple sub-accounts. Add money, quasi real-time update the balance, the sub account amount change inserted into the temporary table, the scheduled task according to a certain frequency of summary amount, the summary amount update into the corresponding sub account, and delete the amount change record; The reduction follows the logic of the previous frequency reduction account.
3) Deadlocks in accounting
In high concurrency cases, deadlocks can occur when multiple accounts have previously transferred money to each other.
For example: A balance account – > B balance account (thread 1)
B balance account — >A balance account (thread 2)
Two transfer request concurrent, account system for each transfer request will update the balance of A and B, the two update need in A transaction, thread 1 to update A normal process, to update the B, thread 2 update B first, then update A, thread 1 update after A lock waits for B, do not commit the transaction, thread 2 after updating the B will wait for A lock, do not commit the transaction, The two threads then wait for each other to lock, causing a deadlock.
Creditease payment settlement account system proposed A solution to this situation. It sorted the account numbers and then updated the balance, so that each thread would update A first and then UPDATE B, which solved the deadlock problem.
2. Storage layer architecture of the account system
Creditease Payment and settlement account system uses Mysql for database and Redis for cache.
• Mysql database adopts master-slave architecture, with one master and two slave. The master database synchronizes data to the slave database. For some tables with a large amount of data, the representative one is the account flow table, which should be queried according to the account dimension and summarized according to the time dimension. Therefore, according to this characteristic, a redundant table, one table according to the account, one table according to the date.
• Redis adopts a cluster architecture with each point in the cluster working in active/standby mode.
3. Network layer architecture of the account system
All services of the account system are deployed in the same machine room. The billing subsystem and asynchronous billing module are deployed on four different physical machines, and other subsystems and modules are deployed on two different physical machines. The front end uses NGINx to achieve load balancing.
-END-
Source: Creditease Institute of Technology author: Li Ruicheng reserved