As an efficient and stable open source distributed database, TiDB is widely used in banking, securities, insurance, online payment and fintech industries at home and abroad, and supports users’ critical computing in about 20 different financial business scenarios. In the practice of TiDB in the key business scenarios of the financial industry (Part 1), we introduced the application of TiDB in the core transaction scenarios of banks. This article will mainly share the practice of TiDB in the core and peripheral key business scenarios.

The practice of TiDB in payment business

We also have many cases in core and peripheral key business scenarios, such as the typical online payment business today. TiDB is mainly involved in the payment field, including the online transaction database of commercial banks’ Online/UnionPay payment business, and the core payment system of mobile payment business of third-party payment companies. Supported by TiDB’s large-scale throughput, high-performance large-scale concurrent online transactions, multi-center and multi-active DISASTER recovery, and elastic expansion mechanism:

  • On the side of commercial banks, large concurrent payment transaction instructions and payment message data processing from UnionPay/UnionPay cards;

  • On the third-party payment side, transaction instructions and payment packet processing from the background of the embedded payment business system in mobile payment App, merchant mobile POS and payment scenarios;

  • The online processing class of payment transaction mainly includes processing of payment wallet data, online insertion and update of payment transaction data, real-time query of transaction status, transaction history record and bill query;

  • Calculation of rates in payment transactions;

  • Data aggregation and rule calculation of anti-money laundering system in payment process.

Since it was put into operation in 2018, TiDB has stably supported the core system of Bank of Beijing’s online payment and UnionPay’s card-free payment for three years, adopted the multi-activity disaster recovery structure of Beijing city + Xi ‘an, and successfully passed two double Eleven days, which better supported this business. In addition, we have achieved specific cases in payment settlement, billing and anti-money laundering platforms of Tianyi Payment, payment data aggregation of Alipay and online payment core and payment wallet core platforms of PayPay, the no.1 payment company in Japan.

The practice of TiDB in Internet financial management business

TiDB mainly implements online financial services in the wealth management business line of joint-stock commercial banks, and supports financial services through large-scale throughput, high performance and concurrent online trading, multi-center and multi-active disaster recovery and flexible expansion mechanism provided by TiDB:

  • Core processing of online transaction transaction of financial transaction;

  • Batch calculation of daytime and end-of-day data of financial transaction system;

  • Batch calculation and output of financial transaction system data to downstream business systems such as regulatory reporting and data warehousing;

  • Two-center active-active DISASTER recovery (Dr) deployment provides high level of business continuity.

In the first half of this year, we completed the operation of core database of financial transactions and online batch database with Everbright Bank, which better supported the processing of core online transactions of the whole everbright Bank’s financial services and batch calculation of daytime and end-of-day data. In addition, we also landed on the financial sales platform of Bank of Beijing and the financial transaction flow of webank.

TiDB practice in real-time risk control business

One of our key financial applications is the real-time risk control business. Different from traditional risk control, with the increase of internet-based business scenarios, banks and pan-financial institutions have very high requirements for real-time risk control. At present, TiDB has been applied in real-time risk control data aggregation, storage, management, processing and calculation scenarios in risk control business. Through the core mechanism of TiDB distributed storage, it can deal with real-time writing of massive data. At the same time, the design of distributed computing layer and column and column hybrid engine can provide real-time processing capability for point checking calculation and batch summary statistical calculation of risk control index. The traditional “T+1” risk control business processing capability based on big data will be directly upgraded to “T+0” level, such as risk control data calculation and query up to the second level.

In terms of financial business scenarios, we have implemented a series of scenarios including Bank of Beijing online business risk control model management platform, Webank CNC anti-fraud system, Tianyi Payment anti-money laundering platform, Lakala Financial real-time risk control platform and so on. At the same time, in the Internet and e-commerce business scenarios, the risk control platform of Shopee, a well-known e-commerce company in Southeast Asia, xiaohongshu anti-fraud system, real-time risk control platform and Pinduoduo risk control platform have all been implemented.

Typical scenario of TiDB insurance industry

Besides banking, TiDB is also widely used in the insurance industry. In the insurance industry, it mainly has production business in the front desk, middle stage and back stage.

At the front desk, it mainly focuses on online transactions on the Internet and mobile terminals, including insurance policies, wealth appreciation, member activities and other support for online transactions. Last year, we successfully supported the business of Ping An Property Insurance Nubao Insurance, and completed the production and online operation of the whole cluster and a peak transaction of Ping An God of Wealth Festival in a very short time.

Business in China, mainly involves in China business group of the background data support for the front end, including China’s, unit module of service transformation and renovation work of the business, TiDB can through the cloud native architecture better adaptation micro service application environment, eliminate dependence on distributed data access framework, without the introduction of data middleware, And can achieve online elastic expansion. We have landed in both the “Golden Steward” business of Ping An Life Insurance and the “inquiry and issuing” business of Ping An Property Insurance.

In the background, due to TiDB’s large-scale massive data storage architecture and bulk data processing capability, we have implemented several cases in authentication, payment, settlement and risk control businesses mentioned above. In this type of business, there is a preference for real-time OLAP analysis, which involves TiDB providing multiple upstream data source aggregation solutions.

How do I migrate from the legacy architecture to TiDB

From the financial industry, the traditional business migration to TiDB, I would like to share with you a few points.

The overall architecture of TiDB is multi-centric, especially for core transactions and critical business scenarios. We also implemented a multicentric architecture on the two core transaction repositories including Bank of Beijing and Everbright Bank.

Planning and management of TiDB core trading system migration and commissioning

Because it involves very important and critical systems in the whole financial institution, the technical principle for the operation of TiDB, the core trading system of **, is to be checked everywhere and be reversible step by step. ** has the following production strategies for your reference.

** The first strategy is the double-write strategy. ** implements a dual-write route on the application side to forward the traditional group library, the transaction group library, and TiDB as a cluster mirror library. Its advantage is that the control particles are very fine. TiDB obtains all the transaction loads and characteristics from the time dimension and transaction load dimension, and the cutover window is short input. However, we need to construct double-write routing mechanism and data verification mechanism, of course, we also have some tools and technical means to provide.

** The second strategy is to conduct production and migration as a master-slave approach. ** Trade group libraries, such as Oracle, continue to carry all trade group library read/write traffic. TiDB uses specialized third-party tools such as Golden Gate to easily capture real-time changes in Oracle’s entire data volume and increments. The advantage of this strategy is that the lead time is very short and the retrofit effort across the business is very low. However, some additional work needs to be done. First, more detailed data validation is required, because some changes in the data are obtained through the master-slave structure. In addition, when the master-slave check, if there is a data problem, it needs to be corrected by technical means before cutover. Of course, we have some calibration and finishing technology, a mechanism to help the customer do the calibration and finishing before cutover, delivery and production.

** The third strategy is to use TiDB as the main library directly, using the traditional library as the backing scheme. ** This program is a complete adaptation of our products with the main domestic database head program manufacturer DSG. TiDB can now be a cluster based on DSG with Oracle system as the backing. The advantage of this strategy is that it can be put into production quickly and guarantee the bottom, but it also needs to construct data verification and correction means, and the overall backoff time is long.

The last strategy is to make a gray scale, through the segmentation of transaction modules, according to the boundary of the module, part of the transaction is undertaken on the main database, and the other part of the transaction is placed on TiDB. The advantage of this strategy is that gray scale is migrated according to business modules and the overall risk of the whole business is well controlled. However, it may require more complex application mode-level subdivision migration scheme and supporting tools.

Data backup and recovery guarantee

Last year we implemented a distributed Backup solution for the physical layer of the storage engine layer, called BR(Backup & Restore). BR can be implemented in a fixed number of cases, the more nodes, the faster the backup, to achieve a non-logical layer and physical layer backup.

** In addition to the Raft Based solution that TiDB is native to, we also implemented a strongly consistent two-centric backup solution in our product last year. ** Some banking institutions or financial institutions may not need the services of the two centers and three centers, but only the disaster recovery plan of the two centers is sufficient. Based on Raft framework, we have implemented a strongly consistent scheme of two centers, which is suitable for the scenarios of same-city and short-distance remote centers with good communication delay between centers, and can achieve the financial level RPO=0.

TiDB 5.0 key planning

In the first half of next year, we will launch our next milestone release, TiDB 5.0.

In this release, we will accumulate all of our improvements and refinements on the core of the product and the surrounding supporting tools for the financial core scenario, including further improved throughput, reduced latency, and improved overall storage stability. At the same time, we will combine HTAP’s engine capabilities with streaming and real-time computing. In addition, with the further adaptation of Geo-partition and cloud native environment, we will have significant enhancement and improvement in private cloud, hybrid cloud and even public cloud environment.