“We’ve already used it” is one of our favorite words. Behind these simple words is a lot of trust and commitment. We will share in-depth cases through ** “Believe in the power of Openness” series to see the business value that a database brings to users in various industries from a business perspective. This article will introduce the practice of TiDB in expanding the capability map of zheshang Bank’s data architecture system.
Data-driven decisions make financial services more efficient.
As one of the 12 national joint-stock commercial banks, Zheshang Bank Co., LTD. (” Zheshang Bank “for short) is committed to building a platform service bank and providing customers with open, efficient, flexible, sharing and ultimate comprehensive financial services. In The Banker’s “2020 Top 1000 Global banks” list, it ranks 97th in terms of tier 1 capital and total assets. By the end of June 2020, Zheshang Bank has set up 260 branches in 19 provinces (municipalities directly under the Central Government) and Hong Kong Special Administrative Region, effectively covering the Yangtze River Delta, Bohai Rim, Pearl River Delta and some central and western regions.
Business challenges
With the business development of Zheshang Bank and the surge of data volume, the shortcomings of data processing and storage capacity of foreign commercial databases are gradually exposed, and the original data architecture system cannot meet the needs of new business scenarios. Under the advocacy of industrial policies and drawing on the experience of financial peers, Zheshang Bank turned its attention to distributed database products widely used on the Internet and started special selection work.
The goal of Zheshang Bank is to find an OLTP-type database that can carry massive data, support complete distributed transactions, and provide financial high performance, high concurrency, and continuous high availability. The architecture can be flexibly and horizontally expanded to smoothly cope with all kinds of traffic. At the same time, it has a strong ecosystem to support the connection of all kinds of data application ecology. After several rounds of competitive product comparison tests and application compatibility verification, TiDB database leads in scalability, query performance under massive data scale, transaction integrity and other inspection items. Zhejiang Merchant Bank chooses TiDB distributed database to expand the capacity map of data architecture system.
TiDB solution
Considering the long-term construction of the two centers and three centers, Zheshang Bank has deployed TiDB distributed database cluster in the dual data centers in Hangzhou, using the data copy mode, which can be further expanded in the future. At present, it has launched and put into operation three applications: risk chain inquiry of telecom fraud event (short for: Telecom anti-fraud), foreign exchange transaction management and executive cockpit, and will gradually extend to the ODS of the whole bank, Internet C-end inquiry transaction scene and other business fields.
Telecom fraud risk event inquiry system
The telecom fraud Risk Event Query system keeps nationwide telecom anti-fraud data, including detailed data such as transaction accounts and transaction amounts. The system supports the inquiry and monitoring of suspicious transactions, analyzes the characteristics of transaction amount, number of transactions, type, time, frequency and receiver and payer, and timely takes measures such as suspension of transactions or notifying public security organs if abnormal transactions are found.
The scale of single table data of telecom anti-fraud system has exceeded 2 billion pieces, and 1 million pieces of new data are added every day. The original OLAP database cannot reach the business expectation in terms of data scale and business query performance. TiDB has the ability of elastic scaling. By simply adding nodes, the performance and throughput of the entire system can be linearly improved. It supports the storage and query of massive structured data, and the return time of SQL query can be shortened from tens of seconds to tens of milliseconds. The TiDB Lightning data import tool can be used to quickly import millions of data records in a single data file, meeting the operation window requirements of services.
Foreign exchange transaction Management
Based on the requirements of foreign exchange supervision, foreign exchange transaction management system needs to complete the collection, import, storage, statistical analysis, data mining and other functions of foreign exchange transaction data. Relevant data requires long-term storage, and it is expected that the data scale will reach one billion levels in the next few years. The original Oracle database requires the use of partitioned tables, resulting in an increase in operation and maintenance management and development costs. The distributed feature of TiDB can meet the requirement of data scale expansion, and the elastic capacity expansion can realize automatic data rebalancing. It is completely transparent to services and does not require the intervention of OPERATION and maintenance personnel. In addition, the high availability mechanism of TiDB provides high availability of remote multiple activities. The recovery process does not require manual operation. The cluster automatically implements DISASTER recovery and ensures consistent data recovery, freeing O&M personnel from worries.
Executive cockpit
The executive cockpit is an analysis system that provides real-time business indicators for the management in the industry. By breaking data isolation, it can reflect the running state of various businesses in real time, visualize, visualize and concretize the collected data, and realize the indicator analysis and decision-making scenarios. Zheshang Bank uses data synchronization tools to write data changes in heterogeneous databases such as UPSTREAM DB2 into TiDB in real time, and synchronizes real-time change data in TiDB to downstream Kafka at the same time. Flink receives messages from Kafka for streaming calculation, and the whole system forms an efficient and easy-to-use real-time computing platform. Realize the real-time statistics of various business conditions of the bank, and display them on the large visual screen. Through online capture and real-time analysis of massive financial data changes, it provides business decision-making basis for managers at all levels of banks and improves service efficiency.
Future scene exploration
Distributed real-time ODS
TiDB database has the service capability of large-scale data platform. After the integration of TiFlash column storage engine, IT has become a real HTAP database (ONLINE transaction processing OLTP + online analysis processing OLAP). It can be used as the business database of the data source for business query processing, and as the real-time OLAP engine for analytical scenario calculation. There are many OLTP heterogeneous databases in Zhesang Bank, mainly DB2 and MySQL, as well as some Oracle and SQL Server. All kinds of heterogeneous databases lack efficient means of data synchronization, gradually forming the state of data island. In order to cope with the real-time Data requirements of various businesses, Zheshang Bank plans to further expand on the basis of the existing platform, build distributed real-time ODS (Operational Data Store) based on TiDB, and realize the quasi-real-time synchronization of various Data in heterogeneous databases. TiDB’s incremental data synchronization tool, TiCDC, can provide real-time, high-throughput and stable data subscription services for downstream data consumers, and connect with MySQL, Kafka, Pulsar, Flink, Canal and other heterogeneous ecosystems through open data protocol. It meets the requirements for application and analysis of various types of data in big data scenarios.
Internet transaction inquiry
Zheshang bank for enterprise users with payroll, every month need to focus on tens of millions of flowing water, the original business database is difficult to meet the longer retention time, on the one hand, the slow query speed affect the customer experience, on the other hand, can’t with other data linkage, subsequent zheshang bank will also consider to choose TiDB to pilot and alternative.
Why TiDB?
At the functional level, TiDB supports complete distributed transactions, simultaneously realizes online transaction processing and online analysis processing of massive data, and provides high reliability and high availability of financial level. When any data center goes down, the whole cluster can automatically realize business switching and recovery without perception. At the architecture level, as a NewSQL distributed database, TiDB adopts the architecture design of storage and computing separation, which can more efficiently solve the data expansion problem that traditional relational database cannot solve, and has more extensive scene adaptation ability. At the level of operation and maintenance, TiDB compatible with MySQL protocol significantly reduces the hidden cost of personnel learning and development. Flexible query interface for service operators, easy to monitor and maintain.
Walk with customers, believe in the power of openness
Every time database architecture improvement and implementation, no matter TB or PB level, need to pay efforts, but it is also worth every enterprise to practice. In this era, regardless of the size of the enterprise, we should learn to use the power of open source to avoid repeating the wheel. Behind every seemingly easy, there are unknown efforts, behind every seemingly bright, there are unknown efforts. The road to build distributed database is long and difficult. TiDB is willing to work together with Zheshang Bank and every customer to do things well.