Traditional data service mode is put forward with business language requirements, technical personnel and the business sector business language how to convert science and technology make confirmation data size, development statements provide business after testing, test report often found the difference in their implementation and business requirements, but also repeated communication, from business reporting requirements is put forward to the final production, quick, transplanting, slow months.

Based on the above reasons, we want to build a data platform to support fast, flexible, interactive and exploratory data query and analysis. Let business people no longer rely on, no longer wait for a large number of simple drag-and-drop operations to complete the daily data analysis work.

Then the problem comes, how to plan the technical architecture, data model and selection of BI tools to build a set of self-query platform based on detailed data.

Here provides a big data platform case of The Bank of Jiangsu.

Traditionally, a bank customizes a statement to analyze certain business data, mainly through the business department to put forward the requirements, and the technology department to write the program. Among the problems, the subject also said, from the demand to the department of science and Technology to the final development of the report, there are repeated caliber communication, test number process. A report, from scheduling consideration, requirements, to the final completion of a week or two, or a few months.

The effective way to solve this problem is:

1. Give detailed and wide table data that they can understand to the business department for self-exploration and analysis by the business personnel.

2, basic query statements: from the grassroots business and daily work, functions in a particular job, such as sales enquiry, inventory query, in road inventory, purchase order query, etc., to form a fixed purpose class query statements, users, when the job is need through the query such statements, to get the data, they want to support their work.

The latter is mainly the responsibility of the IT department, the former IT department can be pushed to do. Let’s focus on the former.

BI Platform selection

A BI platform is needed to solve the problem of self-service data collection and self-service analysis. The BI tool FineBI on the market supports fast, flexible, interactive and exploratory data query and analysis, which is also developed by Bank of Jiangsu based on FineBI.

IT personnel and business personnel can jointly customize the business understandable subject package (here IT personnel directly prepare the data by business and subject in FineBI business package), and the data can be designed by business personnel. Through statistical analysis in this way, business personnel can query data independently on the basis of customized data packets. What you see is what you get. Analysis in the process of data query and statistics can greatly improve work efficiency. In addition to building this system, they have also done one thing, which is to train talents with data analysis and mining ability in each business line to solve data analysis problems in daily work.

The platform architecture

1. Big data Platform Construction:

A reliable underlying data processing platform is required to support the highly free real-time query of internal and external data of large volume. From the point of view of economic cost and nonlinear growth trend of future data. When designing the architecture, traditional trading system uses relational database to process OLTP transaction operation, and the transaction data generated is updated to Hadoop platform through batch replication of heterogeneous data or quasi-real-time message queue. Hadoop platform can analyze and mine large volume data. It also provides a real-time retrieval mode based on big data.

2. Data resource integration:

Continuously integrate transaction data, account data and customer basic data of dozens of businesses such as core system, credit management system, credit card system and individual loan system, establish data standards and data governance system, develop multiple internal data marts such as risk data mart, asset and liability management mart and regulatory filing mart. Client risk early warning information gets introduced including regulators, pedestrian customer credit report data, state administration for industry and commerce enterprise registration information data, corporate tax, customs import and export trade data, customer information involving lawsuit court, dishonest person subjected to execution information, environmental information to make punishment, tax information, business information, citizen id card information, personal degree student status information, and public media Negative information and other 19 external data sources, thousands of external data fields, and the use of web crawler technology and named entity recognition technology, capture public network media public opinion information, forming a massive external data mart;

Through the integration of in-line and off-line data, online and offline data, structured and unstructured data on the big data platform, it effectively solves the problem of “information island” commonly faced by traditional banks. On the basis of data integration, it is possible to use intelligent big data analysis tools to conduct statistics, analysis, query and modeling of all kinds of data.

3. Tool selection:

Investigated all kinds of data mining and analysis tools in the market, integrated a variety of tools for people with different needs and different data analysis capabilities to provide use:

  • Data mining and modeling personnel: SAS and distributed R language tools, can use professional data analysis tools for mining and modeling;

  • Personnel with database operation ability: provide SQL – like customized rapid report development tools, all report design and menu controls are configured visually through the browser;

  • For business personnel who understand business system data: provide intelligent BI multidimensional analysis tool FineBI, which can realize all kinds of complex statistics and chart functions by dragging and dropping;

  • For ordinary front-line employees: various templates that have been designed on the query platform can be conveniently used, and visual query tools such as customer relationship atlas and Internet public opinion analysis can be used.

The final result

1. Lower the threshold of data mining analysis

The use of BI platform greatly reduces the threshold of data analysis, and data analysis work can be lowered to the most basic level. No need to know database, or even much professional knowledge of mathematical statistics, as long as they know business, they can customize analysis and mining according to one of their own concerns, and can share it with other employees.

2. Realize discrete management

The traditional mode of report development, maintenance and management are concentrated in part of the development and management personnel, in the face of the growing demand for data analysis, this mode shows the lack of response ability and resource bottleneck. The platform broke the original centralized management mode where the head office customized fixed reports and branch offices could only query, and changed into a discrete management mode where everyone could be a report developer, realizing thousands of creativity and maximizing the value of data.

3. Balance of data security and convenience

While improving the convenience of data analysis, data security is also a key concern of banks. Intelligent multi-star platform also needs to be intelligent in authority management.

First of all, the sensitive fields, such as customer name, address, mobile phone, etc., can be automatically desensitized when defining reports;

Second, the data permission has two dimensions: report and organization. Users in different branches can only view their own organization’s data even if they have obtained the permission of the same report, avoiding arbitrary data transmission.