Anson, a CTO

Graduated from Zhejiang University, he is now fully responsible for technology selection, RESEARCH and development innovation, operation and maintenance management, and has led the team to develop a number of cutting-edge intelligent data solutions for mobile Internet, financial risk control and other industries.


He used to be the chief architect of MSN China. He has more than ten years of experience in senior technology development and project management, and has rich practical experience in big data processing system, large-scale concurrent platform, distributed search system, mobile application development, wireless communication field and smart financial system.

The introduction

The development of China’s mobile Internet has witnessed the vigorous development of China’s big data industry. As a natural product of the mobile Internet era, data intelligence is also the core of a long period of development in the future. Getui (Daily Interaction) coincides with the consensus of the industry. After years of development from its establishment in 2010 to now, Getui has grown from A basic push platform service provider serving developers in the era of mobile Internet to A listed company on gem and the first data intelligence company listed on A-share market in China. As a professional data intelligent service provider, Getui will continue to devote itself to promoting industrial intelligence change with data based on developer services.


I will cover the topic of “data intelligence” in a series of articles. This paper mainly discusses the various aspects involved in data intelligence from the technical perspective, hoping that through this series of content, we can let you have a clear understanding of data intelligence and the technical system involved.


This series will cover the following five aspects:

01

The Coming of the Age of Data Intelligence: Nature and Technical System Requirements

Core content: We will talk about our understanding of data intelligence based on our years of practice in the field of data intelligence, and put forward the corresponding technical system requirements on the whole.


02

Data asset governance under data intelligence


Core content: It mainly discusses how to govern data as an asset, the foundation it needs to have, and how to implement it, so as to ensure the security, rational use and value creation of data assets.


03

Secure computing system under data intelligence


Core content: Current technologies and methodologies that can be used on the premise of keeping the ownership and use rights of data assets separate.


04

Data quality assurance system under data intelligence



Core Content: Big data is big because of its scale and diversity. Different from traditional small data, its correctness can be verified quickly. Then what methods can be adopted to ensure the quality and testability of data?




05

Business exploration and practice in different industries under data intelligence

Core Content: Different industries are like mountains, and data intelligence has distinct industry differentiation. This topic will describe the exploration and practice of several industries that we are deeply involved in, and summarize some experience and lessons.




The body of the

The history of big data

This is the first in a series of articles about the nature of data intelligence as we understand it; At the same time, as the technical director of the company, I would like to discuss with you the requirements based on the technical system, that is, the era of data intelligence, in order to reflect intelligence from data, what needs to be done in terms of technology.


What is data intelligence, and where does the concept come from?


I remember that since 2010, with the rise of mobile Internet, big data also appeared on various media websites and industry forums, and everyone would ask: “Have you engaged in big data?” It’s not really clear how big data should be applied.


What is the evolution of big data like? The diagram below illustrates this somewhat clearly.


I call it the Big Data Maturity Model. This process is essentially what we understand to be the process by which data is transformed from a tool into an asset, from an auxiliary object into a means of production. Many people are trying to make a theoretical definition of the digital economy in order to distinguish the digital economy from the real economy in terms of concepts. My suggestion is to define the digital economy from the perspective of whether it is the main means of production and whether it is the core asset, which is easier and clearer.


According to the actual development in recent years, big data is basically evolving and developing in accordance with the model shown in the figure above.


Around 2013, enterprises began to recognize the value of data, and various industries with big data production environment, such as telecom operators, governments, public security, finance, etc., began to build big data platforms to collect and store data generated by enterprise business. At the same time, financial and other industries also began to purchase a large number of external data, hoping to quickly mine the value of data through external data and make up for their own data shortage. Many companies engaged in data aggregation and related services have gained development opportunities.


In 2015, big data entered the monitoring stage, and business monitoring was realized in the form of data big screen, which is the earliest and the first mature application direction of big data. For the government, central enterprises and large state-owned enterprises, data display applications such as data screen and leadership kanban are the most direct ways to reflect the value of big data.


In 2017, the big data platform construction basic perfect, simple data show start is difficult to meet the diverse needs of the enterprises, large data began to combine with business scenarios, based on the insight into the big data to business problems, present the situation of flowers, respectively applied in precision marketing and risk control in the field of financial fraud, the public security in the field of forensic investigation, Industrial field of fault prediction and early warning.


Simple mathematical statistics are no longer enough to satisfy the enterprise’s insight into business scenarios. Therefore, data mining and data modeling emerge as The Times require. AI modeling platform and data science platform have begun to come into people’s sight, and some startups focusing on modeling platform have emerged. However, more companies internalize AI modeling platform into their own capabilities, and form solutions based on AI modeling platform to help enterprise customers implement big data applications.


Around 2019, big data began to enter the stage of business decision-making, that is, the machine forms data reports or data reports, and the business personnel make decisions instead of the machine directly gives decision-making suggestions, so that the machine has reasoning ability. For example, in take-out and travel scenarios, the systems of Meituan and Didi directly form the optimal scheduling mode. The system automatically completes the decision-making process and sends tasks to riders and drivers. This relatively common scenario of consumer Internet will gradually appear in industrial Internet and enterprise business scenarios. That is to say, big data from business digital stage to data intelligent stage.

Characteristics and definition of data intelligence

From the development of big data in the previous section, we can see that data intelligence currently corresponds to the stages of decision making, optimization and business remodeling, that is to say, making machines capable of reasoning; These capabilities mean the gradual maturity of cognitive technologies such as natural language processing (NLP) and Knowledge Graph, which is why NLP and Knowledge Graph have become hot topics in the market in 2018. Therefore, the new demand of data-driven decision-making and data-driven business development will inevitably lead to the rise of a number of data intelligent companies.


In the future, as technology matures, big data will move from decision making to the final stage of business reinvention. A lot of the execution can be done by machines, but there’s still a lot of human involvement. Therefore, human-machine collaboration will usher in rapid development, from Artificial Intelligence AI (Artificial Intelligence) to human Intelligence enhancement IA (Intelligence Augmented).


So far, let’s try to define data intelligence: Intelligent data is the data as the means of production, through a combination of mass data processing, data mining, machine learning, human computer interaction and visualization technology, derived from a large amount of data, discover, access to knowledge, for people in the decision-making to provide effective data intelligence support, reduce or eliminate the uncertainty.


The history of big data

Data intelligence first needs to be provided by data, and data plays the role of core assets and means of production, so data governance is particularly important. What is Data Governance? We often hear the word corporate governance. In economics, corporate governance mainly solves the following problems:

How are ownership and management separated?

How does company owner conduct scientific authorization and supervision to professional manager?

Similarly, data governance should address several similar issues:

What are the data (assets)?

How to separate data ownership from usage?

How can data asset owners scientifically authorize and supervise data users?


Data intelligence is all about solving these problems. This aspect of data governance will be described in detail in Part 2 of this series.


At the same time, we know that the difference between the poor and the rich lies in their attitude towards wealth. The rich treat wealth more from the perspective of asset appreciation, thinking about how to create more assets and keep them appreciating. The poor tend to view wealth from the perspective of consumption, and earn more money for consumption. Intelligent era in the data, if we want to be a “rich”, you need to consider how to make data exert greater value, how to find other partners to jointly create value, but the data is different from other assets, it has to the nature of the replication, difficult to approval, this needs us to solve the problem of data security, This is the industry’s current focus on secure computing, which I will elaborate on in Part 3 of this series.


Another point we need to pay attention to is that because of the 4V characteristics of big data, especially the large quantity and variety, we sometimes have doubts about its aggregation or results. Although some of them can be judged by common sense or intuition, they are always indescribable. This requires a quality assurance system, which will be described in detail in Part 4 of this series, that allows a complete inspection of data from generation to completion.


To sum up, the technical system of data intelligence should include at least three aspects:

Data governance system

Data quality assurance system

Data security computing system


conclusion

As an important and exciting stage in the era of big data, data intelligence has both opportunities and challenges. As the first article in this series, this article gives a general overview of the topic content, and the specific content will be gradually expanded in the future, hoping to help you.


One day


The article was conceived on July 24, 2019, and suddenly found this number quite appropriate. 7*24 is the attitude and commitment of service in many industries, which means providing service 24 hours a day a week. In the era of data intelligence, Our products and services must be online 24/7!


We have been deeply engaged in the field of developer services. Based on message push, we have developed a series of products for APP development and operation, such as “user portrait”, “application statistics” and “one-click authentication”, to build a new ecosystem for developers. At the same time, INDIVIDUAL push continues to broaden the service boundary with data intelligence as the core, and provides customized big data solutions for various vertical fields such as mobile Internet, brand marketing, financial risk control, smart city and public service with innovative technologies. In the future, Individual push hopes to use the power of data and technology with more industries to build a data intelligent win-win ecology!


【ArchSummit 】 Push big data financial risk control algorithm practice

Optimization of Spark Streaming — from Receiver to Direct mode

On the implementation path of IPv4 to IPv6 evolution

Based on Zipkin’s distributed link tracking practice