The author is Pomelo. LCW Ant Financial · Data Experience technology team

00. Preface

Today, we introduce graph analysis and graph visualization, opportunities and Challenges. On the front end, visualization is a little-understood area, and with the concept of a Graph Graph, maybe even less. Today, we will start with a Gartner technology curve and introduce Graph Analysis, an emerging technology, from the height of the industry. Then we will focus on the concept of Graph, Graph Analysis and Graph visualization. From the perspective of business, we will consider the product design and user experience of Graph Analysis. I’m trying to give you a technical RoadMap. Finally, allow me to add an article about the AntV team’s graph visualization engine, G6, and the Graphin graph analysis component, which will be open source on November 22 of this year. Throughout the introduction, we can clearly see both opportunities and challenges in the field of graph analysis and graph visualization.

01.Gartner Curve: Emerging technology maturity in 2019

Gartner’s Emerging Technology Maturity 2019, released on August 6, 2019, divides each technology into Innovation triggers based on The Hype Cycle. Peak of expectation, the coolness period of disillusion (Disillusionment), Slope of Enlightenment, There are five stages of Plateau of Productivity, which comprehensively evaluate the application status and development prospects of various technologies, and predict when they will reach maturity and stability. Each year, the Gartner curve is released to the attention of the market and has become a barometer for major investment decisions in the industry.

In this Graph, we can see two technologies that contain the word Graph. There is no doubt that Knowledge Graph is the hottest application in enterprises at present, because it is at the junction of innovation trigger period and expectation expansion period.

Graph Anlytics, the subject of today’s article, sits at the intersection of inflated expectations and disillusioned cooling-off periods. This is where our title “Challenges and Opportunities” comes from. Opportunity lies in the early public attention and capital to deduce a series of looks is going to be successful, the public is expected to peak of the technology innovation, at the same time the challenge is in the industry that there might be a failed to hit, a large number of experiments deployment failures, public interest is abate, developers and entrepreneurs to waiver or ended in failure.

In an era of rapid innovation and rapid iteration, where new technologies can be quickly embraced or dismissed, the industry needs not only technological innovation but also the value that actually generates for the business. This is both an opportunity and a challenge.

02. Industry opportunities and challenges

In the chart analysis industry, we selected three most representative companies and products to discuss. When it comes to analytics, Palantir is in a league of its own. If you look at their customers, you can see that. In visualization technology, Cambridge Intelligence provides us with a great solution, but unfortunately they don’t work with us in China. In the field of graph database, the competition is extremely fierce, and Neo4J is the leader among them.

Palantir: The realm of analytics

One of the biggest names in graph analytics is Palantir, which was founded in 2004 and has been called America’s most secretive startup. Its clients include the NATIONAL Security Agency (NSA), the FEDERAL Bureau of Investigation (FBI), the CENTRAL Intelligence Agency (CIA), and many other U.S. counterterrorism and military agencies. Played an important role in the intelligence analysis that led to the killing of Osama bin Laden. For more information about the company, you can go to the official website. What exactly does Palantir Technologies do? There are also lively discussions.

Cambridge Intelligence: Visualization technology

And then there’s Cambridge Intelligence, whose slogan is “Bring your Connected Data to Life,” and they have two graph-analysis tools, Keylines (a graphics visualization library for Javascript developers) and ReGraph, a component library for graphics visualization for React developers. Customers include: CISCO, Microsoft, VISA and other large companies, among which the most classic case is the cooperation with Microsoft to help the British government to build Police Management center, see KeyLines Chosen for New Police Contact Management Platform.

03.Neo4j: Graph database

There are also application scenarios for graph analysis in many graph database domains. For example,Neo4j, whose products include Neo4jDB,Neo4j Desktop, and its application fields include Fraud Detection,Knowledge Graph, See Graph Database Use Cases on the official website for more details, which includes a built-in Graph analysis platform. TigerGraph is also very competitive

04. Core challenge: data privacy protection

All of these companies use graph technology as the core technology of their products. Palantir focuses on government, military and medical cooperation. These data are largely endorsed by the government and allowed to be analyzed. Cambridge Intelligence directly do general graph analysis technology, avoid data problems. Neo4J chose more abstract data sources and stored them in the database source layer for analysis. Without exception, we find that these companies are avoiding one big challenge: “data privacy and protection.” In this rapidly developing information society, data privacy protection has become a challenge for all mankind:

  • As we rely more and more on information technology, such as the Internet and cloud storage, the risk of our private data being shared and transferred is getting higher and higher
  • As we demand more and more from our communication experiences, such as 5G communications and IOT devices, the risk of our private data being discovered and collected is getting higher and higher.

The essence of graph analysis technology is to connect the isolated points of data into lines to form a network, so as to realize network abstraction, data analysis, and value output

03. Conceptual understanding

From the perspective of the industry, we may still be confused, what is graph, what is graph analysis, what is graph visualization, what is graph application? Ok, we are going to explain each of these concepts

01. What is a picture?

What do you think of the graph? Images? The graphics? Or graph structure? Graph is the Graph in Graph theory. In 1736, Euler solved the Konigsberg seven Bridges problem. When solving the problem, he created a new branch of mathematics — Graph theory and geometric topology.

In discrete data, graph G is composed of two sets V and E (denoted as G = (V, E)) : V is vertex, the set of nodes, and E is the set of edges. Graph is a useful tool for us to describe the relationship between discrete subjects in the real world.

For academic understanding, please refer to the links I gave you. From the perspective of engineering applications, we refer to the book “Graph Analysis and Graph Visualization — Finding Business Value in Linked Data”, which will not be explained too much.

02. What is graph analysis?

Graph Analysis: There are a lot of wikis on the Internet, but my intuitive understanding is that “use Graph to analyze a problem”. What can Graph analyze? Graph Theory can tell you the answer. The Konigsberg Seven Bridge problem mentioned above is to analyze the problem by using the topological structure of Graph.

In life applications, graph analysis is everywhere. Imagine going out the door: Turn on your mobile phone and check the subway map. First, find vertex stations for your starting point and destination. Then, the navigation will automatically map out the shortest path (a-star-algorithm), and the path you want to take is the Edge. Before you know it, you’re already analyzing graphs.

In addition to graph theory, graph analysis has two solutions in engineering:

  • Background solution: according to data processing, algorithm calculation, AI model training and other ways to analyze a graph structure
  • Front desk solution: visualization display of graph information, exploration and guidance of graph analysis.

The foreground solution, on the other hand, is an alias for our next topic: graph visualization

03. What is graph visualization

As for graph visualization, I am also a semi-monk. Here I recommend the article “Hello World Graph Visualization” written by Shigo from AntV team. I try to supplement it from definitions, advantages, and difficulties.

Definition 01.

The graph data through computer graphics and image processing technology, into graphics or images, complete information display, interaction and other functions.

Advantages of 02.

  • Our brains are very good at processing visual information and visualization is very intuitive.
  • The graph can be described as a link structure of nodes and edges to help users find more value in the network

The difficulties 03.

  • Drawing technique: The input may be a graph data without geometry, and the output should be a graph already drawn.
  • Layout techniques: layout algorithms should be easy to use and interpretable.
  • Interactive technology: user operation interaction, guided exploration interaction, data update interaction.

04. Core challenges: Technical difficulties

  • Difficulties of front-end technology: drawing, layout, interaction
  • Difficulties of back-end technology: algorithm, analysis, processing, retrieval

04. Product design and user experience

After understanding the relevant graph concepts, we began to enter the stage of product research, design and user experience polishing.

01. Product research

  • Figure analyzing commercial products
    • Foreign countries: Palantir, Neo4j, TigerGraph, Cambridge-Intelligence
    • Domestic: Eye inspection, wind report, prism, tongshield
  • Front-end visualization: Product demo
    • Infranodus, Linkurio, Gephi

02. Product positioning

When we investigated so many competing products, we found that there were two main categories

  • Platform diagram analysis of product design
  • Vertical charts analyze product designs

Vertical as long as the grasp of a vertical field, such as anti-money laundering, it is done deeply. The platform type requires a deeper understanding and design of “graph visualization/graph computation”, which is a challenge to improve usability while ensuring universality.

So when we start to design, we have to think clearly, what is the positioning of our product.

03. Product design

If I were to explain, in one sentence, why we do product design, my answer would be

Understand and deliver what users want to help them better analyze their data

Taking the platform product as an example, centering on the goal of “better helping users analyze data”, our product features are disassembled as follows:

01. Information display module

  • Interactive scheme: click /Hover/ right-click menu: view node and edge attribute information and operation information
  • Layout scheme: Use different layout schemes to show users the information they care most about
  • Filtering scheme: help users reduce data interference, direct to the data he needs

02. Auxiliary analysis components

  • Toolbar General purpose Toolbar
  • ContextMenu Right-click menu
  • MiniMap thumbnail
  • PropertiesFilter Property filter
  • Legend Legend

03. Basic analysis module

  • Node Expand: Use the risk aggregation principle to Expand from one risk Node to find other risk nodes.
  • Find Connections: According to the six-degree partition theory, discover relationships that might be hidden.
  • Group Analysis: It is difficult to find gang crimes, and the methods are complex and changeable, so it needs to be focused on

04. Advanced analysis module

  • Geographical Analysis
  • Time-based Analysis
  • Group Navigation Analysis

04. Experience design

01. Model foundation

Our Model is based on the Graph Model, a data Model of graphs. The first step in the design is to visually reflect the model

  • Nodes: physical Nodes that users care about
  • Edges: The relationship between entities
  • Property: Secondary information or descriptive characteristics of nodes and edges

02. Basic programmes

  • Color: The color should help the user distinguish the information, the number of colors should not be too much to disturb the user
  • Graphics: Graphics should meet layout needs, be extensible, and can carry key information
  • Interaction: Helps users complete the analysis process and guides users to next operations

03. Design principles

  • Design for user intuition: If users believe that graphical visualization is an accurate representation of data, their experience will be more insightful.
  • Experience consistency: Ensure that the entire visual design of the relationship has the same user experience, from color to interaction.
  • Traceability: In order for the user to trust the diagram presented, the user needs to understand how it was generated. For example, it may be tempting to run filtering, computation, and layout entirely on the back end, but displaying these processes gives users more trust. Animation is a good way to do that.
  • Reversible: Don’t make your users worry that an unexpected click will ruin an afternoon’s work. It is necessary to support undo/redo operations

05. Technology RoadMap

  • Introduction to data visualization produced by AntV Ink Institute
  • Graph Theory explains some of the theories of graphs via d3.js, which you can quickly browse
  • Data Visualization Course 2018 You can take a closer look at D3.js if you want to use it
  • Front-end underlying technology MDN SVG, Canvas family bucket
  • Graphic visualization technology framework: you take your own ha
    • D3 88.1 k star
    • Sigma. Js 8.7 k star
    • Cytoscape 6.1 k star,
    • G6 4.8 k star
    • Vis.js 8k star not maintained
    • Viz.js3.2 k starread-only

06. G6 & Graphin

01. Graph visualization engine: G6

Introduce 01.

G6 is the graph analysis and graph visualization engine of Ant Financial -AntV.

02. Origin of name

  • The G of G6 comes from Graphic and Graph, which means we need to make Graph visualization based on Graph analysis technology.
  • The six of G6 comes from The Six Degrees of Separation Theory, which expresses our awe and fascination with relational data and relational networks.

Think 03.

Since the birth of G6, the business undertaken by G6 can be divided into two categories, one is graph analysis, the other is graph editing. Although the two are similar to some extent, their emphasis is different. The former pays more attention to the analysis experience of graph data. The latter focuses more on interactive editing. We used the G6 to support these two different types of businesses at the same time, and in practice, it was not as good as we had hoped. Starting with G6 version 3.1, we focused more on the graph analysis area.

02. Graph visualization component: Graphin

01. About introduction

Graphin is a G6 packaged React component library focused on graph analysis.

Graphin means “Graph Insight” for analysis, Graphin contains potential for the future.

  • The bottom layer uses G6 as the rendering, interaction, and layout engine for diagrams.
  • React is used for componentized development in the outer layer to improve developer efficiency.
  • Superior precipitation polymerization industry map analysis of product features.

As can be seen from the above, Graphin’s G6 bottom layer gives us the ability of graph visualization, while the React outer layer encapsulation allows us to improve development efficiency. The product functions of the upper layer accumulate, making our analysis more professional and brilliant

02. Functions and features

Graphin is still in development. The Gif below gives you a glimpse of its features. Some of its features are inherited from G6 and some from the React framework. The core 6 functions are as follows.

  • 1.Data Driven
  • Automatic layouts
  • 3.Analysis Components
  • A: I think we need to get some customization done
  • 5.Basic Analysis
  • 6.Advanced Analysis

Open source time

On the early morning of November 22, 2019

I was asked, “What are your expectations for Graphin once it’s open source?” I said there were only four words: “throw bricks to attract jade.” We have investigated many excellent graph analysis products, and many of them have been tried out and received emails saying “Sorry, we are not doing business in China for the time being”. Every time, I guess its product logic from the video demonstration of fragments, and sometimes wonder why foreign products are so detailed.

However, in China, many graph analysis products focus on back-end solutions, and front-end design experience and graph visualization analysis are obviously insufficient. We hope that when other developers come into contact with graph visualization, they can see Graphin in their research. It is a brick, waiting for more jade to appear. As its logo shows, a graphene shape contains huge energy. In the future ERA of 5G+IOT, it is expected to help people find greater value in the field of data analysis by using relationship analysis methods.

7. To summarize

At the end, I hope to echo the challenges and opportunities of the title, graph analysis and graph visualization. From the perspective of product design, how to understand and deliver what users want is a challenge. From the perspective of front-end technology, rendering performance, layout and interaction are all great challenges. Opportunities are accompanied by challenges, and challenges contain opportunities.

We are just a learner and a follower in this field, and Graph Analysis is currently at the intersection of inflated expectations and disillusioned cooling-off periods. Most of the academic papers we collect are 20 years old. Seeing the mature commercial products, which were born almost 10 years ago, many framework libraries for graph visualization are also iteratively slowing down. Are they disillusioned with inflated expectations, or are they going ahead in the rush and constantly exploring in this field to find the business opportunities brought by technology? I want us to work together.

reference

  • Graph Visualization and Navigation in Information Visualization: A Survey
  • What is the Difference between UI and UX?
  • Graph Visualization: How to Avoid Wrecking Your Graph Visualization
  • What exactly does Palantir Technologies do?
  • Graph Analysis and Graph Visualization — Finding Business Value in Linked Data

If you are interested in our team, you can follow our column, follow Github or send your resume to ‘tao.qit####alibaba-inc.com’.replace(‘####’, ‘@’)

Original address: github.com/ProtoTeam/b…