As the launch of the main network is approaching, the knowledge map ecology is also being built quietly. As the current most popular track and field of knowledge map in the new theories and new concept, in order to give you synchronize the knowledge map of the world’s latest cutting edge in the field of dry goods, EpiK program will be open special knowledge sharing, hope everybody can participate together, join our knowledge map content of cutting edge community, jointly contribute strength for knowledge map.
Presentation Learning on the Knowledge Graph of RDF and LPG by Michael Galkin. From the several common ways of knowledge map construction, such as triples, RDF, LPG, etc, based on the knowledge map of triples the standard paradigm of general analysis, points out the shortage of cognition for the entities, the introduction of RDF and LPG to the importance of knowledge map construction, especially in the relationship between the knowledge map construction.
In this paper, we will review the structural paradigm of triplet in the construction of knowledge graph, and also learn new concepts — hypergraph, hyperrelational knowledge graph, the development process of hyperrelational knowledge graph, the whole process of coding and decoding of hyperrelational knowledge graph, etc. The original text (see reading) is translated as follows:
Knowledge graphs (KGs) are the cornerstone of modern natural language processing and artificial intelligence applications — recent achievements include question-and-answer, entity and relational linking, language modeling, information extraction, and even text RPGS played with reinforcement learning. In addition, the knowledge graph has been widely adopted in the industry, for example, a series of outcomes at the recent Knowledge Graph Conference (KGC) :
Triples vs worlds
Traditionally, KGs has been encoded as (RDF) triples, and many publicly available KGs, such as DBpedia and YAGO, initially followed this paradigm and standards supported by expressive logical forms, such as RDF and OWL.
Triple facts. Did Einstein attend two universities at the same time? The picture is from the author.
Using triples, the example above describing Albert Einstein’s college can be coded as two triples:
Albert Einstein was educated at eth Zurich
Albert Einstein, educated at Zurich University
Well, that seems fine for a simple application, but it turns out that our world is a little more complicated to divide everything into triples. For example, do these two triples mean that Albert Einstein was educated in both places at the same time? Or did they give him the same degree?
In fact, Einstein earned a bachelor’s degree in mathematics from eth Zurich and a doctorate in physics from the University of Zurich.
Can we have a mechanism for expressing facts in more detail?
Yes. In the KG world, there are at least two ways to do this — RDF graphs and labeled property graphs (LPGS). Both of these approaches allow further instantiation of each fact by attaching auxiliary key-value (relational entity) pairs to the edges in the KG. Both approaches are already supported by major vendors in the graphical database market.
In the LPG world, both nodes and edges can naturally have key-value attributes. Neo4j is probably the largest name in the LPG family. Users can query LPG with an (open) password. A recent COVID-19 KG can be used as a Neo4j dump.
RDF was originally proposed by Olaf Hartig (whose blog was the starting point for research into RDF and related technologies) to mitigate many of the problems with the notorious RDF reification mechanism (see this survey by Frey et al., for a comprehensive overview of reification), while preserving the reasoning power associated with RDF graphs. Supported by a solid theoretical foundation, RDF offers several ways to enrich triples with more detail. You can query RDF graphs with SPARQL, an extension of SPARQL for RDF. Apache Jena, RDF4J, n3.js, Blazegraph, AnzoGgraph, StarDog, and GraphDB support RDF and SPARQL.
Our example in RDF syntax could look like this:
Hypergraph or hypergraph?
Is that an appropriate term for such a knowledge graph? There was a bit of a vocabulary gap before rosso et al. suggested the use of “hypergraphs” in their recent work. However, there is a common misuse of the term “hypergraph”, so we want to promote “hyperrelational” diagrams as well.
The main difference between the two words lies in the expression of fact. A hypergraph assumes that there is a (named) hyperedge that unifies several entities:
Each node consists of four nodes. Notice that we have lost predicates related to academic majors and degrees. The image is from the author.
It looks like an n-element relationship, right? We have a few questions here:
1. We have lost the typification relationship with academic_degree and academic_major with Bachelor/ graduate and Mathematics/Physics respectively. In contrast, the super-edge types are abstractions (or rather strange semantic mixes) of EDUCated_at, academic_degree, and academic_major. What if a fact also contains an auxiliary predicate academic_supervisor? We need to define a new super-edge, such as education, to mix these relationships, which grow exponentially with the number of predicates and determiners.
2. In addition, we have lost the auxiliary characteristics of degrees and majors, the main triad designed to describe them. Baccalaureate and mathematics, for example, are an aid to Albert Einstein and ETH Zurich and should therefore be treated as such. Entity tuples in super-edges assume that their elements are equally important. That is, we will stick with the hyperrelational approach for the rest of the section.
Hyperrelational knowledge map of savage growth
As of 2020, most open domain KG make extensive use of hyperrelational facts. Wikidata and its wikidata statement model are a good example of a hyperrelational knowledge graph. Each fact in the wiki data is a statement with a primary triple and a set of auxiliary qualifier entity-relationship pairs. With wiki data statements, our Albert Einstein example can be modeled simply as follows:
In these statements, (academic_degree, bachelor’s degree) and (academic_major, mathematics) are qualifiers for the threefold < Albert Einstein, Educated_at, ETH Zurich >. It is important to note that the data (usually a super relationship paradigm) will not be used exclusively for major triad or qualifier between separate entities and predicates, namely all predicates and entity can be used in the three terms or qualifier (though in the current version of wikipedia data, some of the entities and relationships can only be seen in the qualifier). We will use this property in the following sections.
As for the other KG, starting in 2018, a new version of DBpedia contains externalized statements similar to Wiki data. YAGO 4 also takes advantage of the fact that RDF is encoded. What about Freebase? Well, in 2020, you probably shouldn’t practice voodoo because Freebase is no longer supported or updated. However, the compound value type (CVT) node in Freebase does resemble the embodiment of a triple [but looks more like an n-element relationship].
Graph representation learning
Our task here is to learn the representation of a hypergraph. By representation, we refer to entity (node) and relational (typed edge) embeddings. These embeds can be used for downstream tasks such as link prediction, node classification, entity alignment, and more for NLP, RESUMES, and other artificial intelligence domains.
The graph representation learning (GRL) field is one of the fastest growing fields of machine learning, with several articles (series by Michael Brownstein, reviews of ICLR 20 and NeurIPS 19 papers (mine, Sergei’s), books (William Hamilton’s, Ma and Don’s), courses (CS224W, COMP 766, ESE 680) and there is even a GraphML telegraph channel covering basic and advanced topics. In the encoder-decoder paradigm, encoders are usually GNN (graph neural network), and decoders are embedded functions that return values or vectors associated with a particular downstream task, for example, the probability of an entity becoming an object of a given pair.
What is the knowledge graph based on ternary?
Encoders: A series of multi-relational GNN encoders, such as R-GCN (Schlichtkrull et al., ESWC 2018) and CompGCN (Vashishth et al., ICLR 2020), which extend the original graph convolutional network (GCN) algorithm within a messaging framework.
Decoder: In fact, traditional KG embedding algorithms, such as TransE, ConvE, RotatE, etc., are the best examples of a link prediction task decoder. Initially, they can also be trained as decoder-only models for direct 👉 link prediction tasks for end-to-end optimization.
What is a hyperrelational knowledge graph?
Well, not so much. (As of Autumn 2020)
Encoder:??
Decoder: HINGE, proposed by Rosso et al., is a CNN-based end-to-end model for link prediction on hypergraphs. Well, we can’t cope with such an obvious abyss in the GNN encoder section 🕳, and in our recent EMNLP 20 paper, “Messaging in Hyperrelational Knowledge Graphs”, presented StarE, The paper is co-authored by Priyansh Trivedi, Gaurav Mahehwari, Ricardo Usbeck and Jens Lehmann. StarE is a multi-relational GNN encoder that extends CompGCN to superrelational KG. The name was inspired by RDF StarE’s design with the following features in mind:
Explicit modeling of relations, including qualified relations;
The separation of the auxiliary entities and relations in the qualifier from the entities and relations in the principal triplet;
Nevertheless, any entity and any relation can still be used for primary triples as well as determiners;
Constancy of determiner order – they do not have any particular order and can be rearranged freely. That is, it does not matter whether (degree, bachelor’s degree) came before or after (academic specialty, physics) the main triad of Albert Einstein’s education at ETH Zurich.
Some math problems for math lovers
Let us trace the evolution of relational-aware GNN encoders in their neighborhood aggregation schemes:
In StarE, the main triplet relationship h_r that appears between nodes U and V is enhanced by a qualifier aggregation vector of the function gammah_q () which can be a weighted sum, multiplication, conat, or any other binary function (we chose a weighted sum). We get the vector hq:
That is, we first aggregate the qualifier relationship and the entity embedding h_{qr} and h_{qv} in a vector by combining functions, which may be the scoring functions of the KG embedding family, such as RotatE. We then apply the permutation invariant aggregation function to aggregate any number of qualifiers into a vector, and finally project it to W_q through a transformation matrix. Because all entities and relationships are typically seen in primary triples and qualifiers, W_q aims to learn qualifier specific representations of entities and relationships. We still have the CompGCN component: phi_ () is a composite function similar to phi_q, but now it incorporates a node with a rich edge representation. W_{\lambda} is the weight parameter of the afferent, efferent, and self-cyclic relationships.
Sparse coding of hyperrelational knowledge graph
To improve efficiency, GNN operates on the sparse matrix.
You can render it as the [2, num_edges] tensor in COO format, and add a line to the edge type
Hyperrelational facts with qualifiers can be written as follows:
The first three entries always represent “primary” triples, followed by pairs that are qualifiers in no particular order (remember the order invariance in the Wiki data). What is a sparse representation of a superrelation KG, where each “column” of the COO matrix may have any number of qualifiers? In this paper, we propose the following codes:
In other words, we have two COO matrices:
Ordinary “triple” COO with implicit column index K
A “qualifier” COO of shape [3, num_qualifiers], where the first row contains the index of the columns in the “triple” COO and the second row contains the qualifier relationship and the third qualifier entity. The index row has a list of qualifiers connected to the main triple. That is, the columns in the “qualifier” COO that share the same index K belong to the KTH triplet in the “triple” COO matrix. This allows us to in memory of the number of qualifiers in KG for O (q), and the total memory for O (edge | | + | qualifier |). ️
We need to talk more about data sets
We talked briefly about encoding hyperrelational facts as sequences of entities and relationships. But is there a reliable data set for conducting experiments on such a KG? Traditionally, KG embedding has been evaluated on link prediction tasks, while graph ML tasks include node classification, graph classification, entity matching, and so on. So far, only two link prediction datasets exist: Kwan et al. ‘s proposed Wikemen, which is a dump of wiki data describing people, and JF17K is an outlet for Freebase. However, we found major drawbacks:
WikiPeople has too many qualifiers with text (year). It is not recommended to treat text as another entity, as numbers are continuous values and should be treated as such (well, this is a common problem with KG embedded in text in the literature 🤔). That is, in most cases, the qualifier will be removed. This results in only 2% of the facts in the dataset having a qualifier, and 80% of the facts having only one qualifier pair: /.
JF17K has test set leaks. In fact, the authors themselves found “a large number of redundant triples” and do not recommend their use in experiments. HINGE was originally more of an N-metadata set, which was transformed into a hyperrelational format with auxiliary predicates. We conducted further research and found that over 40% of the test statements shared the same (S, R, O) primary triplet as the training set. That is, in the subject/object prediction task, a simple triple heuristic can outperform all the previous hyperrelational methods we have shown in the paper.
Since neither dataset is suitable for evaluating hyperrelational methods, we sampled WD50K from the Wiki data according to the following guidelines:
Preserve a qualifier distribution similar to wiki data. In Vanilla WD50K, about 13% of statements were qualified (close to 17% of all statements in the Wiki data);
All qualifiers are entity-relational pairs, no text;
Entities and relationships can be seen in primary triples and qualifiers;
99% of statements have at most 6 qualifier pairs For further experiments we sampled 3 additional data sets:
WD50K (33) – about 33% of statements have qualifiers;
WD50K (66) – about 66% of statements have qualifiers;
WD50K (100) – All statements have qualifiers Naturally, these data sets are smaller than the original WD50K, with more qualifiers unique entities and relationships.
Link to StarE in prediction
At this stage, we finally have a StarE encoder and a link prediction dataset suitable for experiment. Our main research question is: Can determiners help predict the subject and object of hyperrelational facts?
StarE+ decoder for link prediction. The image is from the author.
That is, given a subject, a predicate, and all determiners, we predict the subject’s object, and vice versa. To do this, we linearize a given fact into a sequence, as shown in the figure, and use a 2-layer transformer with an average pool and a final FC layer as the decoder. Transformers also allow us to feed sequences of varying lengths using padding tags that are shielded from self-attention calculations. For comparison, we applied the decoder-only HINGE and 2-layer transformer in the same task to measure whether the StarE encoder provided any benefit. And it turns out it is!
We observed:
StarE significantly improves link prediction performance compared to the decoder only approach;
StarE was even more effective (with a larger performance gap) when there were more qualifiers in the dataset;
The hyperrelational approach really helps to better predict topics and objects, because qualifiers only have three baselines. How many qualifiers do you need to see an improvement in quality? Only two would be enough
Our experimental plans with specific numbers and interactive charts are reported here with weights and deviations, so the summary for the KG community is:
Seek to assign descriptive qualifiers to more triple facts in the graph — the more the better;
If you assign qualifiers – add 2 or more!
Conclusions and Resources
Hyperdiagrams are closer to reality and describe facts in more detail than ordinary triples
RDF and LPG provide a means to build a hyperrelational knowledge graph
Hypergraphs are different from hypergraphs
Hyperrelational knowledge maps are already in use — both in open domain knowledge maps and in industry
RDF* powered StarE- GNN encoder for superrelation KG that can be paired with decoders for downstream tasks
StarE improves link prediction compared to the decoder only approach
The WD50K dataset series better captures the challenge of link prediction over the superrelation KG