CIKM is the premier international academic conference in the field of information retrieval, knowledge management and databases. Since 1992, CIKM has successfully brought together leading researchers and developers in these three fields, providing an international forum for exchanging the latest developments in information and knowledge management research, data and knowledge base. The purpose of the conference is to identify future challenges and problems in the development of knowledge and information systems, and determine future research directions by collecting and evaluating top applied and theoretical research results.
This year’s CIKM was originally scheduled to be held in Galway, Ireland, in October, but was moved to online due to the pandemic. A total of six papers (4 long papers and 2 short papers) from Meituan AI Platform/Search and NLP Section /NLP Center/Knowledge Graph Group were accepted by the international conference CIKM 2020.
These papers are the results of cooperation between Meituan Knowledge Atlas and Xi ‘an Jiaotong University, University of Chinese Academy of Sciences, University of Electronic Science and Technology of China, Renmin University of China, Xidian University and Nanyang Technological University. It is the technology precipitation and application of multi-mode knowledge Graph, MT-Bert, Graph Embedding and interpretability of Graph. I hope these papers can help more students to learn and grow.
01 query-Aware Tip Generation for Vertical Search
| this paper Meituan knowledge map and xi ‘an jiaotong university hao handsome classmate Li Canjia, Chinese Academy of Sciences university classmate, xi ‘an university of electronic science and technology paper Wang Zili students cooperation.
Paper download
Interpretability reason (and why) were found in the search results page and page (scenario decision, will eat list, etc.) to show users highlight recommended a natural language text, can be regarded as a real user comments highly concentrated, explain the recall results for the user, mining business characteristics, to attract users to click on, and the scene is changed to the user guide, Help users make decisions to optimize the user experience in vertical search scenarios.
Most of the existing text generation work does not consider the user’s intention information, which limits the implementation of generative recommendation reasons in scenariographic search. In this paper, a query-aware recommendation reason generation framework is proposed. The user Query information is embedded in the encoding and decoding process of the generation model, and personalized recommendation reasons adapted to different scenarios are automatically generated according to the user Query. In this paper, Transformer and recursive neural network (RNN) are modified respectively. Based on Transformer structure, this paper introduces Query information by improving the self-attention mechanism, including the introduction of Query-aware Review Encoder to make the Query related information be considered at the initial stage of comment coding. The query-Aware Tip Decoder is introduced on the Decoder side to take query-related information into account at the last stage of comment encoding. Based on RNN structure, Query irrelevant information is filtered out in Selective Gate mode at Encoder end, Query related information in original comments is selected for encoding, and Query representation vector is added into Context vector calculation of Attention mechanism at decoder end. To guide the decoding process, the problem that the generation method decoding is not controllable is solved to some extent, so as to generate the recommendation reason for Query individuation.
Experiments on open data sets and Meituan business data sets show that the proposed method is superior to existing methods. The algorithm proposed in this paper has been applied online, and has been implemented in meituan’s search, recommendation, category screening and list.
02 “TABLE: A Task-Adaptive Bert-based ListwisE Model for Document Retrieval”
| this paper Meituan knowledge atlas group and the institute of software, Chinese Academy of Sciences paper Tang Hongyin classmate, bei-hong jin teacher cooperation.
Paper download
In recent years, more and more MRC and QA datasets have emerged in order to improve the natural language understanding of models. However, these data sets are more or less flawed, such as insufficient data volume, reliance on manual construction of Queries, and so on. To address these problems, Microsoft proposes a Reading Comprehension dataset MS MARCO (Microsoft Machine Reading Comprehension) based on large-scale real-world data. The dataset is based on real search queries in the Bing search engine and Cortana intelligent assistant, and contains 1 million queries, 8 million documents, and 180,000 human-edited answers.
Based on MS MARCO data set, Microsoft proposed two different tasks: one is given a problem, retrieve all the documents in the data set and sort them, which belongs to the document retrieval and sorting task; The other is to generate an answer based on a question and given relevant documentation, which is part of the QA task. In Meituan business, document retrieval and sorting algorithms are widely used in search, advertising, recommendation and other scenarios. In addition, the time consumption of QA tasks directly on all candidate documents is unacceptable. QA tasks must rely on sorting tasks to filter out the top documents, and the performance of the sorting algorithm directly affects the performance of QA tasks. For these reasons, we focused on document retrieval and sorting tasks based on MS MARCO.
Since its launch in October 2018, MACRO has attracted the participation of many enterprises and universities, including Alibaba Dharma Institute, Facebook, Microsoft, Carnegie Mellon University, Tsinghua University and so on. On the pre-trained MT-BERT platform of Meituan, we propose a BERT algorithm for this text retrieval task called TABLE. It is worth noting that the TABLE model proposed in this paper is the first model to exceed 0.4% on the MARCO list of Microsoft, the authoritative evaluation in the field of information retrieval.
As shown in the figure above, this paper proposes a bert-based document retrieval model TABLE. In the pre-training stage of TABLE, a domain adaptive strategy is used. In the fine-tuning stage, this paper proposes a two-stage task adaptive training process, that is, the Pointwise fine-tuning and the List fine-tuning. Experiments show that this task adaptive process makes the model more robust. This work can explore richer matching features between queries and documents. Therefore, this paper significantly improves the effectiveness of BERT in document retrieval tasks. Then on the basis of TABLE, we proposed two methods to solve OOV (Out of Vocabulary) mismatching: precise matching method and word reduction mechanism, which further improved the effect of the model. We called this improved model DR-Bert. Details of DR-Bert can be found in our technology blog, MT-Bert in Practice in text retrieval tasks.
03 “Multi-Modal Knowledge Graphs for Recommender Systems”
| this paper Meituan knowledge atlas group and the institute of software, Chinese Academy of Sciences paper Tang Hongyin classmate, Jin Beihong teacher cooperation.
Paper download
With the development of knowledge graph technology, its structured data has been successfully applied in a series of downstream applications. In the direction of recommendation system, structured atlas data can make use of more comprehensive auxiliary information of the target commodity to carry out information dissemination through atlas association, so as to effectively model the representation of the target commodity and alleviate problems such as sparse user behavior and cold start in the recommendation system. In recent years, many studies have successfully combined atlas data with recommendation system by means of map path feature and graph embedding based representation learning, improving the accuracy of recommendation system.
In the existing work of combining the map and recommendation system, people often only focus on the map nodes and node relations, but do not use the data of each mode in the multi-modal knowledge map for modeling. Multimodal data includes image modes such as movie stills and text modes such as business reviews. These multimodal data can also be propagated and generalized through knowledge graph graph relations, and bring high-value information to downstream recommendation systems. However, because multi-modal knowledge modeling is usually the auxiliary information relationship of different modes, rather than the semantic association relationship represented by triples in traditional maps, the traditional map modeling method cannot model multi-modal knowledge maps well.
Therefore, MKGAT model is proposed in this paper according to the characteristics of multi-modal knowledge graph, and it is proposed for the first time to use the structured information of multi-modal knowledge graph to improve the prediction accuracy of downstream recommendation system. The overall model framework of MKGAT is shown in the figure below:
In MKGAT model, the embedded representation learning of multimodal maps is mainly divided into three parts: 1) We first use MKG Entity Encoder module to encode different types of input data (images, texts, labels, etc.) into high-order implicit vectors; 2) Next, based on the MKG Attention Layer, we use the nodes around the entity node (including multimodal and entity nodes) to provide corresponding information for the description of the node. 3) After integrating multi-modal information by using attention mechanism, the traditional h+ R = T training method is used to carry out the learning of graph embedding representation.
When accessing the downstream recommendation system model, we also reused the multi-modal entity coding and multi-modal graph attentional mechanism module to represent the target entity and connect it to the recommendation system model. Through the above methods, we conducted detailed experiments on two real data sets, meituan food search scene and MovieLens public data set, and the results showed that MKGAT significantly improved the quality of the recommendation system in these two scenes.
S^3-Rec: Self-supervised Learning for Sequential Recommendation with Mutual Information Maximization
| this paper Meituan Zhou Kun group and the Chinese people’s university students of knowledge map, maike classmate, zhu Yu Tao classmate, Zhao Xin paper, ji-rong wen teacher cooperation.
Paper download
Sequence recommendation refers to the product that predicts the future interaction of users by using the long-term interaction history sequence of users, and enhances the accuracy of recommendation to users by modeling sequence information. Existing sequence recommendation models use commodity prediction to train model parameters, but they are also limited to a unique training task, which is easily affected by data sparsity problem. Although it optimizes the final recommendation goal, it does not adequately model the underlying relationships in the context data, let alone use this partial information to help the sequential recommendation model.
To solve the above problems, this paper proposes a new model S^ 3-REC, which is based on self-attention network structure and adopts self-supervised learning strategy for representation learning, so as to optimize serialized recommendation tasks. The model is based on four special self-monitoring tasks that learn about potential relationships among attributes, goods, self-sequences, and primitive sequences. Since the above four kinds of information represent four different information granularity perspectives of input data, this paper adopts the mutual information maximization strategy to model the potential relations of these four kinds of information, and then strengthens the representation of such data. In this paper, a large number of experiments are carried out on six real data sets, including Meituan scenario, to prove the superiority of the proposed method over the existing advanced sequence recommendation method, and the model can still maintain good performance in limited training data scenarios.
05 Leveraging Historical Interaction Data for Improving Conversational Recommender System
| this paper Meituan Zhou Kun group and the Chinese people’s university students of knowledge map, maike schoolmate, Zhao Xin paper, ji-rong wen teacher cooperation.
Paper download
In recent years, conversational recommendation system has become an important research direction, and it also has many applications in real life. A session recommendation system needs to be able to understand the user’s intention through the dialogue with the user, and then give the appropriate recommendation, so it includes a session module and a recommendation module. The existing session recommendation system usually completes the recommendation based on the learned user representation, which needs to encode the conversation content. However, in fact, it is difficult to accurately predict user preference information only by using conversation data. This paper hopes to help complete the recommendation by using the historical interaction sequence of users.
Based on the idea, the session user recommendation system need to be considered at the same time the history of the sequence and the session data, this paper puts forward a new method for preliminary training, through training methods will be based on the merchant preference sequence (data from historical interaction) and based on the attributes of the merchants preference sequence (data) from the dialogue, promoted the session the effect of the recommendation system. To further improve performance, the paper also designs a negative sample generator to produce high quality negative samples to aid training. Experiments are carried out on two real data sets, and the proposed method is proved to be effective in improving the session recommendation system.
Structural Relationship Representation Learning with Graph Embedding for Personalized Product Search
| this paper Meituan knowledge atlas group, high plexus with nanyang technological university, liu is still classmates teacher’s working papers.
Paper download
Personalization is very important in commodity search, and users’ preferences affect their purchase decisions to a great extent. For example, when a young user searches for a “baggy T-shirt” on an e-commerce platform, he is more likely to buy a branded fashion or shirt that interests him. The purpose of personalized product search (PPS) is to generate user-specific product suggestions for a given query, which plays an important role in many e-commerce platforms.
In this work, we use the logical structure representation learned from user-query-item to naturally keep collaborative signals and interactive information between users/queries/items on the logical path to improve personalized product search. We call these logical structures “Conjunctive Graph Pattern”. For example, as shown in Figure 1, there are three key patterns. Note that when a branch has three or more branches, we can randomly sample two of them and get the following pattern:
Specifically, we propose a new method: Graph embedding model based on logical structure representation learning (GraphLSR). The conceptual advantage of GraphLSR is that it is an embede-based framework that effectively learns the representation of logical structures, as well as the approximate relationships of users (queries or items) in geometric operations, and integrates them into personalized product searches. The key idea behind it is that we have learned how to embed three types of link graph patterns into low-dimensional space to enhance personalized product search by embedding graphs. The framework is shown in Figure 2, which consists of two main components: graph embedding module and personalized search module. The graph embedding module at the bottom of Figure 2 makes use of the designed three connected graph modes to learn the embedded nodes for logical representation learning, which is also convenient for learning the similarity between users (queries or goods). Then the presentation information is introduced into the personalized search module.
The personalization search module takes users, queries, products, and representations learned from graph embedding as inputs and integrates the corresponding information using a multi-layer perceptron (MLP). The short and dense features of the extracted users, queries, and goods are input into the MLP network respectively, the user-specific query representation and user-specific goods representation are learned, and then we input them together into another MLP to calculate the probability score of the prediction.
Table 3 compares the performance of GraphLSR and four personalized search methods in MRR, NDCG@10 and Hit@10 in personalized product search tasks:
conclusion
The above are some research work done by the knowledge Graph group of search and NLP department on multi-modal knowledge Graph, MT-bert, graph-embedding and interpretability of the Graph. The results of the paper are also the specific problems we encounter and solve in practical work scenarios. Most of the work has been implemented in practical business scenarios such as content search, product search, recommendation reasons and other projects, and achieved good business benefits. Meituan AI platform/Search and NLP center has been committed to transforming academic achievements into technical productivity through the combination of production and research. We also welcome more ambitious people to join our team.
| want to read more technical articles, please pay close attention to Meituan technical team (meituantech) WeChat public official. In the menu bar of the public account reply [2019 goods], [2018 goods], [2017 goods], [algorithm] and other keywords, you can view the collection of technical articles of Meituan technical team over the years.
