Participation in the award: This article has participated in the "Newcomer Creation Ceremony" activity, together with the start of the road of gold digging creationCopy the code
Thesis Title:Predictive and Contrastive: Dual-Auxiliary Learning for Recommendation [1]
Source: ARXIV2022
A, Motivation

The rich semantics reflected by social relationships and item categories, which lie in the recommendation data-based heterogeneous graphs, are not fully exploited.

There is rich semantic information in social connections and item types, but this information hidden in heterogeneous graphs based on data recommendation has not been fully explored.

The authors first quantitatively analyze the heterogeneous interaction data and find a strong positive correlation between the interactions and the number of user-item paths induced by meta-paths.

For the first time, we quantify interaction data in heterogeneous graphs and find a strong positive correlation between interaction records and the number of user-item meta-paths.

In the figure above, the LastFM data set in the music field is used. UUA represents a meta-path of user-user-artist [2], UAUA represents the meta-path of user-artist-user-artist [2], and the abscissa represents link-score. The ordinate represents the likelihood that the user will interact with the last element of the meta-path [2]. As can be seen from the figure, when the link-score is larger, the user is more likely to interact with the last Artist [2] in this meta path.

As shown in Figure (a), user U1-U4 and commodity M1-M3 form a heterogeneous graph relationship. Meta path [2] is illustrated by examples. For example, the meta path in the form of UUM includes U1-U2-M1, U2-U3-m2, etc. Then the communication matrix [2] is introduced, for example:

There are two paths from U2 through the meta path UUM to M1, so there are:

Others are similar.

On this basis, it is proposed that after standardizing the above results, link-score can be obtained, that is, divide all the values by the maximum value to obtain the following link-score.

Second, the Model

The framework of the model diagram refers to the framework of CGC[3], as shown in the figure below:

The model style is still a typical two-tower structure. After the input data, it goes through three encoders, among which pink and green are two different encoders, the blue part in the middle is shared by both sides, and G is a gated unit. Finally, it goes through two different towers to get different outputs.

The model diagram proposed in this paper is as follows:

The model graph adopts the framework of CGC[3], with four input parts in total. GUI is the original user-commodity bipartite graph. ET and ES are two randomly initialized Embedding, which makes the input ambiguous. The blue part in the middle uses the same LightGCN, which is equivalent to using three LightGCN[4] as GNN encoder. Then the obtained representation is spliced, and then the representation obtained after a single layer FFN plus softmax is compared to learn. The paper points out that if there is no meta-path, the link-score is 0, and it is taken as a negative example, while the link-score with the largest link edge is 1. Take it as a sample for comparative study. At the same time, the author also used the outputs of both sides to calculate recommended Loss and predicted Loss respectively. The recommended Loss refers to a paper published by C. Chen et al. 2020 [5], eliminating the process of sampling. The three losses are as follows:

Here, the c+vWith c–vThese are all user-set parameters.

Direct use of mean square error calculations to predict losses.

Use InfoNCE directly to calculate the contrast loss.

Data & Experments

The dataset uses three public datasets, LastFM[6], Yelp[7]Douban, Book[8].

Four, the Performance of

In terms of performance results, DuAL has achieved the effect of SOTA in all three data sets.

Fifth, Ablaton Study

In the ablation experiment, no prediction and comparison task was used, and the experimental results showed that the prediction and comparison module had a great influence on the model.

At the same time, the author also made the use of different models as GNN encoder effect, can see the use of LightGCN effect when the best.

At the same time, the author also did experiments using different meta-paths, and it can be seen that the choice of meta-paths has a great influence on the experimental results.

Six, the Conclusion

By capturing the hidden information under the meta path, this paper proposes DuAL recommendation model for DuAL auxiliary tasks. The biggest highlight of this model lies in two aspects: the first is to form a comparison and forecast task through the meta path; the second is that the recommendation task does not need sampling. However, the disadvantage of the model is also obvious, that is, how to choose the optimal meta path, which has a great influence on the model.

Seven, the References

[1] Tao Y, Gao M, Yu J, et al. Predictive and Contrastive: [J]. ArXiv Preprint arXiv:2203.03982, 2022

[2] Sun Y, Han J, Yan X, et al. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks[J]. Proceedings of the VLDB Endowment, 2011, 4 (11) : 992-1003.

[3] Tang H, Liu J, Zhao M, et al. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations[C]//Fourteenth ACM Conference on Recommender Systems. 2020: 269-278.

[4] He X, Deng K, Wang X, et al. Lightgcn: Simplifying and powering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 2020: 639-648.

[5] Chen C, Zhang M, Zhang Y, et al. Efficient neural matrix factorization without sampling for recommendation[J]. ACM Transactions on Information Systems (TOIS), 2020, 38(2): 1-28.

[6] The Last.fm Dataset | Million Song Dataset

[7] Yelp Dataset

[8] Douban Book