Session-based Recommendation is a sub-branch of Recommendation field, and the Growth Technology department of Meituan Platform is also constantly exploring in this field. Not long ago, the time convolution Neural network model (CA-TCN) proposed by the department for cross-session information perception was accepted by the international conference ICDM NeuRec Workshop 2020. This paper will introduce the CA-TCN model in this paper, hoping to be of help or inspiration to students engaged in related work.

International Conference on Data Mining (ICDM) is the world’s top Data Mining research Conference held by IEEE. The conference covers data mining related areas such as statistics, machine learning, pattern recognition, databases, data warehouses, data visualization, knowledge-based systems and high performance computing. ICDM NeuRec Workshop aims to systematically discuss the latest advances in both shallow and deep neural algorithms for recommendation systems from an application and theoretical perspective. The Workshop solicited the latest and important contributions on the development and application of neural algorithms and theories to build intelligent recommendation systems.

background

In the era of big data, recommendation system, as the infrastructure of the system, begins to play a more and more important role. Recommendation system can select the goods or content that users are interested in, so as to reduce some influence caused by information explosion. At present, many recommendation models proposed by the industry have achieved great success, but most of the recommendation methods often need to be recommended based on clear user portrait information, but in some specific fields, the user portrait information may not be utilized.

In order to solve this problem, session-based Recommendation task is proposed, which predicts the next behavior of users based on the behavior sequence of users in the current Session without relying on any user portrait information [1]. At present, the task of session recommendation has been widely used in many fields, such as the next page recommendation, the next POI recommendation, the next product recommendation and so on. To cover multiple domains, the concept of “session” is not limited to transactions, but refers to a collection of elements consumed or accessed at a time or over a period of time.

Each Session is an item transfer sequence, so the Session recommendation task can be naturally regarded as a sequential recommendation task. The Session recommendation model [2] based on recurrent neural network (RNN) is the mainstream model of application. However, the RNN model only models the continuous one-way transition relationship between items, and ignores the transition relationship between other items in the session. With the hot spot of graph neural network, session recommendation models based on graph structure such as SR-GNN[3] and GC-SAN[4] were proposed to overcome this shortcoming. The graph structure-based session recommendation model constructs the item transfer sequence of the session into a graph structure, and then uses the graph neural network model to explore the complex transfer relationship between multiple items. At present, graph structure-based session recommendation models have become state-of-the art solutions, but they still have certain limitations, as observed below:

  • Observation 1: Almost all existing session recommendation methods focus only on the internal information of the session and ignore the external information (cross-session interaction) that often contains valuable supplementary information to more accurately infer the user preferences of the current session. As shown in the figure below, for example, Item_3 Airpods in Session3, existing methods only focus on the effect of Item_9 in Session Session3 on Item_3 and ignore the effect of other sessions. For Session1, users may compare the same category with the intention of buying earphones, so Item_2 and Item_4 will have a category effect on Item_3. For Session 2, users may prefer the Apple brand, so item_5 and Item_6 will have a brand effect on Item_3. From the observations above, the cross-session influence at the Item_Level level is critical to better inferring the global representation of an item. At the same time, different sessions may have similar user intentions and behavior patterns, so the cross-session effect on session-level plays a very important role in more accurately predicting the next action of the user in the current Session.

  • Observation 2: Session recommendation method based on graph structure In the process of graph construction, the same item appearing in different time steps is regarded as the same node, so that the location information in the sequence will be lost, so that the Session graph structure constructed by different sequence sessions is completely the same. For example, two different Session Session S1: v_i – > v_j – > v_i – > v_k – > v_j – > v_k and Session S2: V_i –> V_J –> V_K –> v_J –> V_I –> v_K, in Figure 2, their corresponding graph structure is exactly the same, which inevitably limits the model’s ability to obtain an accurate session representation. In addition, in the construction of session graphs, edges are established between only two adjacent items that are directly connected, which means that only the item clicked last before the current item is the first-order neighbor of the current item, as shown in Figure 2. However, the items that appear in the same session, even if they are not clicked continuously, are related to each other, so the graph structure has a limited ability to preserve the long-term dependence of the sequence. On the contrary, for the sequential convolutional neural network (TCN) [5] model, results Convolution enables items in the receiving domain of the current item to be directly convolved as first-order neighbors. In addition, Dilated Convolution enables distant items to directly affect them as first-order neighbors.

Related work Introduction

Existing session recommendation methods can be roughly divided into two categories, which are based on collaborative filtering method and deep learning method:

  • Collaborative filtering method: Collaborative filtering method is a general method widely used in recommendation system. Collaborative filtering method can be divided into two categories: KNN-based search method and similarity modeling method. The knN-based lookup method realizes the recommendation by searching top-K related users or items. The KNN-based lookup method realizes the session-based recommendation by searching the item most similar to the last item in the current session. Recently, KNN-RNN[6] explored the combination of RNN model and KNN model to extract Session sequence information through THE RNN model, and then find the item that appears in the Session similar to the current Session to implement recommendation. As for the method based on similarity modeling, CSRM[7] uses memory network to model the relevant information contained in m sessions closest to the current session time, so as to obtain more accurate session representation and improve the performance of session recommendation.
  • Based on deep learning method: Deep learning method has achieved satisfactory results in many fields with its powerful feature learning ability. For session recommendation task, recurrent neural network RNN is an intuitive choice, and its advantage of extracting sequence features can be used to capture complex dependencies within the session. GRU4Rec[2] uses a gated loop unit (GRU) as a special form of RNN to learn long-term dependencies between items in order to predict the next action in the session. Some subsequent work improved and extended the model by adding attention mechanism and memory mechanism on the basis of RNN model. Among them, NARM[8] explored a hierarchical encoder with attention mechanism, which could model the sequential behavior and main intention of users in the current session. Recently, with the rapid development of graph neural network model, the session recommendation model that relies on graph structure appears. Sr-gnn firstly proposes to map each session to a graph structure, and uses graph neural network model GNN to model the complex transition relationship between items. Later, GC-SAN extended the SR-GNN model further by adding a self-attention mechanism, thus becoming a state-of-art solution.

There are obvious differences between ca-TCN model and existing methods. On the one hand, CA-TCN explores cross-session effects of item-level and session-level to improve recommendation performance. There are two differences between ca-TCN and other collaborative filtering methods: 1. Ca-tcn considers the influence of cross-session information on different levels of item and Session, while CSRM only considers the Session level. 2. Ca-tcn constructs a global cross-session item graph and session-Context graph across sessions to explore complex cross-session effects through GNN. On the other hand, compared with RNN and GNN-based models, CA-TCN model overcomes the shortcomings of RNN model’s inability to parallelism, missing position of graph structure and long-term dependence on information.

Temporal Convolutional Neural Network Model for Cross-session Perception (CA-TCN)

1. Overall framework of the model

The overall framework of the network is shown in Figure 3 below. Given Session sequence data, we first construct a cross-session item-graph to link items that appear in different sessions and are related, and then output the Item vector containing global information through the Graph neural network. The item vector obtained is input into TCN model to output the item representation containing Session sequence information, and the Session representation is obtained by integrating the item representation according to the item-level Attention mechanism. Afterwards, the session-Context Graph is constructed based on the similarity between Session representations to model the cross-session relationship at the Session level. Finally, the prediction is made according to the representation of Session and item.

2. Cross-session item-graph

In the first stage, we construct cross-session item-graph directed Graph G_ Item, where each node in the Graph represents an Item, (v_s_i, v_s_i+1) as an edge, representing that the user in Session S clicked v_s_i+1 after v_s_i. Compared with the existing methods, the cross-session G_item graph can establish a link between all the items that appear in the session, so that G_item can get not only the internal information of the session, but also the external information of the non-current session. The core of the G_item graph is that all items are considered together and links are established between items by clicking behaviors in each session. Clicking information of different sessions is summarized together to enrich the relationship between items.

To take full advantage of the information in the G_item graph structure, CA-tCN takes into account the click order and co-occurrence of items. For click order, adjacency matrices A_in and A_out with directions are built to model the input and output directions. On the basis of adjacency matrix, weight matrices Weight_in and Weight_out are obtained by setting different weights for different edges according to the co-occurrence times between items. By assigning different weights, items with more co-occurrence times will play a greater role, and vice versa, thus avoiding noise effects.

Next, we develop a GNN model to capture the effects of complex cross-session information on item__level. GNN maps each item to a D-dimensional embedding V ∈R^ D to obtain a global item vector (item_vector) containing cross-session information.

3. Time convolution Neural Network Model (TCN Model)

In the second stage, we use time convolution neural network TCN to model the session sequence and obtain the global and local representation of session S. Each session S is composed of multiple items. The global vectorized representation (Item_vector) of item contained in the input session S is put into the TCN model. For each item in the session, causality and dilation convolution are calculated, and session sequence information is extracted.

The TCN output of the last item in the session is used as the local information of session S to correctly obtain the current interest of the user:

In addition, the representation of items contained in session S is used to obtain the global representation (session_vector) of session by means of weighted summations to capture the global information of users. In order to distinguish the different effects of different items on the session, item hierarchical attention mechanism is adopted to make the session representation focus more on the items with high importance.

4. Session-context-graph

The local and global representations of a session focus only on the current session and ignore the influence between sessions. To overcome this shortcoming, we build a context-aware session-context-graph to consider the complex relationships between different sessions. In the session graph, each node represents a session S, and the edge links represent similarities between the two sessions. An important question to consider is how to determine whether an edge exists or not. For each pair of sessions, we calculate the similarity of their two representations, and then use knN-Graph [9] model based on the similarity value to determine the neighbor of a session node. After constructing the structure of the session graph, we use the attention mechanism of the session layer and the graph neural network model [10] to integrate the influence of the session neighbor nodes on themselves. At the same time, the attention of the session layer also takes the similarity between sessions into consideration, and finally obtains the context-sensitive session representation.

5. Click Forecast

In order to better predict the next behavior of the user, we use the fusion function to fuse the local representation of the session, the global representation and the representation based on the cross-session information to get the final session representation:

Finally, according to the representation of item and session, we predict the probability of each candidate item becoming the next click of the user, sort the item in reverse order according to the probability, select the item whose probability is in the first preset digit, and recommend it as the preferred item of the user.

The experimental evaluation

In order to evaluate the performance of the proposed CA-TCN, we used two widely used benchmark data sets, namely Yoochoose and Diginetica. The model performance evaluation results are shown in the table below. Ca-tcn is superior to the current state-of-art solution based on RNN and graph structure.

In addition, we performed ablation experiments to assess the impact of each component of ca-TCN, including the TCN model, cross-Session Item-Graph, and Session-Context Graph. The experimental results in the following figure prove that CA-TCN achieves a gradual performance improvement in the session recommendation task by using both TCN model and cross-session information.

  • Ca-tcn (ca.exl) : is a variant of CA-TCN, which only contains cross-session information of time-convolution neural network, cross-session item-graph and session-Context Graph.
  • Ca-tcn (sc.exl) : a variant of ca-tCN that contains cross-session item-level information of the cross-session item-graph, but excludes the session-level cross-session item-graph.

The future work

At present, this paper has applied for a patent, and we will explore and implement the task of session recommendation and sequence recommendation in meituan’s multiple business lines in the future. In particular, the performance of CA-TCN model is verified on e-commerce data set Yoochoose, which proves that CA-TCN model is suitable for e-commerce market landscape with commodity attributes, and can be applied in future business lines with commodity attributes such as “Tuanhao Goods” and “Meituan Preferred”.

reference

  • [1]. S. Wang, L. Cao, AND Y. Wang, “A Survey on Session-based Recommender Systems”, ArXiv preprint arXiv: 1902.04864, 2019.
  • [2]. B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, “The Session – -based recommendations with recurrent neural networks,” arXiv preprintarXiv: 1511.06939, 2015.
  • [3].S. Wu, Y. Tang, Y. Zhu, L. Wang, X. Xie, AND T. Tan, “Recommendation for Neural Networks based on Slock-based Neural Networks,” In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 346-353.
  • [4]. C. Xu, P. Zhao, Y. Liu, V. S. Sheng, J. Xu, F. Zhuang, J. Fang, and X. Zhou, “Graph Contextualized Self-attention Network for Session-based Recommendation.” inIJCAI, 2019, pp. 3940 — 3946.
  • [5]. S. Bai, J. Z. Kolter, and V. Koltun, “An empirical Evaluation of Generic convolutional and Recurrent networks for Sequence Modeling,” ArXivpreprint arXiv: 1803.01271, 2018.
  • [6] d. Jannach and M. Ludewig, “When Recurrent Neural Networks Meet the Neighborhood for Session-based Recommendation,” InRecSys’ 17, 2017.
  • [7]. M. Wang, P. Ren, L. Mei, Z. Chen, J. Ma, and M. de Rijke, “A collaborative Session-based Recommendation Approach with Parallel Memory Modules,” in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019, pp. 345-354.
  • [8].J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, “Neural Awake Sess-based Recommendation,” In Proceedings of the 2017 ACM Conference on Information and Knowledge Management, 2017, pp. 1419 — 1428.
  • [9]. W. Dong, C. Moses, AND K. Li, “Efficient K-Nearest Neighbor Graph Construction for Generic Similarity Measures,” In Proceedings of the 20THInternational Conference on World Wide Web, 2011, pp. 577 — 586.
  • [10].P. Veli, ckovi ́c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph Attention Networks,” ArXiv preprint arXiv: 1710.10903, 2017.

The author information

This article is written by Rui Ye, Qing Zhang and Liang Heng, all from Meituan Platform Growth Technology Department.

Recruitment information

Meituan User growth Technology Department, Meituan user growth core team, long-term recruitment of search, recommendation, NLP algorithm and background engineers, coordinate in Beijing. If you are interested, please send your resume to [email protected] (please mark the email subject: Meituan User Growth Technology Department).

| want to read more technical articles, please pay close attention to Meituan technical team (meituantech) WeChat public official.

| in the public, the menu bar reply goodies for [2019], [2018] special purchases, goodies for [2017], [method] the key word, can see Meituan technology team calendar year essay collection.