Evolution of tourism recommendation system

background

Vacation business occupies a very important position in the whole online travel market, how to do a good job to expand the cake is the focus of the industry. Unlike food or hotels, where the user’s interests are clearly defined (for example, finding a certain restaurant or hotel near a destination), the user’s interests in travel scenarios (for example, where to go for a fun weekend) are difficult to determine and vary with the season, weather, user attributes, and so on. These characteristics lead to the traditional information retrieval can not well meet user needs, we urgently need to build a tourism recommendation system (vacation = tourism in this paper).

The tourism recommendation system mainly faces the following challenges:

1. The difference is great in different places. In the local life scenario, most user demands are concentrated locally, while in the tourism scenario, more than 30% of orders come from different places, that is, the user residing in city A buys the travel order of city B. Outsiders visit Beijing recommended the Forbidden City, the Great Wall no problem, Beijing people visit recommended Beijing joy Valley, wildlife park is more appropriate.
2. Recommendations come in many forms. In addition to recommended scenic spots, there are also recommended with group Tours, scenic wine packages. There are a large number of duplicate tickets under the scenic spots, which are not suitable for the “Deal” style display; With group tour, scenery wine package will generally bind multiple scenic spots, and is not suitable for POI (store) style display.
3. Seasonality is obvious. For example, hot springs and skiing are popular in winter, and more people choose water parks in summer.
4. Personalize your needs. For example, the needs of parent-child users and lovers will be different, and further subdivided, the needs of parent-child users aged 1-4 and over 6 will also be different.

To solve the above problems, we customized a complete set of recommendation system framework, including recall sorting strategy based on machine learning, and recommendation engine from offline calculation of massive big data to highly concurrent online service.

Policy iteration

The strategies of the recommendation system are mainly divided into recall and ranking. Recall is responsible for generating recommended candidate sets, and ranking is responsible for personalized ranking of the results of multiple recall strategies. The iterative evolution of recall and sorting strategies will be described in the following sections.

Iteration of recall strategy

We started the construction of the tourism recommendation system at the end of 2015. At this time, the holiday business has an independent home page of surrounding tourism channel, and the platform is responsible for the recommendation strategy of the booth you like, which cannot solve many problems in the tourism scene. Below is a chronological review of how various recall strategies can be used to address these issues.

Hot Sales Strategy 1.0

Recommendation strategy is mainly based on the city, the first edition is different from the sell like hot cakes, based on statistical points Deal city city resident this edition strategy based on user city to statistics, the reason is that in different cities of different tourism resources distribution, lack of existence (group), rich in tourism resources (supply) and local requirements to the surrounding city. That is, for each city, there is a Deal library corresponding to the “city circle”. For example, there is no ski resort in Langfang, but the users whose resident city is Langfang often buy the ski resort in Beijing, so the users in Langfang will recommend the ski resort in Beijing when browsing the local travel channel.

In the specific implementation, the characteristics of tourism products changing seasonally are taken into account, and the sales volume gradually decreases with time. It is assumed that 4 weeks is a change cycle, and the formula of Deal score is: Deal_score = ∑((count(payOrder) * α ^ I), where count(payOrder) * α ^ I) = count(payOrder) * α ^ I), where count(payOrder) * α ^ I) = count(payOrder) * α ^ I) Deal score is the sum of daily sales score within a certain period.

According to the formula above, the Top N hot deals can be counted for each city, and the deals that are more than 200km away from the current city can be filtered according to the POI associated with the deals. For example, it is not good to recommend Shanghai Disneyland tickets when visiting Beijing, which does not conform to the positioning of surrounding tourism.

This stage also tried hot single, low single, new single strategy. New deals and low prices are easier to understand, giving these deals some exposure. Hot orders are similar to hot orders, and Deal browsing data is counted. Hot orders recall deals are similar to hot orders. However, since the recommended evaluation index is visit rate (paid UV/ recommended UV), these strategies are less effective than hot sales, and none of them are online.

There was also a preliminary attempt to make recommendations based on time context, such as weekday/non-weekday, weekday tickets filtered from Monday to Thursday, and weekday tickets filtered from Friday to Sunday, but it was discontinued with the recommendation POI.

There are two main innovations in this phase of strategy:

1. Based on the statistics of the resident cities of users, the products are sold well, breaking through the restrictions of the cities where Deal is located, and the tourism products of surrounding cities can be recommended locally.
2. Through sales attenuation, basically solved the seasonal problem.

Recommend POI,

There are usually multiple kinds of tickets under each scenic spot, and there are usually multiple deals under each kind of ticket. For example, there are adult tickets, student tickets and elderly tickets under the Forbidden City ticket. There are multiple deals under the adult ticket due to different suppliers of Deal, and the price and purchase restrictions of these deals may be different. If displayed according to the “Deal” style, both adult and student tickets of the Palace Museum may be recommended. On the one hand, a large number of repeated similar “Deal” occupy the recommended booth; on the other hand, the long summary information of “Deal” is not conducive to users’ decision-making. Therefore, at the beginning of 2016, POI recommendation was launched. The first version of POI recommendation scheme was based on POI associated with Deal. In other words, adult tickets of the Palace Museum were popular, and the displayed POI of the Palace Museum was actually recommended. There are two problems with this scheme:

1. The recommended Deal may come from the same POI, and the POI needs to be de-duplicated. If there are 30 recommended booths, the number of candidate recommended deals must be >=30, but it may be less than 30 after POI.
2. POI sales backwards deduced from Deal are not accurate, and the actual POI sales need a more accurate statistical method.

So in 2016 Q2 launched POI hot selling strategy based on the value F, F value is the point of a buried Meituan comment on internal tracking method, can be simply interpreted as: the user browse POI details page will be buried in the F value point log POI ID, this tag will then have been taken to the order, so it can relatively accurately calculate each order POI belongs.

Hot Sales Strategy 2.0

The main problem with version 1.0 of the bestseller strategy is that it only takes into account the local buying preferences of users residing in the city. In short, it only solves the problem of Shanghai people’s recommendation when browsing Shanghai, and the result of Beijing people’s recommendation when browsing Shanghai is the same as that of Shanghai people. Zooming in is the problem of the local scene. The definition of the local scene is shown in the table below. The hot-selling policy of version 2.0 collects statistics of orders from other places. When a user visits Meituan, it determines whether the user is a local user or a remote user, and then recalls the corresponding POI. The user who cannot obtain the resident city is regarded as a local request by default. From the recommendation results, Beijing locals love to go to happy Valley, while outsiders love to go to the Great Wall and the Forbidden City.

classification	scenario	The recall strategy
Local demand	Browse City = Resident city (Example: Beijing people browse Beijing)	Hot selling POI purchased by local users (POI city does not necessarily mean browsing city)
Different demand	Browse the city! = Resident city (Example: Chongqing people browse Beijing)	Hot POI purchased by remote users (Hot POI purchased by all non-Beijingers)

In this version, the subdivision recommendation by time context is continued to be tried, and the order distribution in every hour of every day within a certain period of time is counted. There are three saddle points, and the day is divided into three time segments, namely morning, middle and late, and POI sales are counted by time segment. From the perspective of recall, POI ranking changes a lot compared with before, but due to Rerank in the following, it has little influence on the overall recommendation.

User history behavior strongly correlated policies

Although the hot-selling strategy can distinguish the differences between local users, it lacks personalized recommendation for specific individual users. Therefore, the recommendation strategy with strong correlation of user history behavior is introduced. The poIS that users browse or collect and do not purchase within the last month are grouped by city. The weight is removed by POI ID. The more real-time the weight is, the higher the weight is.

Location-based recommendation strategies

The above strategy either has a large amount of POI data or has user data. If the user or POI has no historical behavior data or is sparse, the above strategy will not work, the so-called “cold start” problem. In mobile scenarios, users’ geographical location can be obtained in real time through the device, and then recommendations can be made according to the geographical location. Specific recommendation policies fall into two categories:

1. Search for POI within a few kilometers of the user’s real-time position and sort the list of Top POI according to recent sales attenuation.
2. Find user groups within a few kilometers of a user’s real-time location and recommend Top POI based on their recent purchase behavior. For example, the user is located in Huilongguan. There is no POI near Huilongguan, but the users of Huilongguan will buy some popular POI in season.

The geo-location recommendation policy needs to filter out the case that the selected city of the user is inconsistent with the selected city of the client. For example, it is inappropriate for the user of Beijing to recommend POI around Beijing when browsing Shanghai.

Collaborative filtering strategy

Collaborative filtering is the most classical algorithm in the recommendation system. Compared with the strategy of strong correlation of historical behavior, it abstracts and generalizes the user interest and POI attributes. Collaborative filtering algorithms are mainly divided into ItemCF and UserCF. ItemCF was implemented first, mainly for the following reasons:

• Performance: The number of POI of Meituan tourism is far less than the number of users. The core of collaborative filtering algorithm is to maintain a similarity matrix (Item/User similarity), which costs much less to maintain POI similarity matrix than to maintain User similarity matrix.
• Real-time: new behaviors of users will definitely lead to real-time changes in recommendation results;
• Cold start: New users can recommend other POIS related to a POI as long as they have a behavior for that POI.
• Explainability: Users can be convinced by recommending explanations based on their historical behaviors.

Collaborative filtering based on POI browsing behavior

The similarity between POIS is calculated based on the browsing data of the UUID dimension. Browsing behavior is more dense than ordering and paying behavior. The time window takes one month of data. Theoretically, as long as the computing capacity is not the bottleneck, the time window should be as long as possible. The similarity formula is defined as follows:

The denominator | N (I) | | N (I) | is browsing the POI I the number of users, molecular | N (I) ⋂ N (j) | | N (I) ⋂ N (j) | read at the same time is one month POI users of I and j. After calculating the similarity of POI, the following formula is used to calculate user U’s interest in POI J:

Here N(u)N(u) is the set of POIS browned or purchased by users, S(j,K)S(j,K) is the set of K POis most similar to POI J, wjiwji is the similarity between POI j and I, and Ruirui is user U’s interest in POI I. Collaborative filtering can be regarded as a generalization of strong correlation strategy of user history behavior. Example of final recommendation result: after browsing “Beijing Happy Valley”, users recommend “Beijing Aquarium” and “Xiangshan Park”.

The behavior table of users’ POI is updated after offline production every day, which is equivalent to only the data before the day, without feedback on users’ real-time behavior of the day. Therefore, collaborative filtering recommendation based on users’ real-time POI behavior is added to reuse the calculation results of POI similarity mentioned above.

Collaborative filtering based on user search behavior

Search behavior is a kind of strong intention behavior. Most travel orders come from the search entry, and a considerable proportion of search users do not click on any POI. Recommendation based on user search behavior can be used as a supplement to BROWSING recommendation of POI. Firstly, the similarity matrix of Query and POI is constructed, and the PAIR of POI browsed within 10 minutes after the user searches Query is constructed. The similarity algorithm is consistent with the SIMILARITY formula of POI.

Query+City is used as the Key in the concrete implementation, because there are some national chain POIS in the tourism scene, such as: For Happy Valley and Fonte, if only Query is used as the Key, the POI most related to “Happy Valley” Query may be “Beijing Happy Valley”, so users will recommend Beijing Happy Valley after searching “Happy Valley” in Shenzhen, which does not meet users’ needs.

Similarity improvement

The similarity calculation formula had two improvement points: one is not considering the order of user behavior, such as user has browse the POI, before two similarity calculation, actually only calculate the similarity of A and B and C can, because the user is browsing the browse again B first, can be recommended when browsing A so B, but browsing the B recommended A necessarily appropriate. Secondly, the time series span between POI is not considered. Theoretically, the similarity of A and B should be higher than that of A and C.

The improved similarity formula is as follows, where LL represents the sequence span length of POI I and j, NijlNijl is the number of sequence lengths of POI I and j, and αα is the attenuation coefficient of sequence span (<1) :

Overview of recall policies

After one year’s iteration, the current online recall strategy is shown in the figure below. In addition, the matrix decomposition based on ALS has also been tried, but the recommended results are relatively unpopular and poorly interpretable. In addition, the user tag-based recommendation is enabled to label both users and POI with corresponding attribute labels. Single-dimensional labels can be directly recommended, for example, parent-child POI can be recommended to parent-child users. Labels can also be used as dimensions to calculate the correlation between users and POI in multi-dimensions.

The results of each type of recall strategy need to be filtered. There are mainly several types of filtering strategies:

1. Blacklist filtering. For example, there is dirty data at the source or cases requiring manual intervention.
2. No POI filtering for sale. The filter does not sell Deal’s POIS.
3. POI distance filtering. Filter poIS that are currently viewed hundreds of kilometers away from the city.
4. Not the current city filter. Filter poIS for cities that are not currently being viewed.
5. POI filters have been purchased.

Among them, the first two filtering strategies are common to all recall strategies and need to be implemented. Blacklist filtering takes into account real-time data update and is processed online, while other filtering strategies can be processed uniformly at the offline data layer. The last three types are only required by specific recall policies because they rely on user requests and can only be processed online. The specific rules are as follows:

The recall strategy	Filtering rules
Sales strategy	POI distance filtering
Historical behavior is strongly correlated	POI filters have been purchased Not the current city filter
Location-Based	Not the current city filter
ItemCF	POI filters have been purchased Not the current city filter

Sorting strategy iteration

Each type of recall strategy will recall certain results, and these results need to be sorted uniformly after being de-duplicated. In the early stage, when there was only one hot selling strategy, Rerank was not needed, and the ranking was directly Based on the hot selling score. After adding the strong correlation of historical behavior and location-based strategy, it was also displayed according to the fixed booth cross, such as: The first, third, fifth, and seventh positions were assigned to the strongly correlated historical behavior strategy, and the second, fourth, sixth, and eighth positions were assigned to the location-based strategy.

At the beginning of Q1 2016, I tried the first version of Rerank strategy. At that time, the recommended style was still Deal, so the sorting object was Also Deal. The main feature was sales/score data of 30/180 days.

At the beginning of Q2, the POI display was basically completed, and the sorted objects were changed into POI, whose main features included sales volume, score, price and refund data, but the effect was still not obvious after the launch.

Since the recommendation list page is similar to the filter list page, we tried to access the filter Rerank directly in the middle of Q2, but the effect was not ideal. Then the training was conducted again based on the recommended data samples, and some features were added, which can be roughly divided into the following categories:

Feature dimension	Characteristics of the name	instructions
context	HOUR_OF_DAY	The number of hours in a day
	DAY_OF_WEEK	The day of the week
	CITY_ID	The client selects the city ID
	distance	Distance between the user and the POI
POI	REC_POI_CTR_DAY7 .	POI 7 day click through rate
	POI_ALLCATE_PAY_F_CNT_DAY7 .	POI payment data for 7 days
	POI_COMMENT_CNT_DAY7 .	POI score for 7 days

It can be seen from the above table that contextual features, distance features and visit related features are mainly added on the basis of sales volume and evaluation. It is noted that HOUR_OF_DAY, DAY_OF_WEEK and CITY_ID do not adopt one-hot encoding, and the effect of one-hot encoding in online experiments is not better than using the original value directly. The possible explanation is that HOUR_OF_DAY discrete value can be used to classify the tree model. For example, 0~11 can represent morning, 12 ~18 can represent afternoon, and 19 ~23 can represent night. In the same way, DAY_OF_WEEK can be considered weekdays from Monday to Thursday, and weekends from Friday to Sunday. The smaller the ID, the earlier the city, the more popular the city.

The sample before the last click on the model is taken as the candidate sample set, with payment as the positive sample and other samples as the negative sample. The sampling ratio of positive and negative samples is 1:10. If no sample sampling is done, it is assumed that there is only 1 payment per 100 visits, and each visit to the list page assumes that users can see 10 POIS on average, that is, the ratio of positive and negative samples is about 1:1000. The sample distribution is extremely unbalanced and easy to lead to overfitting. XGBoost algorithm was used in model training, and the click-through rate and purchase rate were significantly positive after Rerank was online, which proved the effectiveness of Rerank.

On the basis of the above, contextual features are gradually enriched. For example, recall may trigger POI in surrounding cities, so POI is added with the feature of whether the POI is local to the city. In addition, the hot-selling recall strategy breaks down the feature of whether the user requests are local to the city, and Rerank also adds the feature of whether the user requests are local to the city. User-poi combination features are added: whether a User has browsed/saved POI within 7 days, real-time features, user-POI correlation based on collaborative filtering, etc., which are strongly correlated with historical behaviors and can echo the recall strategy of collaborative filtering. The static attributes of POI were added, such as star rating, and the sales volume of POI was also divided according to the location. After these features are online, the effect is basically positive, in line with expectations.

Feature dimension	Characteristics of the name	instructions
context	SCENE_LR IS_POI_LOCAL	Is a local OR remote user POI is local to the city
User-POI	POI_VIEWED_DAY7 .	POI Has been accessed in the last 7 days
	POI_RT_VIEWED .	Real-time feature: Whether the user has browsed it recently
	REC_POI_CF_SCORE .	The semantic correlation of User and POI calculated by POI CF
POI	PLACE_STAR .	Star of the scenic spot
	POI_SCENE_PAY_F_CNT_DAY7 .	Sales volume of POI (Use local sales when the user is requesting locally, Use sales in different places)

On the model, the fusion of short-period model and long-period model was tried. The short period was the recent data of one month, and the long period was the recent data of three months. According to the online results, the short-cycle model is the best, which may be related to the rapid change of tourism season. In addition to the above features, you can add personalized User features, weather context features, POI features, CTR/CVR can split local remote features, etc.

Panoramic view of sorting policies

The recommended offline training process is consistent with search and screening, and the flow chart is as follows:

• The data source is the original sample log, which is recorded in Hive. The output is an ISample object and labeled with a label. In addition, some features may need to be produced online and written into the sample log. For example, real-time features cannot be collected by offline ETL.
• Sample selection: filter the initial sample, such as: filter the data after the last click on the sample, the output is still ISample;
• Feature extraction: There is POI ID in the sample, according to which POI sales volume, evaluation and other features can be extracted; Similarly, user characteristics can be extracted based on UUID in the sample. This generates Sample with Feature;
• Data sampling: abstraction of samples according to a pre-defined proportion of positive and negative samples;
• Training set construction & Output: Output the training set in XGBoost format.

The whole training set construction process is written in Scala and run on Spark cluster. However, due to the unstable effect of Spark version of XGBoost, the single version of XGBoost is used in the final model training and evaluation, and the training parameters of the model (number of iterations, tree depth, etc.) are generally selected as experience values. The data of one month is selected for the training set, and several days after the date of the training set are generally selected for the test set. The off-line evaluation indexes mainly refer to AUC. If the off-line effect is improved, the ABTest experiment will be conducted and iterated gradually.

Engineering structure Design

The overall engineering architecture of the recommendation system is shown in the figure below, including offline computing layer, core data layer, recommendation service layer and application scenario layer from bottom to top, as well as background configuration management system and data scheduling service.

Offline computing layer

The offline computing layer mainly includes basic data and application data in addition to features and training logs required by Rerank. Among the basic data, the most important is the data of Deal and POI. In order to ensure the accuracy and real-time performance of data, the data of Deal and POI are directly taken from the tourism product Center and updated in real time through timed full pull supplemented by message queue. Application data can be divided into three categories according to production mode:

1. Hive ETL production data: for example, the offline table (such as the main store) used for POI filtering, and other types of statistical data, such as city POI sales, line tour sales, and users’ browsing/purchasing behaviors on POI.
2. Data generated by Spark, such as User CF, POI CF, and matrix decomposition algorithm, is complicated in logic and cannot be calculated using ETL directly.
3. Data produced by Storm: real-time user behavior is used in recall and sorting. Currently, the company provides a unified real-time user behavior data stream user__action_basic, including: browsing/collecting POI/Deal, ordering, payment, consumption and refund, from which travel POI/Deal behavior can be filtered.

Core data Layer

An important reason for the abstraction of the core data layer is the need for offline computing engineering and online service engineering reuse DataSet, which can be divided into three types from the storage mode for online use:

1. Data stored in ElasticSearch. It is mainly POI/Deal index, for example, geographical Location and city of POI. When online filtering needs to be Based on geographical Location, ES can be used to query, for example, distance limit of city circle and location-based policy recall within a certain distance. In addition, ES is more suitable than KV storage for multi-dimensional query scenarios. Such data is scheduled through the company’s unified task scheduling system and is usually updated every few hours. Create an alias for the ES index. Update the index offline to ensure atomicity.
2. Data stored in the DataHub. DataHub is a data management system developed by the wine travel search team, which integrates data storage, management and use. DataHub mainly uses Tair as the internal storage and is transparent to the client. The client interface supports one or two dimensional keys, and the interfaces are basically consistent with the application. In addition, the application does not need to maintain Tair cluster configuration management by itself. DataHub has the scheduling function, and automatically writes Tair after HDFS partition is generated by scanning.
3. Data stored directly in Tair. DataHub is not supported by two types of scenarios, one is real-time data storage landing, the other is value directly store objects, the benefits of storing objects as objects can be read from Tair directly for online use, without self-serialization and value assignment. Real-time data does not need to be scheduled. Data written into Tair using Spark can be executed only when the upstream Hive table is Ready. Therefore, the unified data collaboration platform of the company is used for scheduling.

Recommendation Service Layer

Service upstream and downstream

The architecture diagram of upstream and downstream recommendation is shown in the following figure. The client invokes the API, and the API invokes the recommendation service to get the ID of the recommendation, and then adds relevant fields for App display and sends them to the App. The main reason why recommendation and search are not integrated into one service is that the recommended recall strategies are complex and diverse, and multiple recall strategies may be hit in each request. However, the intention of a single search request is usually single, and usually there is only one recall strategy. In addition, recommendation service focuses on recall and filtering. Rerank calls independent rank service, because many features of recommendation Rerank and search and screening Rerank can be reused, such as user features and POI features.

The whole process

The recommendation service obtains data from dozens of data sources and supports dozens of application scenarios after service logic processing. The entire invocation process is as follows:

1. RecommendServicePublisher as a service entrance, after received the Request from the Client Request first authentication Request is legal, such as: Request parameters in the scene Booth and UUID cannot be empty.
2. Construct the request Context, in which a unique global ID is generated to identify a request, the resident city is obtained by querying the user portrait service according to the UUID, the city is located according to the latitude and longitude query, and the recall sorting Strategy of processing requests is obtained according to the ABTest split configuration.
3. Obtain the corresponding AbstractHandler based on the Booth of the request scenario. By default, you can use the uniform AbstractHandler, including recall, filter, rerank, and Post Rerank.
4. Package the result returned by the Handler, add the recall and sorting policy names and scores, and finally return the result to the Client.

Core processes and models

Handler is the core of the whole process and its call flow is as follows:

1. Handler obtains the corresponding SelectRule set based on different strategies. A scenario Booth may correspond to multiple strategies, similar to ABTest. For example, Baseline is a Strategy. Each Strategy may have multiple SelectRule. For example, the Baseline policy is composed of historical behavior strongly correlated SelectRule, location-based SelectRule, and hot Rule.
2. Recall: Each SelectRule is corresponding to multiple SelectRule SelectRule. Multiple SelectRule SelectRule can obtain results concurrently through the thread pool. For example, location-based Rule can be subdivided into recall Based on hot selling POI and recall Based on purchasing POI of surrounding users. The Selector can be abstracted again, for example: the city selling strategy of different scenes, both Meituan and Dianping platforms need, but the data source is slightly different, in addition, the data obtained from ES and DataHub can be added Cache.
3. Merge de-duplication: The results of multiple recall strategies need to be de-duplication by Merge. For example, the location-based strategy was fixed at 2, 4 and 6 digits in the early stage without Rerank.
4. Filtering: There are two levels of filtering. The first level is for SelectRule. For example, for the results of browsing behavior and collection behavior recall in the strategy with strong correlation with historical behavior, POI that users have purchased should be filtered. The other level is filtering for all policies that pass, such as blacklists and travel agents.
5. Reordering: Call the POI Rerank service for the POI list and Deal Rerank service for the Deal list.
6. PostRerank: Generally used to deal with the needs of advertising operations and human intervention cases.

The core object model is shown below:

Monitor the drop

Monitoring is divided into offline monitoring and real-time monitoring. In offline monitoring, Falcon is used to monitor the following indicators:

• JVM monitoring: such as FullGC count, memory usage, Thread Block status
• ES monitoring: The number of ES queries and average response time
• Service monitoring: The number of requests and average response time for each interface and policy

Real-time monitoring access to the company’s unified real-time data statistics platform, time-sharing, multi-granularity statistics Booth request times and response time.

Demotion is mainly achieved through Hystrix. For example, if the response time of Rerank service exceeds the set threshold within a certain period of time, Rerank service is directly cut off.

tools

The Debug tool is developed for the recommendation service. Parameters such as city, booth, UUID, latitude and longitude are input, and the output displays the header diagram, title, distance between POI/Deal and user, recall sorting strategy and score, etc. Convenient PM and RD testing, location tracing Case.

Application scenarios

The recommendation system supports a total of 20 application scenarios of Meituan/Dianping. The main scenario is the home page of the surrounding game channel guess you like it. Its recall strategy has been described above, and the other recommended scenarios are mainly described here:

Recommend a group tour

The recall strategy is similar to the hot-selling POI strategy. According to the results, locals in Beijing will recommend “Gubei Water Town one-day tour”, while outsiders will recommend “The Forbidden City and the Great Wall One-day tour” when visiting Beijing.

Screening remote recall

Users in the screening of first choice when they check in city and screening of the city hotel POI, and problems of tourists is not rich in tourism resources, screening need to break through selecting city limits, and can recommend the surrounding city hot POI, screening beyond recall a percentage of orders increased after online, is an effective supplement to local recalled.

Filter topic tag mining

POI is labeled, users can use these labels to screen, such as: popular nearby, suburban area, where to go on the weekend, family fun, night leisure. Each tag can define a set of mining methods, for example: “parent-child fun” has the following categories of methods:

• POI has parent-child tickets
• Deal title contains “Parent and child”
• Include both “Adult ticket” and “child ticket” under the same POI
• POI purchased in the last month by a user whose profile is “parent-child”

The mining method mentioned above is irregular, and it is expected to mine POI descriptions and comments through semi-unsupervised methods to automatically mark POI.

Few searches/no result recommendations

The recommendation of less search results means that when the number of POI clustering results is 1, the recommendation information is provided for users to enrich the page content. This paper focuses on the use of POI results to trigger recommendations according to POI CF, and the use of the category of searched POI to recommend the same category in the same city.

Searching for resultless recommendation can directly count POI browseen by users within a certain period of time after searching Query to make recommendations, but the coverage of this strategy is limited. Further, Query CF within a period of time can be calculated, and then collaborative recommendation can be made. On the other hand, it can judge whether there are category words in Query through intention recognition to trigger recommendation of the same category.

Wine travel cross-recommended

At present, only the cross-recommendation between hotels and tourism is realized. When users search in the hotel channel, they first judge whether Query is the tourism intention. Two types of intentions are analyzed: one is the POI intention of scenic spots, which recommends POI within several kilometers of the scenic spot; Second, the category intention, such as: hot spring, skiing, will recommend users to locate the nearby hot selling POI of this category.

On the hotel POI details page, you will get the location of the hotel POI and recommend attractions near the hotel. For remote users, scenic spot recommendation will be triggered when they browse hotels, while for local users, tourism recommendation will be triggered only when they browse suburban hotels. This assumes that local users may not have obvious intention of traveling for vacation when they browse urban hotels.

In addition to the application in various recommendation scenarios, these strategies have also been applied in operation. For example, after users browse or purchase POI, they PUSH similar POI to users according to POI CF. Experiments prove that the click rate of PUSH of recommendation strategies is higher than the average level.

Challenges ahead

After more than a year of iterative optimization, a considerable proportion of orders in the surrounding game channels came from recommendations, and about 20 recommendation scenarios were supported online. Many recommendation strategies were added as features to search and screen Rerank, which had obvious positive effects and had a preliminary exploration in user operation. Based on the current recommendation system itself, there are many optimization points:

• Recall strategy: there is a lot of room for improvement in the breadth and depth of the strategy. In terms of breadth, we can continue to explore matrix decomposition FFM, User CF, User image-based recommendation and graph mining. In depth, multiple similarity calculation methods, such as LLR, and multi-time/multi-user dimension recall strategies were tried. The data can be extended to the user data of the hotel and even the whole platform of Meituan. In addition, the offline implementation of the strategy should be more modular and abstract. For example, the similarity improvement algorithm can be verified to be effective in one scene and quickly promoted to other scenes
• Ranking strategy: In feature engineering, personalized User features and weather /Listwise context features can be added. DNN and other methods can be tried in the model. The evaluation index can be improved from visit rate to visit rate (consumption UV/ visit UV)
• Engineering architecture: less search/no result recommendation is migrated from search project to recommendation project. In addition, “light reconstruction” is required for boundary division of storage mode of core data layer, caching of online service layer, demotion of Selector/Rerank, logic combing of Filter/Merge, etc.
• Application scenarios: In addition to cross-recommendation before hotel purchase, post-purchase recommendation can be added, as well as cross-recommendation related to air and train tickets. More scenario-based construction can be explored within tourism, such as parent-child tour and couples tour

Jumping out of the current single POI/Deal list as the main body of the recommendation form, we can see how to do a good job in tourism recommendation from the user, scene, content, touch mode four aspects:

The user needs

First of all, who is the user? What are the user needs to be satisfied? It can use the hundreds of millions of users of Meituan/Dianping to “label the crowd”, whether they are high-end quality female users in first-tier and second-tier cities, middle-aged uncles who live frugally or young mothers who are affordable in third-tier cities. Then analyze the needs behind these groups, whether they are local leisure users, business travel users or frequent vacation users, the needs of different users are different.

Scene classification

What is the scenario that you need to know when you know the user? Scenarios can be defined from four dimensions: time, location, behavior, and channel.

The time is easy to understand. When users search “ski resort” on Thursday and Friday, they will be considered as leisure holiday weekend users, who can jointly recommend ski resorts in the suburbs of Beijing.

Geographical location is the core factor. Local or remote demand should be judged according to the resident city of the user and the city selected by the client. Business hotels can be recommended for business travel users in different places.

Behavior is the most direct response to users’ demands. For example, when users search “Gubei Water Town”, they can recommend gubei Water Town related hotels and scenic spots tickets regardless of whether they have browsing behavior later.

The channels include multiple terminals such as Meituan/Dianping dual platform App, I version, PC and so on. Taking Meituan App as an example, the user characteristics of peripheral travel, hotel and air ticket/train ticket channels are different. For example, the major transportation channel is mostly for business travel users, while peripheral travel channel is mostly for local vacation and leisure people.

Content of the form

Knowing who the user is and in what context, what kind of content product should be considered? For Meituan, the core is transaction, content is not the core goal, but content is a very good diversion measure. Take the local scene as an example to strengthen the scene construction, such as: parent-child, group construction, hot spring, etc. The setting before the departure can strengthen the construction of destination, comment on travel notes, hotel transportation schedule and other contents.

Touch of way

In addition to the current search recommendation, it can also increase targeted delivery, content guidance, advertising placement, activity operations and other ways to reach.

In a word, tourism recommendation issues are complex and diverse, requiring comprehensive consideration and planning from the six elements of holiday travel: food, accommodation, travel, travel, shopping and entertainment. There are still great challenges and opportunities for product form, business strategy and technical structure.

Author’s brief introduction

Zheng Gang, Senior technical expert of Meituan Dianping. Graduated from the Institute of Computing Science of Chinese Academy of Sciences in 2010, joined Meituan in 2011, participated in the early data platform construction of Meituan, and was successively responsible for the construction of platform, wine and tourism data warehouse and data products. At present, I am in the data RESEARCH and development center of wine and Tourism business group, mainly responsible for search sorting recommendation and data mining in hotel tourism scenarios. It is committed to solving business pain points with big data and machine learning technologies and improving user experience.

Don’t want to miss tech blog updates? Want to comment on articles and interact with authors? First access to technical salon information?

Please follow our official wechat account “Meituan-Dianping Technical Team”. Take out your phone now and scan it: