At present, there are many levels in the recommendation system. The whole online recommendation system includes online recommendation service at the top, recommendation data recall set (data subject, classification pool) at the middle level, and various recommendation models at the bottom level.
Recommendation system is involved in all kinds of online businesses. Currently, recommendation system has been involved in content: various business systems such as article, question and answer, comment, etc. Commodity SKU: pure commodity, message push, material, mixed with multiple businesses at the same time. With so many businesses, it is also possible to develop a set of similar codes for each business, but the disadvantage is that the investment is huge, and each business code is similar, and it is not friendly to improve our ability. The development of recommendation engine is a way to not only improve our technical level, but also respond to the rapid development of each recommendation channel.
Recommendation engine must be done, business development is very fast, which business should access personalized recommendation. There is nothing more to discuss about building a recommendation engine. What we need to think about is how to build a recommendation engine.
Which way to think about it? Cedar Maple starts from three similar but different systems: personalized recommendation system, advertising system and search engine. Search engine has excellent open source implementation and a large number of architecture to share the article, determined to start from the search engine, learn from the search engine to build our own recommendation engine.
With search engines can draw lessons from, but although recommendation systems like search engines, but after all is not search, recommendation is more than search and recall process more and more widely, and the need to recall the scope of the portrait is based on the user to build, search the core is the degree of match between input words and articles, search engine core is still today, On the basis of the search architecture, we need to expand the recall process and recall times, and then carry out scoring and sorting according to the feature set drawn from the recall set.
The whole process of recommendation engine core is the first step to pull category recall sets, online service receives the user request, according to customer’s request to pull the theme, labels, material, category recall sets, according to the current recall pull preference, recall similar sets, pull the first step to complete category, build the category filter collection including, but not just have bought, has exposed, click, Each filter set contains real-time, offline filtering for category recall sets.
The second step is to build the material filter set, purchased, exposed, clicked and so on, according to the pulled category recall set, pulled material recall set, such as: article, SKU and so on. Filter the material recall set through the filter set.
The third step is the strategy algorithm. Category, brand and category are separated according to the strategy to improve user experience and return the results. The other is CTR score estimation based on current machine learning and deep learning. This scenario is based on material, pull dozens of dimension characteristics, characteristics of real time into the model on forecast, according to the machine learning model, deep learning model in real time, prioritize, material and the sorting result category, brand, category of partition, partition the purpose of users to show the experience of ascension, as to why the partition? Imagine if a refresh of Today’s headlines sent you all your content on your phone, or a dropdown of Taobao sent you all your content on your laptop.
Step 4 The collection pulled by each service is different, so each recall collection can be flexibly configured on the configuration platform. Machine learning and deep learning models are managed by model management platform to realize dynamic loading and flexible dynamic updating of models. The partition policy platform can be configured flexibly based on each service.
Through the above services, configuration and platform, the general recommendation engine is preliminarily realized, so that most businesses do not have to repeatedly develop similar but different codes every time, and the technical level of everyone in the team is gradually improved.
Personalized recommendation is a booming technology, and recommendation engines are constantly evolving with internal, external, and other domain results.
How to carry out recommendation, popular collection, general collection, real-time news construction and so on by non-preference users will be introduced in detail in the following article.