background

With the rapid development of Xianyu business, home page feeds and other large traffic scenarios have become the traffic entrance for products and operations of various business lines. With massive amounts of delivery requirements mentioned technical team, the old the disadvantages of the development, operation mode are evident: more than existing technology is difficult to guarantee the business side of pv goals, click on the conversion efficiency, cannot solve business flow for conflict, the more difficult to the operating instructions of the increasingly diverse the rapid development of online business high cost of trial and error. We urgently need to design a new technical scheme to solve these problems.

Train of thought

The fundamental problem we need to solve is how to control and maximize the efficiency of global traffic conversion from a global perspective when the total traffic is constant. To solve this problem, we need to start from the following perspectives:

  • Ensure the achievement of multi-party traffic targets in a single scenario (such as home page feeds, search results page), and improve conversion efficiency on the premise of meeting the targets.

  • On the premise of meeting the data indicators of a single scene, we can optimize and improve the global transformation efficiency through multiple scenes

  • Enable the business side through engineering means, accelerate the rapid launch of new business, quickly adjust the launch strategy, and reduce the business trial and error cost.

Based on the above considerations, we decided to cooperate with the algorithm team of Damo Institute to open up the Dujiangyan algorithm platform in engineering, so that each delivery business can enjoy the algorithm bonus at a very low cost, and ensure the achievement of business objectives and the improvement of transformation efficiency. At the same time, we designed a set of new delivery system, the idea is that various operational capabilities can be rapidly expanded, can be componentized precipitation, and special material delivery needs to support a new parallel research and development mode, giving business side more opportunities to try.

Kun Peng system construction

In order to achieve reusable, easy to manage and flexible delivery strategies of operational capabilities, we abstract the three-level structure of activities, scenes and materials, and support defining multiple sets of templates in each scene to manage materials. Scene is the biggest concept in Kunpeng system, and it is the stage for material release. The scene must be defined before targeted business development, material configuration and delivery. Taking the search result page as an example, it can be divided into the following five scenarios: giraffe operation scenario, search result feeds scenario, POplayer scenario, Query word intervention scenario, and background atmosphere scenario, as shown in the following figure. Only when these scenarios are defined can you decide where the operational material will work.

After having the scene, we create personalized material templates for each business according to the difference of the delivery needs of each operation, and then the operation students can create materials independently under the constraints of the template. All kinds of accumulated operation capabilities (such as UV fatigue filtering, search keyword filtering, platform filtering, version filtering, etc.) can be checked or filled in on the interface independently by operation students when configuring materials. After the material is created, through the creation of activities, the designated crowd, effective time to put the material out, you can complete the entire release work.

Businesses always have expected goals when making delivery plans. The operation only needs to fill in the delivery goals on the console during delivery, and kunpeng system will automatically cooperate with Dujiangyan algorithm platform to help achieve business goals and optimize conversion efficiency. Operation students configured the release target (e.g., 1 million pv exposure in 3 days), release strategy (release as soon as possible, smooth release, free release) and other material contents through Kunpeng’s offline data link T+1 to the algorithm platform for the mixed algorithm model to formulate the global release plan of the next day. The actual exposure and click data of each material contain the unique business identifier issued by Kunpeng, and this data is recycled to the mixing algorithm platform for continuous optimization and iteration of the model. The actual exposure data of vertical and vertical algorithms (customized algorithm data providers of business, such as live vertical and vertical algorithms and purchase vertical and vertical algorithms) after mixing and sorting will be recycled into the overexposure table of vertical and vertical algorithms through Kunpeng universal exposure filtering data link to solve the repeated exposure problem of vertical and vertical data.

After accessing Kunpeng platform, the data processing process of a request initiated by users is shown as follows:

As can be seen from the figure above, there are many modules involved in the data processing process. In the design process of Kunpeng system, we tried our best to realize various templates through extensible architecture, and precipitated the customized functions of various business implementations into common components, helping more businesses to iterate quickly horizontally.

The architecture design of the whole system is shown in the figure below:

DataFetcher extends the idea system

Kunpeng’s DataFetcher extended idea system is one of the keys to realize low-cost concurrent development. Data delivery requirements if you need to various lines of business from remote real-time access to services (such as according to user’s guide search term real-time recommendation related information), then the business development of students through inheritance DataFetcher extension point base classes, subclasses callback method invokes the remote service to get the data, write, DO the logic of the transformation, you can easily complete the business development and delivery. As for the functions of RPC concurrency, resource isolation, index monitoring and other functions of multiple DataFetcher, they have been packaged at the bottom of Kunpeng, and are completely transparent to business students. Business development students only need to pay attention to the business itself. However, before the birth of Kunpeng system, developers were required to be very familiar with the main code of the scene in the launching of each business. As a result, only the scene owner was competent for the launching and development of scenes such as search and homepage feeds, which became the bottleneck of single point resource development. Kunpeng completely eliminated this situation.

The DataFetcher subsystem is an important realization of “building block” characteristic of Kunpeng. For example, the scene of Xianyu home page feeds needs to release a batch of high-quality rights and interests products recommended in real time for new users, which can be realized through DataFetcher. In the future, when the operation needs to put such rights and interests on the search results page and “I like you” page, the DataFetcher can be reused directly and the corresponding scene can be registered on the Kunpeng console, which effectively saves development resources and improves the efficiency of business on-line.

Filter subsystem

As the business has an increasingly high demand for refined operation, we built a set of building block filter subsystem on Kunpeng, and the precipitation became the basic component for all businesses to use. In addition to the well-known sub-population selection, we also provide pre-filters such as sub-platform (iOS/Android), sub-version, grey traffic ratio, strict matching of search terms, fuzzy matching of search terms, page number filtering, UV fatigue filtering and so on. If a service needs to be operated, you can select a filter on the console and use it directly.

If a business has special requirements that cannot be met by existing filters, it can easily implement special business logic by inheriting the MatFilter base class and precipitate it into a common filter component that can be reused by all businesses. In this way, kunpeng’s filter subsystem capability will continue to accumulate and accumulate, realizing cross-business capability sharing.

Take a practical example of searching for bamboo dragonfly drops: The requirement of the game operators is that when users search for the words “King of Glory” and “Karting” in Xiangyu, the materials of the game bamboo Dragonfly should be targeted at specific game enthusiasts, and only 15% gray flow should be put on versions 6.6.7~6.7.1 of Andoird client, and each user should be exposed for 10 times at most every 3 days. Many filtering conditions are used in such a requirement. With the support of Kunpeng, these filter capabilities can be directly reused and quickly put into use without development. Prior to this, these requirements need to be reviewed, developed, coordinated, tested and released by PRD before they can be put into effect.

Dujiangyan algorithm platform

An important goal of Kunpeng is to help achieve business goals and achieve global traffic allocation optimization with the help of Dharma Institute algorithm. We jointly built the Dujiangyan algorithm platform with Dharma Institute, and realized a set of general mixed algorithm, which connected the offline data path and the online service link from the engineering side. All businesses can reuse this algorithm capability at low cost. In addition to ensuring that the business side achieves the pv target, the PVCTR index has been improved by 60%~100%. With the continuous iteration and optimization of the algorithm model, there is still a lot of room for improvement in the effect index.

Group management and approval flow

Kunpeng system was launched in multiple high-traffic interfaces (home page, SRP search, guess you like, etc.), and each interface was put in by many operation students of business lines. Reasonable group permission management was needed to avoid the problem of incorrectly modifying other business materials. Kunpeng system built a group management subsystem, and each business student had to belong to a certain business group. Students in this group could only see the material under their own group, but not the material under other groups. You must belong to a group when creating stories and activities. In this way, effective group management of people and materials is achieved. Any change in the configuration of a material or activity will automatically trigger an approval flow to the group of approvers. Avoid the risk of human error through the approval process.

Role division after access to Kunpeng

In the old development model, the line between operation and development was blurred. Due to the imperfect capability of the delivery platform, many operational configuration changes need to be implemented by the server side to push the switch or even modify the code. After kunpeng was connected, the division of roles between development and operation became very clear:

  • Scene development OwenR is responsible for docking kunpeng system.

  • Business development students are responsible for realizing their business requirements on the Kun Peng extension point.

  • Operation students are responsible for configuring materials on Kunpeng console, putting out the extension points of business development students, and selecting appropriate basic components of operation according to actual needs to achieve the putting effect.

After the development and realization of business requirements, all subsequent changes in operations were completed by operation students on the console. If the delivery does not meet expectations, the operation can self-help modification, self-help offline, the whole process does not need to participate in development.

The effect

At present, The Kunpeng system of Xianyu has been implemented in many scenarios, such as the homepage feeds, homepage King Kong position, search results page, “I Like you”, and search shading recommendation, etc., which greatly contributes to the goal achievement of each business side. Pv conversion rate has increased by 60%~100%, uv conversion cost has reduced by about 40%, and there is still a lot of room for optimization. The DataFetcher extension point mechanism enables the development resources of all business parties to be invested in parallel, reducing the overall time to bring business requirements online by more than 50%.

Looking forward to

Algorithm empowerment, parallel development of extension points, scene material management and delivery, and building block operation capacity precipitation provided by Kunpeng system have all brought significant positive improvement to business delivery effect and R&D efficiency. In the future, we will continue to dig into the general ability of vertical and vertical algorithms, develop a more flexible extension point development mode, and horizontally open up more scenarios to optimize the global flow control effect, so as to bring more growth points to the business.


2020, thank you for having you

I will be at your side

Douyin: @Xianyu Technology

Today’s headlines: @Idle Fish Technology

● Scan code to pay attention to us