Welcome toTencent Cloud + community, get more Tencent mass technology practice dry goods oh ~

Author: Yang Leon, senior engineer of Tencent Big Data, joined Tencent in 2011, responsible for the design and development of Tencent big data products, massive user portraits, Tencent Mobile Analysis (MTA), Tencent Mobile Push (carrier pigeon) and other core products, focusing on product value mining, system architecture optimization and other directions.

Hello everyone, the topic I share today is Tencent Mobile analysis and product operation. First of all, I would like to introduce myself. I am from Tencent Big Data. I joined Tencent in 2011.

There are about three topics to share today

  • The first is what mobile analytics can do for us and what its value is.
  • Second, as a one-stop product marketing platform, how do we improve the process of product marketing promotion?
  • Third, Tencent’s data accumulation and big data capabilities enable enterprises to establish a mutually-beneficial service ecosystem.

Mobile analysis tool

Evolution of movement analysis

The analysis of mobile APP needs can be divided into four stages. First of all, the most basic indicators, reports, mobile portraits, data from different platforms can be accessed for collaborative analysis. When the APP reaches a certain stage of development, payment begins to appear and we have our own profit model, we need to focus more on fine operation. At this time, we will pay more attention to the groups with the characteristics of user loss and potential payment.

After the promotion of APP, we pay more attention to the whole process from the click and download of APP promotion to the final activation, even registration and payment, as well as the effect of marketing promotion. Finish these things, in fact you can also do more step, should is the use of large data capacity, to solve some of the pain points in the industry, such as the financial sector assessment model, hope to be able to output a set of industry solutions to help some of new businesses and users can more quickly into his industry, solve some operation on the spot.

Looking back, the whole process is actually from knowing and understanding the overall state of the business, to having targeted insight into specific target groups, taking actions, and finally becoming an industry expert.

Business indicators and valuation system

When it comes to business metrics, companies may focus on different things at different stages. In the product minimum value phase, or MVP phase, companies may focus more on the function points that leverage user needs. During the rapid growth phase, we became more focused on user acquisition and retention. In the paid phase, we pay more attention to a monetization cycle, channel conversion, etc. Based on the pirate model and the 2A3R marketing theory we are familiar with, we created a whole set of business indicator system, including user acquisition, user activity, retention, communication and income, and then combined with channel effect analysis, user life cycle management and event analysis to build a whole set of APP system.

In addition, we should combine competitive products and industry trends. For example, some industries of AI or blockchain are hot now. People are more optimistic about the future development of some enterprises in these industries compared with those in sunset industries. Combining with APP itself, competitive products and industry trends, a nine-dimensional value evaluation system can be launched to describe the overall operation status of APP in a more immediate and objective way.

Multi-platform Access

Business data exists in many forms, on different platforms, giving a specific solution. First of all, the mobile terminal, the two major operating systems, Android and IOS, both provide the ability of 10-minute fast access. At present, this piece of access has been simplified to a line of code. HTML5, as a unique development mode of mobile terminal, has a shorter development cycle, good portability and fast update. It is widely used in mobile public accounts, Web pages and so on. We have also launched a set of analysis framework for H5. There are also some apps, such as the container integrated with H5 in wechat, in which we have made some efforts to get through H5 and Hybrid.

Wechat small program, we also launched our own statistical analysis tool, this piece also has many industry benchmarking in use. When it comes to smart hardware, Google and Apple have started to push in recent years with some of their own software development kits, and we’ve done some layout here. Some data is updated from the server side, such as internal status updates for some user accounts, so we also provide server-side access.

Mobile device portrait

A common question asked by users connected to the MTA is how do we know the distribution of users, which brings us to our mobile user profile. To be a data platform, in fact, the most basic thing is to have its own portrait system. As we know, Tencent’s business lines are very extensive, and it has many layouts in social networking, news, entertainment, finance and other fields. Many apps have hundreds of millions or even billions of users, generating hundreds of billions of streams of data each year. How to make an effective order to these data, break through the data island, is also a difficult problem we have to solve.

The data is connected, analyzed, and finally a standardized result is obtained. It includes both structured data, such as user attributes, business interests and behavioral characteristics. It also includes some unstructured data, such as corpus, sound, image and other information. After the whole process, it is transformed into Tencent multidimensional massive data assets. This involves a lot of algorithms and specific domain problems, the most basic of which is the recognition of device features.

Equipment identification

When it comes to device recognition, Android system can obtain the identity of IMEI through API, but this has a defect. The IMEI of shanzhai phones in the market is the same. In addition, the emulator of the terminal will also cause interference to our statistics, and some users will tamper with the device number. For example, there are many IMEI can be modified in the market under the condition of Android root. It is also possible to report some attacks, forge log requests, tamper with the device number field inside.

In all the above cases, we can get the equipment number, but the equipment number may not be accurate enough or the availability is poor. There is also a case because of limited system access to the device number, such as Android 6.0, the device number management authority is also receiving more and more tight. As the public has become increasingly aware of and concerned about user privacy, Google’s Play store forbids collection of IMEI, and the European Union has recently updated its standards for collection of personal information. It is also forbidden for APP to obtain the IMEI information of the device when it is not necessary.

Based on the above facts, we launched our own scheme to solve the problem of device identification in different scenarios by means of device identification on mobile terminals, device fingerprint information and encryption distribution on the server side. Let’s call that MID. In addition, through an offline analysis system, the device, brush, reset and repeated allocation of MID can be associated, so as to achieve the final consistency. All of this is what we’re building on in the index statistics, in a whole set of equipment portraits.

Attribution analysis

When an enterprise enters a stage of refined operation, the most important thing is attribution analysis. As an example, take the loss of users in a game industry. The analysis of a well-known game APP found that its lost users can be divided into three categories:

  • One is the feeling that there are obstacles to growth, no matter how to work hard, can not feel the big players, so angry and leave.
  • Second, the burden of the game is too heavy. I feel that I spend many hours here every day, and every day’s gameplay is monotonous and boring.
  • Third, I encountered some obstacles in PVP, such as the official crackdown on plug-ins, or the monopoly of some gangs.

We obviously have different strategies for losing users for different reasons. The operator of this APP, if maximizes the cost at the same time, to solve the problem of user loss. Are you going to hand out a questionnaire to everyone? This requires the ability to do attribution analysis. We provide various means and methods here, such as subdividing the characteristics of lost users, modeling in some ways, and finally making a prediction of potential lost users. We have a special team to do such things.

Marketing promotion and effect monitoring

The first part is about some of the services that mobile analytics can provide, and the next part is about what processes we can improve in terms of marketing and advertising performance monitoring.

Advertising performance monitoring

In user acquisition, we actually have many ways, the most traditional is paid advertising, advertising promotion form. We improve the whole effect from the four processes of advertising promotion.

  • The first is the selection of the crowd, how do we choose these people to promote, better demarcate our target users.
  • The second is the advertising link, which can be quickly and seamlessly put into the advertising platform with one click.
  • Third, effect monitoring.
  • Fourth, the flow cleaning, the abnormal flow out, to better save the cost of putting party.

The user group

User clustering, both rule-based and algorithm-based models will be provided here. Rule-based models include active and inactive users, or known custom practices, user and device portraits, on which some rule-based clustering can be done with or without rules.

Based on the algorithm, it can predict the loss of users and the classification of high potential users.

Similar population expansion

If you are not satisfied with the results of the previous group, or want to expand new users, a lookalike form is also provided here. Here is a car industry offline lookalike scenes, through the correlation and the market users, superposition of sorting, can find its target groups in different interest correlation on the category, you’ll notice on not to some common cognitive category, such as household building materials, construction engineering these industries interested in people, as its target users contact ratio, It’s higher.

This is an effective promotion from offline to online. With the help of the ability of Lookalike, we can provide better analysis for each customer who accesses.

Data opening One-click export

Finally, it can be exported to Tencent’s internal advertising platform with one click. This is the actual operation interface, and it can be seen that active users in April have been pushed to the WIDE access platform.

Customize channel connection schemes

There’s a famous saying in advertising, I know half of my advertising dollars are wasted, but I don’t know which half. Therefore, it is very important to track the advertising effect. In fact, it not only connects the top10 platforms, including Tencent, iqiyi, baidu, toutiao, momo and other standard advertising channels, but also supports customized platform docking. Because of this channel docking, it is difficult to connect data with cookies on the PC side, because the traffic side, distribution center and application activation belong to different layers or apps.

MMA standard: Identifies abnormal traffic

In this case, two solutions are provided. One is an extensible dynamic signature scheme for channel installation packages, which corresponds to the channel installation packages in real time during the downloading process. The other adopts the association algorithm to collect the behaviors of download, installation and activation by collecting LBS information and fingerprint information.

When it comes to mobile traffic distribution, it is inevitable to think of the problem of abnormal traffic. Now there are a lot of black industry or commercial interests. MMA standard is the general standard of the domestic mobile advertising industry, which makes a simple classification for the types of abnormal traffic. According to whether the anomaly itself is easy to be identified, its misjudgment and misjudgment rate can be simply divided into two parts: conventional invalid flow and complex invalid flow.

Flow cleaning

We are in the implementation of conventional invalid flow of these scenes at the same time, more attention is that the amount of brush this part, which is commonly known as our fifty fen party. They may complete the download, installation and subsequent activation and registration process of the entire APP manually by themselves through online delivery orders or some paid ones. We cooperated with Tencent’s internal team specialized in combating black industry to complete the identification of this piece of data.

Flow cleaning module, roughly divided into three parts, rule recognition, modeling and final application. This place is constantly improving with the technology of black industry, and there are many automatic means to imitate manual activation, and subsequent registration and other processes. So we are also introducing adversarial networks and complex neural algorithms to enhance this area.

Mutualistic service ecology

How can we make use of Tencent’s massive data and computing capabilities to empower enterprises and establish a mutually beneficial service ecosystem? Big data can help enterprises gain insight into users, industry changes and capital trends to assist in strategic layout and decision-making. However, there are many challenges in the application of big data, such as how to build the support of underlying capabilities and how to cultivate professional talent echelon. We have some practical experience to share here.

Data drives product operations

It can be seen that we have a lot of indicators now, and we have made a hierarchical treatment for them. There are some full indicators, which are related to some historical state characteristics, for example, some cumulative user retention problems, we use the offline analysis module. For the analysis of user clustering, it has its own calculation mode of multi-dimensional real-time analysis. There is also an online prediction model for potential churn, potential payers, and second-level real-time analysis based on monitoring metrics and diagnostic metrics.

Basic Technical Architecture

In order to realize the above functions and architecture, we also have a set of underlying support. The first one is the internal Docker system, which can provide us with strong underlying support ability. These platforms such as Hadoop and Spark are used for intermediate real-time computing. Finally, users are exposed to user portraits, key indicator data, roll-up and drum-down analysis of custom computing events, and a variety of access methods are carried out in the outer layer.

Empower enterprises with big data capabilities

After years of construction, Tencent has accumulated some components on storage, computing and scheduling platforms. The ability to build enterprise big data needs to go through three parts: data collection, modeling and analysis. We offer two solutions here. One is to visually lower the threshold using existing approaches. Or these platforms are open to the outside world and companies can collaborate or build their own.

Enterprises the construction of large data, involves data collection, algorithm and data by using the model perspective, as well as the data from several aspects, such as asset management we can directly use big data transition of a component, such as tencent mobile analysis and gold eye self-help statements, etc., can also be enterprise self-built platform, using a private cloud in our large data sets to the analysis of a complete set of process.

A single enterprise cannot satisfy all the demands of users, and there will be more collaborative cooperation in the future. We hope to use MTA as a bridge to accelerate the ability of enterprises in big data layout for these infrastructures, business platforms and data services. We can work together to build a healthy, perfect and intelligent data ecology, so that enterprises can benefit from big data and provide users with better services.

That’s all for my sharing. Thank you.

Q/A:

Q: Where is this user monitoring data generated? For example, the user monitoring is only for the users of this APP, or there may be some other support to improve the user monitoring results generated by this big data.

A: In the whole analysis process, we mentioned mobile portrait just now. Mobile portrait may involve more data generated by the whole ecology, including not only data inside Tencent, but also some data outside Tencent. In this section, there will be data other than those collected by APP. We also hope to use this platform to improve the entire service ecosystem, hoping that everyone can benefit from it.

Q: We also do some mobile terminal development. You put forward the concept of MID. There are many dynamic changes in MID, such as memory or fingerprint.

A: Maybe the existing system is not 100% guaranteed, but we try to improve it as much as possible. It is true that some terminal fingerprints are relatively fixed. For example, the fingerprint of H5 we used will be different on different terminals, but the same terminal and the same browser will get a unique result, so we can use this similar fingerprint information to calibrate the unique device.

Q: As mentioned earlier, this device may involve some permissions. The permissions of the terminal may be dynamically adjusted. Is this an incremental process? This device already has memory or some other information, but is it possible to make sure that this device is unique during the increment process? Like he added a fingerprint or some other access.

A: Permission is also an important consideration when we collect unique fingerprint characteristics of the device. If it’s easy to get, we’re more likely to adopt the feature. What you mentioned just now is that this permission may not have been obtained before and will be obtained later. At this time, some offline correction is needed. How can we associate devices of different MID’s and do a background de-duplication?

Q: For example, some hackers use simulators or other means to forge some data. With our existing technology, our backend can brush these traffic just like reading the original text on the wechat official account. For example, is there any way to identify the wechat official account? Is there a way to identify it, since it’s completely artificial to implement it?

A: Actually, it is mentioned that A scene of black production might employ some real people to complete the whole process of clicking and registering. In this case, we could do it. Why is that possible? This is because people who do this tend to swipe this order and then take other tasks, swipe this APP and then swipe another APP. Through the recognition of different apps, for example, if there is a hint on another APP, they can associate it with other apps corresponding to this device and also consider it as an abnormal state.

Another is to use Tencent’s own account system to record and track known accounts that have joined the black industry chain or some of their equipment characteristics. In a word, if there is no way to track and identify the cost of illegal production, the cost will be infinitely high, because they have to do some cost considerations, and where are their benefits, so we can now identify them.

For more information, please click the link below:

Tencent Mobile Analytics and Product Operations – Leon Yang. PDF

Question and answer

How to send a string from a client to a mobile server via Bluetooth?

reading

Hu Zerui: Mobile development as a service — Tencent cloud mobile development platform technology sharing

Gan Hengtong: Tencent pigeon mass mobile push service construction

Dong Chao: Build cloud storage service — mobile data storage and distribution

Has been authorized by the author tencent cloud + community release, the original link: https://cloud.tencent.com/developer/article/1138506?fromSource=waitui

Welcome toTencent Cloud + communityOr pay attention to the wechat public account (QcloudCommunity), the first time to get more massive technical practice dry goods oh ~