The author is Yang Yunxin

The head image is from Carlos Muza on Unsplash

1, the preface

Before beginning the text, introduce related concepts, which can be skipped for those familiar with it.

Front-end burying point: a method of collecting product data. The purpose of this method is to report relevant behavioral data, and relevant personnel analyze the use of the product on the user side based on the data, and assist product optimization and iteration according to the analysis results.

BI: Business intelligence, the department within a company that does data analysis.

2, the background,

Now that the traffic dividend is gradually disappearing, data collection, analysis and fine operation are more important. Therefore, burying point is very common in Internet products, which can better assist us to iterate and improve product functions.

Usually we need to develop and complete the buried requirements after completing the basic business requirements. So what we pursue is simple and quick to do a good job of burying points, and will not take up too much of our energy. However, the reality is not so good. At present, our team has some pain points in front burying points:

  • When constructing the buried point field, it is necessary to spliced several fields into one according to the rules of BI, which takes time and effort and risks mistakes.
  • Some exposure scenarios are hard to hit such as: paging list, virtual list; Their exposure buried point implementation is more cumbersome;
  • Logic reuse problem: in particular, the exposure related points need to do extra processing in the business code, so the logic reuse is very difficult, the intrusion of the existing code is also very serious;

Therefore, we need a suitable burial point solution to solve our current problems, improve our development efficiency, and no longer worry about the burial point.

3. Common front end burying point scheme

We have conducted some research on several embedding schemes in the current market, and there are three conventional schemes:

Manual code burying point: Data is manually reported after a user triggers an action

  • Advantages: It is the most accurate and can meet many customization requirements.
  • Disadvantages: Buried logic is coupled to business code, which is not conducive to code maintenance and reuse.

Visual burying point: Configure the collection node through the visualization tool to specify the elements and attributes that you want to monitor. The core is to look up the DOM and bind events, and the industry is better known as mixpanels

  • Advantages: It can be configured on demand without generating a lot of useless data like full buried sites.
  • Disadvantages: Difficult to load some runtime parameters; When the page structure changes, partial reconfiguration may be required.

No buried point: also called “full buried point”, the front-end automatically collects all events and reports buried point data, and filters out useful data during data calculation at the back-end

  • Advantages: Collect all the behaviors of users on the end, very comprehensive.
  • Disadvantages: a lot of invalid data, large amount of reported data.

4. Buried point scheme

After investigating these schemes, I think the above schemes are not entirely suitable for us. What we need is accurate and fast burial point, while decoupling the code of burial point and business logic, and our sound street mobile station can migrate to our new burial point library in a relatively smooth way. Based on our current technology stack React, as well as the current situation, operation and product requirements, we decided to adopt the declarative component-based buried point + buffer queue scheme. Here is our general idea.

  • In order to solve the problem of embedding code and business logic coupling, we think it can be handled in the view layer, embedding can be divided into two categories, click and exposure embedding. We can abstract two components to handle these two scenarios separately.

  • Sliding rapidly in some scenarios, frequent click will hit a lot of points in a short time, causing frequent interface call, this is to be avoided in the mobile terminal, we introduced the buffer queue for this scene, first level information into the queue, through regular tasks in batches reported data, according to different types of report point also can be applied to different frequency.

  • At present, manual stitching is used for some fields, such as _MSPm2 and other related general fields defined by BI, which can be handled uniformly in the library, which is not error-prone and convenient for later expansion.

  • For page-level exposure, we can automatically register the events related to the page exposure after the initialization of the buried point library, and do not need to care about the user.

  • Manage the configuration of burial points by page dimension

    • Our site is a homogeneous application, with our architecture more fit
    • Clearer and easier to maintain
    • At present, this scheme is also used to manage the migration costs will be smaller

5. Key nodes

5.1 Process Combing

There is a problem, can library hasn’t been initialized, some points have been produced, such as exposure, if this time to generate the corresponding points into the buffer queue, is invalid because no loaded into the pit at information, configuration parameters, etc., so in view of the point information produced under this scenario, we open a new queue storage, wait until initialization is complete to deal with;

Flow chart:

5.2 Click the embedding point

We started thinking about providing a component that wraps around the DOM element that needs to be buried, maybe the component, and then binds the child element with the click event, and then does the burying when the user fires the event.

We had to bind click events to the DOM, but we didn’t want to introduce extra DOM elements because that would increase the dom hierarchy and cause problems for the user, leaving us with props. Children, So we recursively went to the children of the TrackerClick component, found the outermost DOM element, and required that TrackerClick must have a container element underneath it, which we did.

export default function TrackerClick({ name, extra, immediate, children, }) {
    handleClick = () = > {
        // todo append queue
    };

    function AddClickEvent(ele) {
        return React.cloneElement(ele, {
            onClick: (e) = > {
                constoriginClick = ele.props.onClick || noop; originClick.call(ele, e); handleClick(); }}); }function findHtmlElement(ele) {
        if (typeof ele.type === 'function') {
            if (ele.type.prototype instanceof React.Component) {
                ele = new ele.type(ele.props).render();
            } else{ ele = ele.type(ele.props); }}if (typeof ele.type === 'string') {
            return AddClickEvent(ele);
        }
        return React.cloneElement(ele, {
            children: findHtmlElement(ele.props.children)
        });
    }
    
    return findHtmlElement(React.Children.only(children));
}


// case1
<TrackerClick name='namespace.click'>
    <button>Click on the</button>
</TrackerClick>

// case2
<TrackerClick name='namespace.click'>
    <CustomerComp>
        <button>Click on the</button>
    </CustomerComp>
</TrackerClick>
Copy the code

It’s easy to use, and it serves our purpose. However, some problems have been found in our practice. For example, the user does not know the implementation details of the container, there may be no container package in the container, and the react. Fragment may be used to cause some unpredictable behaviors and increase the DOM structure layer invisibly (although we did not introduce, But we’re telling users, you better have a Container.

We are also reflecting on the rationality of this scheme. Although it brings convenience in use, it brings uncertainty. After some discussion we decided to leave the binding to the component consumer. We just had to explicitly tell him which methods were available, which was deterministic. The user only needs to bind the triggered callback to the corresponding event.

The transformation is as follows:

<TrackerClick name='namespace.click'>
{
    ({ handleClick }) = > <button onClick={handleClick}>Click on the hole position</button>
}
</TrackerClick>
Copy the code

5.3 Exposure burying point

Exposure for us has always been more troublesome, let’s take a look at some of the requirements of exposure buried point:

  • An exposure is legal only when the element appears in a certain proportion of the window
  • An element is not exposed until it stays in the window for a certain amount of time
  • Count the exposure time of elements

From the perspective of the front end, it would be complicated to implement the three points, and some paging and virtual list scenarios would be more complicated. Therefore, we investigated IntersectionObserver with these problems.

IntersectionObservers calculate how much of a target element overlaps (or “intersects with”) the visible portion of a page, also known as the browser’s “viewport”

IntersectionObservers calculate how much target elements overlap (or “intersect”) with the visible parts of a page, also known as the browser’s “viewports.”

const intersectionObserver = new IntersectionObserver(function(entries) {
  // If intersectionRatio is 0, the target is out of view
  // and we do not need to do anything.
  if (entries[0].intersectionRatio <= 0) return;

  console.log('Loaded new items');
}, {
    // Exposure threshold
    threshold: 0
});
// start observing
intersectionObserver.observe(document.querySelector('.scrollerFooter'));
Copy the code

The above is an example of MDN, so we can know when an element enters and when it leaves the viewport. We can implement all three requirements indirectly.

After investigation, we have corresponding intersection-observer polyfill in ability to meet our needs and compatibility. For paging and virtual lists, we only need to focus on the list items we need to observe, so we need to implement a high-performance ReactObserver component that provides intersection-Observer capabilities and provides corresponding callbacks. How to implement a high-performance Observer is not covered here.

Here are two ways to expose a component’s bound DOM

// case1: bind dom directly
render() {
    return (
        <div styleName='tracker-exposure'>
            {
                arr.map((item, i) => (
                    <TrackerExposure
                        name='pagination.impress'
                        extra={{ modulePosition: i + 1 }}
                    >
                        {({ addRef }) => <div ref={addRef}>{i + 1}</div>}
                    </TrackerExposure>))}</div>
    );
}


// case2: custom component
const Test = React.forwardRef((props, ref) = > (<div ref={ref} style={{
        width: '150px',
        height: '150px',
        border: '1px solid gray'
    }}>TEST</div>))render() {
    return (<div styleName="tracker-exposure">
        {
            arr.map((item, i) => <TrackerExposure
                name="pagination.impress"
                extra={{ modulePosition: i + 1}} >
                {
                    ({ addRef }) => <Test ref={addRef} />
                }
            </TrackerExposure>)}</div>)}Copy the code

In practice we only provide an addRef to get the DOM to do the listening work and leave the rest to the library, making exposure so easy. Based on the above three requirements, we provide the following configurations:

  • Threshold: Exposure threshold. What percentage of the window triggers when element appears
  • ViewingTime: exposure duration of an element, used to determine whether it is a compliant exposure
  • Once: Whether to repeatedly hit the exposure point

5.4 Runtime Parameters

General fixed parameters are managed in the Config configuration file. Of course, there are also some runtime parameters, such as userId and modulePosition, etc. For this scenario, extra props are delivered through the props of the component and assembled inside the component. You only need to pass in the corresponding service field.

5.5 appendQueue

In cases where we can’t bind events to the DOM, such as native elements like audio and video, and business components that have a very high level of encapsulation, we provide a callback that adds points to the buffered queue.

appendQueue({
    name: 'module.click'.action: 'click'.extra: {
        userId: 'xxx',}})Copy the code

5.6 Scheduled Task

Our design is that all generated points are put into a buffer queue and reported via a scheduled task. The current strategy is to report frequency of click class 1000ms, exposure class 3000ms, of course, this interval is not imaginary, through discussion with the algorithm and BI, taking into account the front-end requirements and real-time requirements of the algorithm, the two values are also supported configuration.

For the time interval of scheduled tasks, we take the greatest common divisor of click and exposure reporting frequency to reduce the number of executions.

5.7 Page Exposure

During initialization, we will determine whether we need to process page exposure according to the agreed fields in the configuration file.

The key to page exposure is the time to collect page exposure. Browser page life cycle standards and specifications have not been established for a long time, and various manufacturers do not support them very well. Refer to visibilitychange event in Chrome page life cycle as the time to collect page exposure.

Browser compatibility for VisiblityChange

6, use,

import Tracker, {
    TrackerExposure,
    appendQueue
} from 'music/tracker';

const generateConfig = () = > ({
    opus: {
        mspm: 'xxxx091781c235b0c828xxxx'
    },
    'playstart': {
        mspm: 'xxxx91981c235b0c8286xxxx'._resource_1_id: ' '._resource_1_type: 'school'
    },
    viewstart: {
        mspm: 'xxxxd091781c235b0c828xxx'.type: 'page'
    },
    viewend: {
        mspm: 'xxxx17b1b200b0c2e3xxxxxx'.type: 'page'._time: ' '}});export default Tracker;
export {
    generateConfig,
    TrackerExposure,
    appendQueue
};

Copy the code
import React, { useEffect, useState } from 'react';
import Tracker, { generateConfig, TrackerExposure, appendQueue } from './tracker.js';

const Demo = () = > {
    const [opusList, setOpusList] = useState([]);

    useEffect(() = > {
        Tracker.init({
            common: {
                osVer: 'xxx'.activityId: 'xxx',},config: generateConfig()
        });

        // fetch opuslistsetOpusList(opus); } []);const handleStart = () = > {
        appendQueue({
            name: 'playstart'.action: 'playstart'
        });
    }

    return<> { opusList.map(opus => <TrackerExposure start="opus" startExtra={{opusId: Opus.id}} threshold={0.5}> {({addRef}) => <div ref={addRef} >{opus.name}</div>}</ TrackerExposure>)}< Player onStart={handleStart}> <>; }Copy the code

7,

We have carried on the migration in the Yinjie mobile station, used in a number of operational activities, to achieve our expected goals; In terms of efficiency improvement, the buried point library takes care of the time-consuming part. All we need to do is to put the pit information from the buried point platform into the configuration file, and use the corresponding components for business development. It is almost not too much cost, and also for code reuse and maintenance.

AppendQueue is used much more frequently than TrackerClick because most element click events have their own callback function, but TrackerClick is decouple from the buried code and business code. This also depends on the actual situation to choose.

8. Reference materials

  • MDN/IntersectionObserver
  • react-intersection-observer
  • page-lifecycle

This article is published from netease Cloud Music big front end team, the article is prohibited to be reproduced in any form without authorization. Grp.music – Fe (at) Corp.Netease.com We recruit front-end, iOS and Android all year long. If you are ready to change your job and you like cloud music, join us!