Front end burying point and monitoring scheme

Buried data acquisition scheme

What is a burial point

Buried point, whose scientific name is Event Tracking, mainly refers to the relevant technology and implementation process of capturing, processing and sending specific user behaviors or business processes. Buried point is a technical term in the field of data, but also a common name in the field of Internet.

Buried point is the basis of product data analysis, which is generally used for feedback of recommendation system, monitoring and analysis of user behavior, statistical analysis of new function or operation effect, etc.

Buried points contain two important concepts: events and attributes.

• Event: What happens in the application, such as a user action, system event, or system error. For example, if you take a product, it contains the following events: enter_page (enter page), leave_page (leave page).
• Attributes (PARAM) : Attributes, such as language preference or geographic location, defined to describe the segmentation of a user group. For example, the “Enter practice” event contains the following event attributes: enter_from (from which page), class_id (course ID), and so on.
• Attribute value: The dimension of the attribute, that is, the specific dimension when the behavior is triggered. For example, enter_from: home, system, and so on.

The mainstream plan

• Traceless buried point (full buried point), using the browser or APP built-in monitoring method to collect users’ browsing page, clicking and other behaviors, generally used for coarse granularity data analysis, such as the company’s SLARDAR
  • Advantages:
    • Access is simple, almost non-intrusive, and requires no additional development costs
    • The collection of user actions is so complete that there are few omissions
  • Disadvantages:
    • Data is noisy and will be collected whether it is useful or not
    • Unable to customize burial points, unable to collect specified events and business attributes
    • Less information is available to DA
• Code burials, where front-end developers customize listening and collecting in the code
  • Advantages:
    • Can be precisely buried, with a clear event identification
    • Business attributes are very rich
    • Buried point trigger mode can be defined flexibly
    • DA is more convenient and accurate to use
  • Disadvantages:
    • The workload is large, and the code is very intrusive, and the later maintenance is not very convenient
• Buried point SDK. The SDK exposes the interface for reporting buried points to the outside, and the developer is unaware of the monitoring and collection process.
  • Advantages:
    • Business development only needs to focus on event identification, business attributes, and so on
    • Take into account the advantages of non-trace burying point and code burying point
  • Disadvantages:
    • For the moment

Common buried point properties

Usually, the front end calculates buried points according to the page dimension, and the common event attributes are as follows:

Common buried event

Performance data collection solution

Currently, most performance indicator data comes from window.performance API.

Performance.timing

Common Performance Indicators

FP

The First Paint (FP) indicator usually reflects the white screen time of a Web page. The white screen time reflects the network loading performance of the current Web page. If the loading performance is good, the white screen time is shorter, the user waits for the content, and the loss probability is reduced.

The index can be achieved by the performance. GetEntriesByType (‘ paint ‘) method to get PerformancePaintTiming API provides some information, to find the name for the first – the object of the paint, The indicator data of FP is described, as shown in the figure below:

FCP

First Contentful Paint (FCP) is the time when the First content is rendered. In performance statistics, the period from the time when a user accesses a Web page to the time when the FCP is considered as the time when no content is created. Generally, FCP >= FP.

The index can be achieved by the performance. GetEntriesByType (‘ paint ‘) method to get PerformancePaintTiming API provides some information, Select first- contentfulful -paint, which describes the FCP indicator data, as shown in the following figure:

FMP

First Meaningful Paint (FMP) is the time when Meaningful content is drawn for the First time. When the overall page layout and text content is fully rendered, the First Meaningful content is considered to have been drawn. Therefore, FMP measures the time for users to see the main content of a web page, which is an important indicator of user experience.

One of the most accepted ways of calculating FMP in the front-end industry is “the time it takes to draw a page after the biggest layout change during loading and rendering.” MutationObserver can be used to monitor the overall DOM change of each page and trigger the callback of MutationObserver. When the callback calculates the change score of the current DOM tree, the moment when the score changes most dramatically is the time point of FMP.

TTI

TTI (Time To Interactive) : The Time between the page loading and the page being fully Interactive. When a page is in a fully interactive state, the following three conditions are met:

1. The page already displays useful content.
2. The event response function associated with the visible element on the page has been registered.
3. The event response function can be executed within 50ms after the event occurs.

Resource loading Indicators

Window. The performance. GetEntriesByType (‘ resource ‘) returns the current page loading all of the resources (js, CSS, img…). Can be used to collect static resource performance data.

The main types are: Script, link, IMG, CSS, XMLHttprequest, Beacon, FETCH, and Other. PerformanceResourceTiming – Web APIs | MDN

Calculation methods of other indicators

Error data acquisition scheme

There are three types of errors that can be caught:

• Resource loading error, passaddEventListener('error', callback, true) Catch a resource load failure error in the capture phase.
• Js execution error, passwindow.onerror Catch JS errors.
  • Cross-domain scripts will give a “Script Error.” message and will not get the specific Error message and stack information. You need to add it in the script tagcrossorigin="anonymous"Property, and the resource server needs to add CORS related configuration, such asAccess-Control-Allow-Origin: *
• Promise error, passaddEventListener('unhandledrejection', callback)Catch a promise error, but there is no information about the number of rows, columns, and so on. You can only throw the error message manually.

// During the capture phase, a resource load failure error is caught
addEventListener('error'.e= > {
    const target = e.target
    if(target ! =window) {
        monitor.errors.push({
            type: target.localName,
            url: target.src || target.href,
            msg: (target.src || target.href) + ' is load error'.time: Date.now()
        })
    }
}, true)

// Listen for js errors
window.onerror = function(msg, url, row, col, error) {
    monitor.errors.push({
        type: 'javascript'.row: row,
        col: col,
        msg: error && error.stack? error.stack : msg,
        url: url,
        time: Date.now()
    })
}

// Listen to the promise error
addEventListener('unhandledrejection'.e= > {
    monitor.errors.push({
        type: 'promise'.msg: (e.reason && e.reason.msg) || e.reason || ' '.time: Date.now()
    })
})
Copy the code

Data reporting scheme

In this scenario, two issues need to be considered:

• If the data reporting interface and the service system use the same domain name, the browser has restrictions on the number of concurrent requests. Therefore, there is a possibility of network resource competition.
• Browsers typically ignore asynchronous Ajax requests during page unload, and if data requests are required, they typically delay page unload by creating synchronous Ajax requests in either unload or beforeUnload events. From the user side, that is, the page hops are slow.

Beacon

It can be seen that, except for Internet Explorer, the mainstream modern browsers have a very high support rate for Beacon. Beacon-mdn document

The Beacon interface is used to schedule asynchronous non-blocking requests to the Web server.

• Beacon requests use HTTPPOSTMethod, and no response is required.
• The Beacon request ensures that initialization is completed before the page triggers an UNLOAD.

In layman’s terms, a Beacon sends data asynchronously to the server and ensures that the request is sent before the page offload is complete (to solve the problem that Ajax page offload will terminate the request). The usage method is as follows:

navigator.sendBeacon(url, data);
Copy the code

The data argument is optional and can be of type ArrayBufferView, Blob, DOMString, or FormData. This method will return true if the browser successfully enqueued the Beacon request to be sent, false otherwise

When using Beacon, the background needs to use the POST method to receive parameters. Considering the cross-domain problem, the background also needs to transform the interface and configure CORS. Also, the request header must satisfy the cerS-Safelisted request-header, Content-type must be Application/X-www-form-urlencoded, multipart/form-data, or Text /plain.

type ContentType = 'application/x-www-form-urlencoded' | 'multipart/form-data' | 'text/plain';

const serilizeParams = (params: object) = > {
    return window.btoa(JSON.stringify(params))
}

function sendBeacon(url: string, params: object) {
  const formData = new FormData()
  formData.append('params', serilizeParams(params))
  navigator.sendBeacon(url, formData)
}
Copy the code

Image

Compatibility issues with sendBeacon are inevitable, but you can take advantage of the fact that most browsers load images before the page is unloaded by adding an IMG file to the page to report data.

function sendImage(url: string, params: object) {
  const img = new Image()

  img.style.display = 'none'

  const removeImage = function() {
    img.parentNode.removeChild(img)
  }

  img.onload = removeImage
  img.onerror = removeImage

  img.src = `${url}? params=${serilizeParams(params)}`

  document.body.appendChild(img)
}
Copy the code

Because IMG images are get requests, different servers have limits on the length of URIs. When the length exceeds the limit, HTTP 414 errors will occur. Therefore, it is necessary to pay attention to the reporting frequency and reduce the number of attributes uploaded at one time.

HTTP 1.1 defines Status Code 414 Request-URI Too Long for the cases where a server-defined limit is reached further details on RFC 2616. For the case of client-defined limits, there is no sense on the server returning something, because the server won’t receive the request at all.

Compatible with the plan

SendBeacon is preferred, and Image is used as fallback.

function sendLog(url: string, params: object) {
    if(navigator.sendbeacon) {sendBeacon(url, params)}else {
        sendImage(url, params)
    }
}
Copy the code

The related documents

• In-depth understanding of front-end monitoring principles
• Front-end buried point data collection and reporting scheme