background

Before this solution, the company used privatized sensor deployment to meet the project error monitoring, business event burying point and other functions.

but

  1. Secondary development is limited as it gradually fails to meet business needs. (Main reason)
  2. Capital investment is relatively large.

So there is the front end of the common buried point scheme.

The whole process

These include:

  • Front-end unified SDK: used for H5, wechat applets, Alipay applets front-end buried point
  • Buried Node service: processes buried requests and sends Kafka events for monitoring platform/data platform/AD back consumption.

The front-end SDK

The front-end monitoring SDK is mainly responsible for some behavior data and abnormal information generated by the front-end itself, including the following parts:

  • Buried data (business data)
  • Abnormal data (front-end error, etc.)
  • Indicator data (performance indicators and user-defined service indicators)
    • Performance indicators: FP/FCP, FMP

SDK

In order to maintain, expand and process data, the whole SDK adopts the responsibility chain mode, and the data flow of each event will be processed in each stage according to the whole responsibility chain flow. It includes automatic data collection and manual buried data reporting.

Initialize the

The initialization process includes – initializing multiple automatic collection collectors-initializing data processing links – initializing the Runtime based on the current environment

runtime

Runtime provides apis and global properties that are available to the current environment, such as the env, store, and request methods.

The data flow

SDK based on pipeline to achieve data processing and reporting.

  • SampleRate: Indicates sampling processing. The default value is 1 and you can sample data based on the data type
  • PresetProperty: Reserved data processing, used to complete some preset fields, such as clientId, in the original data to be reported
  • OfflineStorage, which stores data offline
  • ReportStrategy, reporting policy that meets two conditions: cache overflow and timeout expiration
  • RootReport is reported and processed each time. If the report fails, the system retries twice by default. After the report succeeds, the timer is checked

Offline storage

To prevent the loss of buried data, data is first backed up in localStorage (small programs are stored in storage). The first five items of backup data are taken out and sent each time, and deleted after the report is successful.

report

The reported request is simply encapsulated based on XHR and Navigator.sendBeacon, which is used to send leavePage events. To save requests, saving and reporting is an asynchronous process that is not reported immediately after saving, but several seconds later.

Unique user identification

In order to count users, the unique identifier clientId will be carried during each data processing, and the user token will be carried for login users. User data platform binds user information.

standard

Performance indicators

Performance metrics collection should be implemented to keep the page less intrusive. With the development of front-end specifications, Performance series apis are becoming more and more powerful. In implementation, we give priority to native API collection.

FP/FCP

First paint (FP): : first paint time, understood as the drawing time of the first pixel of the browser. FCP (First ContentFull): the drawing time of the first content and the first visible element, including text, picture, and canvas.

From the user’s perception, fp time is close to the blank screen time, and FCP is mainly affected by the structure of index.html DOM and loading. Therefore, we mainly take FP as the reference index. Fp/FCP can be directly collected using the Paint Timing API. The test data shows that fp is close to the blank screen time.

FP indicators are collected in the following manner based on the compatibility of browsers

  • window.performance.getEntries
  • window.performance.timing

FMP (fitst meanningful paint)

The point in time at which the core elements of the page are rendered is the first valid rendering. Different sites, what the core elements are different. Closer to the first screen time.

Currently, there is no standard API support for FMP, and the accuracy of FMP collected in Google Lighthouse is only about 70%. However, it can be seen that the mainstream front-end monitoring basically regards FMP as an important performance indicator. We can extract key features according to our own business characteristics, and then adjust the weight algorithm to achieve indicators that are closer to the real data.

FMP algorithm is mainly based on Element Timing and weight

conclusion

This chapter mainly introduces the front-end monitoring in the front-end buried point SDK related things. Performance counter collection will be introduced later.

More articles: Renzhaosy’s documentation library