In the online project, it is necessary to make statistics on the user behavior and usage of the product, so as to understand the user group from the perspective of the user and the product, so as to upgrade and iterate the product and make it more close to the users. User behavior data can be obtained through front-end data monitoring. In addition, the front-end needs to implement performance monitoring and exception monitoring. Performance monitoring includes the first screen loading time, white screen time, HTTP request time, and HTTP response time. Exception monitoring includes front-end script execution errors.

There are three steps to realize front-end monitoring: front-end burying and reporting, data processing and data analysis. This paper designs a suitable scheme for the whole front-end monitoring. The main contents of this paper are as follows:

  • Why do you need front-end monitoring
  • Summary of common front-end burial schemes
  • Front-end burying scheme selection and front-end reporting scheme design
  • Design of visual display system of front-end monitoring results

The address of the original, in my blog: https://github.com/forthealllight/blog/issues/23

If it helps, your star is the best encouragement to me ~

Why is front-end monitoring needed

The purpose of front-end monitoring is to:

Obtain user behavior and track the use of the product in the user side, and point out the direction of product optimization based on the monitoring data.

Front-end monitoring can be classified into three categories: data monitoring, performance monitoring, and exception monitoring. Let’s understand one by one.

(1) Data monitoring

Data monitoring, as the name suggests, is listening to user behavior. Common data monitoring methods include:

  • PV/UV:PV(Page View), the number of page views or clicks. UV: Refers to the number of people visiting a site or clicking on a news item from different IP addresses
  • The length of time a user spends on each page
  • What entry does the user use to access the page
  • Behavior triggered by the user in the corresponding page

Statistics of these data is meaningful, for example, we know the user source channel, can promote the promotion of the product, know the user in each page stay time, can stay longer pages, increase advertising push, and so on.

(2) Performance monitoring

Performance monitoring refers to monitoring the performance of the front end, which mainly includes monitoring the web page or the user experience of the product. Common performance monitoring data includes:

  • First screen loading time for different users, different models and different systems
  • Bad time
  • Response time of HTTP requests
  • Overall download time of static resources
  • Page render time
  • Page interactive animation completion time

The results of the performance monitoring can be used to show the performance of the front end, which can be used to optimize the performance of the front end, such as compatible with the animation of the lower browser version, faster first screen loading, etc.

(3) Abnormal monitoring

In addition, the front-end code of the product can also experience exceptions during execution, so you need to introduce exception monitoring. Timely reporting of anomalies can avoid online faults. Although most exceptions can be caught with try catches, memory leaks and other occasional exceptions are difficult to catch. Common exceptions that need to be monitored include:

  • Exception monitoring for Javascript
  • Style missing exception monitoring

Two, common front-end buried point scheme summary

The role of front-end monitoring was introduced in the previous section, so how to achieve front-end monitoring? The steps to achieve front-end monitoring are: front-end burying and reporting, data processing and data analysis. The first step is the front-end burying and reporting, which is the data collection phase. The richness and accuracy of the data collected will affect the results of determining the effectiveness of the product line.

At present, there are three common front-end burying methods: code burying, visual burying and non-trace burying. The following is an introduction to each of the burial point method.

(1) Code burial point

Code burying point refers to embedding point in the form of embedded code. For example, to monitor the user’s click event, when the user clicks, a piece of code will be inserted to save the listening behavior or directly pass the listening behavior to the server in a certain data format. In addition, such as the need to statistics product PV and UV, need to be in the initialization of the page, send user access information.

Benefits of code burial:

  • Can at any time, precisely send or save the data information needed.

Disadvantages:

  • The workload is large, each component buried point needs to add the corresponding code

(2) Visual burial point

By means of visual interaction, instead of burying the code. The business code is separated from the buried point code, and a visual interactive page is provided. The input is the business code. Through this visual system, the buried point events can be added in the business code.

Visual burials sound great, but they’re actually not that different from code burials. That is, a system for manually inserting code burials.

Disadvantages:

  • Visual burials there are limited controls that can be buried and cannot be customized manually.

(3) there is no point

No buried point does not mean no buried point, but all buried point, any event on the front end is bound to an identifier, all events are not recorded. By periodically uploading record files and cooperating with file parsing, we can parse out the data we want, and generate visual reports for professionals to analyze, so as to realize “no buried point” statistics.

It is also very simple to realize the non-buried point from the language level, such as from the PAGE’s JS code, find out the dom is bound to the event, and then the full buried point.

Advantages of no buried point:

  • Due to the collection of full data, there is no need to pay attention to the logic of burial point in the product iteration process, and there will be no phenomenon of missing or false burial

Disadvantages:

  • Collecting full data without buried point increases pressure on data transmission and server
  • The data required by each event cannot be flexibly customized

Three, front-end buried point scheme selection and front-end reporting scheme design

In the first chapter, the front-end monitoring information is introduced. In the second chapter, the common way of front-end burial point is introduced. This paper will customize our burial point and report scheme according to the needs.

(1) Monitoring data

First we need to identify a product or web page that generally needs to be monitored and reported. The monitoring is divided into three stages: the user enters the homepage, the user interacts within the webpage and the error is reported during the interaction. The data to be monitored and reported at each stage is shown in the figure below:

(2) Burial scheme

In the actual project, considering the flexible customization of reported data and the reduction of data transmission and server pressure, the common way is to bury the code in the case of few burial points.

Take the user entering the home page as an example. After the home page rendering is completed, we will send the data related to the event type and type to the server to inform the monitoring information of the home page.

(3) Reporting period and data type

If there are not many buried events, the report can be carried out constantly. For example, the user interaction events can be monitored and the type of events triggered by the user can be reported immediately after the event is triggered. If there are many buried events, or the internal interaction of the web page is frequent, you can cache the reported information in the local storage and then report the information periodically.

Next, it determines the data to be reported by the burial site. The reported information includes user personal information and user behavior. The main data can be divided into:

  • Who: AppID (system or application ID),userAgent(user’s system, network and other information)

  • When: timestamp(timestamp reported)

  • From where: currentUrl(user’s currentUrl), fromUrl(which page to jump to), type(type of event to report),element(element to trigger event to report)

  • What: reported user-defined extension data data:{}. The extended data can be customized based on requirements, such as uid information

The data objects to be reported are as follows:

{-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- report interface itself provide -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- currentUrl, fromUrl, timestamp, userAgent: {OS, netWord. } -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- business code configuration and custom report data -- -- -- -- -- -- -- -- -- -- -- -- type, appid and element, data: {uid, uname}}Copy the code

(4) Burial sites and reporting examples

Let’s take the first screen loading event as an example. The DOM provides the document DOMContentLoaded event to listen for the DOM to mount, and the Window Load event to listen for all the resources on the page to complete the rendering.

<script type="text/javascript">
  var start=Date.now();
  document.addEventListener('DOMContentLoaded', function() {
     fetch('some api',{
         type:'dom complete',
         data:{
           domCompletedTime:Date.now()-start
         }
     })
  });
  window.addEventListener('load', function() {
     fetch('some api',{
         type:'load complete',
         data:{
           LoadCompletedTime:Date.now()-start
         }
     })
  });
</script>
Copy the code

(5) communication encryption of front and back ends of the front-end buried point system

In the communication between the front and back ends of reported data, the encryption mechanism needs to be negotiated with the server and the OpenSSL library is used to achieve the encryption. OpenSSL is a widely used encryption algorithm. The front-end can use node’s crypto module.

Crypto.createhash () creates an instance of the hash. The hash algorithm is as follows:

  • md5

  • sha1

  • sha256

  • sha512

  • ripemd160

Take the SHA256 algorithm as an example:

const str="123445"; // Const hash=crypto.createHash('sha256'); // Specify the encryption algorithm hash. Update (STR); Const result=hash. Digest ('hex'); // Convert to hexadecimalCopy the code

Iv. Design of visual display system for front-end monitoring results

After receiving the information reported by the front end, the back end needs to display the results of data analysis in a visual way after data analysis and processing.

Secondary development can be carried out on the basis of the background system Ant-Design-Pro in open source, and the information should be clearly displayed first. The information presented includes both the individual user and the overall application.

For a single user, the following monitoring information needs to be displayed:

  • The number of times that each buried point event is triggered by a single user during the interaction
  • A single user, in a certain period of time, to access the entry source of this page
  • Time spent on each subpage by a single user

The information to be displayed for all users is:

  • The PV and UV of a web page in a certain period of time
  • Device and operating system analysis of all user visits to web pages
  • Source analysis of entries to this web page within a certain period of time
  • The total number of times that each buried event is triggered during the interaction process when all users visit this web page
  • All users in the access to this page, the page reported abnormal collection

Delete function set:

  • Time filter: offers today (00 to current time), week, month and year
  • User selection: Provides the statistics about the behaviors of deleted users based on user ids
  • Delete and select devices: Delete and select the overall display information of different systems