What exactly is front-end data monitoring monitoring?

Front-end data monitoring is generally divided into performance data monitoring and online anomaly monitoring. In this paper, the monitoring principle and method of these two pieces of data are sorted out and explained.

The performance data

Statistical package

• Code to monitor
  • Inject the monitoring code into the page, manually calculate the time difference or use the browser API for statistics.
• Tool monitoring
  • The statistical code is not injected into the page, and the performance data of the page is generally analyzed by virtual machines.

Before monitoring performance data, determine the time period from user access to page loading.

Time phase

Arrange all attributes in the order they are triggered: (See W3C Editor’s Draft for a more detailed explanation of the standard)

• NavigationStart: Timestamp of a previous page (not necessarily the same domain as the current page) unload in the same browser context, equal to the fetchStart value if there was no previous page unload

• UnloadEventStart: Timestamp of the previous web page (same domain as the current page) unload, 0 if there is no previous web page unload or the previous web page is in a different domain than the current page

• UnloadEventEnd: Corresponding to unloadEventStart, returns the timestamp when the callback function bound with the Unload event on the previous page has finished executing

• RedirectStart: The time when the first HTTP redirect occurs. The value is 0 only when there is a redirect within the same domain name

• RedirectEnd: The time when the last HTTP redirection is complete. The value is 0 only when there is a redirect within the same domain name

• FetchStart: The time when the browser is ready to fetch a document using an HTTP request, before checking the local cache

• DomainLookupStart: specifies the start time of DNS domain name query. If local cache (no DNS query) or persistent connection is used, the value is the same as the fetchStart value

• DomainLookupEnd: time when the DNS domain name query is complete. If local cache (no DNS query) or persistent connection is used, the value is the same as the fetchStart value

• ConnectStart: The time when the HTTP (TCP) connection is started. If the connection is persistent, this value is equal to the fetchStart value. If an error occurs at the transport layer and the connection is re-established, this is the time when the new connection is started

• ConnectEnd: the time when the HTTP (TCP) connection is established (the handshake is completed). If the connection is persistent, this value is equal to the fetchStart value. If an error occurs at the transport layer and the connection is re-established, this value is displayed as the time when the newly established connection is completed

  Note: The handshake is complete, including the establishment of the security connection and the SOCKS authorization

• SecureConnectionStart: HTTPS connection start time, if it is not a secure connection, the value is 0

• RequestStart: the time when the HTTP request starts reading the actual document (the connection has been established), including reading from the local cache, and the time when the new connection has been established if the connection error reconnects

• ResponseStart: The time at which the HTTP starts receiving the response (first byte is retrieved), including reading from the local cache

• ResponseEnd: The time when the HTTP response has been fully received (the last byte fetched), including reading from the local cache

• DomLoading: Parsing the time to render the DOM tree, document. readyState becomes loading, and readyStatechange events are thrown

• DomInteractive: The time when parsing the DOM tree is complete, document. readyState changes to interactive, and readyStatechange related events are thrown

  Note: only the DOM tree has been parsed, and the resources within the page have not been loaded

• DomContentLoadedEventStart: after DOM parsing is complete, the starting time of the resources within the web page loading, document DOMContentLoaded event time

• DomContentLoadedEventEnd: after DOM parsing is complete, within the web resource loaded time (JS) script loading has been completed, the end of the document DOMContentLoaded event time

• DomComplete: When the DOM tree is parsed and the resources are ready, document. readyState becomes complete and the readyStatechange event is thrown

• LoadEventStart: The load event is sent to the document, which is the time the LOAD callback function starts executing, value 0 if no LOAD event is bound

• LoadEventEnd: The time when the callback function of the load event completes execution. If no load event is bound, the value is 0

See the difference between DOMContentLoaded and load.

Statistical methods

Navigation Timing API

You can directly use the Navigation Timing interface to obtain the statistical starting point and the time of each stage during loading. Window. performance is a new API introduced by the W3C performance Group and currently supported by IE9 and beyond.

Common calculations:

• DNS query time: domainLookupEnd – domainLookupStart
• TCP connection time: connectEnd – connectStart
• Request Request time: responseEnd -responsestart
• Dom complete – domInteractive White screen time: responseStart – navigationStart
• Domready: domContentLoadedEventEnd – navigationStart
• Onload time (total download time) : loadEventEnd – navigationStart

Resource timing API

The Resource Timing API is used to collect static Resource timing information. For details, see W3C Resource Timing. Here we only introduce the performance. GetEntries method

Index and calculation method

indicators

• The overall index
  • DomComplete – timing. NavigationStart
  • NavigationStart Time to end DNS query timing. DomainLookupEnd – timing. NavigationStart Time
  • Timing. ResponseEnd – timing. NavigationStart
  • First rendering time timing. MsFirstPaint
• Stage indicators (in order)
  • Redirection time timing. RedirectEnd – timing. RedirectStart
  • UnloadEventEnd – timing. UnloadEventStart
  • Appcache time timing. DomainLookupStart – timing. FetchStart
  • DomainLookupEnd – timing. DomainLookupStart Timing
  • Connection time timing. ConnectEnd – timing. ConnectStart
  • Timing. ResponseEnd – timing. RequestStart
  • Request to DOM interactive time (including parsing HTML, non-defer scripts, and CSS) timing. DomInteractive – timing. ResponseEnd
  • DOM can be interactive to DOMReady time (including the time to process the script to defer) timing. DomComplete – * timing. DomInteractive
  • Load event time loadeventend-timing. LoadEventStart

See timing for details

Key indicators

• First paint Time – The time the user opens the page until the page begins to display content
• First screen Time – The time it takes for the user to open the page and for everything to appear on the first screen
• Dom Interactive – The time between the page opening and the user being able to click, type, and so on
• Total download time – the time between the user opening the page and when all the resources on the page are loaded and rendered

Determine the statistical starting point

We need to start counting as soon as a user enters a URL or clicks on a link, because that’s how long the user is waiting. If your user has a high percentage of high-end browsers, you can use the Navigation Timing interface directly to get the statistical starting point and time of each stage of the loading process.

Bad time

The white screen time is the time when the user first sees the content, also known as the first rendering time. The first version of Chrome has firstPaintTime to capture this time, but most browsers do not support it, so you have to find other ways to monitor it. A close look at the WebPagetest view analysis shows that the white screen time occurs near the end of the loading of the header resource, because the browser doesn’t actually render the page until the header resource is loaded and parsed. Based on this, we can approximate the blank screen time by obtaining the time when the header resources are finished loading. Although not exact, it takes into account the main factors that affect the white screen: first byte time and header resource load time.

How do you count header resource loads? We found that the head embedded JS usually need to wait in front of JS/CSS finish loading will perform, isn’t it in your browser in the head at the bottom of the add JS statistical resource loading head end points? You can test this with a simple example:

<! DOCTYPE HTML> <html> <head> <meta charset="UTF-8"/> <script> var start_time = +new Date; // Test time starting point, actual statistics starting point is DNS query </script> <! --> <script SRC = 3 seconds before the js returns --> <script SRC ="script.php"></script> <script> var end_time = +new Date; Var headtime = end_time - start_time; // Header resource loading time console.log(headtime); </script> </head> <body> <p> The page will be blank until the header resource is loaded </p> <p>script.php is simulated after setting 3s, </p> <p>script.php </p> </p> </body> </ HTML >Copy the code

It is found by test that the loading time of the statistical head is exactly similar to the downloading time of the header resource, and when a JS with a long execution time is changed, the statistics will wait until the JS execution is complete. Note This method is feasible (see the browser rendering principle and JS single thread for details).

In addition to the above methods, you can also use the Navigation Timing API to obtain the white screen time. The downside of this approach is poor browser compatibility and it cannot be used in browsers such as Safari.

 var firstPaint = 0;

  // Chrome
  if (window.chrome && window.chrome.loadTimes) {
      // Convert to ms
      firstPaint = window.chrome.loadTimes().firstPaintTime * 1000;
      api.firstPaintTime = firstPaint - timing.navigationStart;
  }
  // IE
  else if (typeof timing.msFirstPaint === 'number') {
      firstPaint = timing.msFirstPaint;
      api.firstPaintTime = firstPaint - timing.navigationStart;
  }
  // Firefox
  // This will use the first times after MozAfterPaint fires
  //else if(timing.navigationStart && typeof InstallTrigger ! = ='undefined') { // api.firstPaint = timing.navigationStart; // api.firstPaintTime = mozFirstPaintTime - timing.navigationStart; / /}Copy the code

The first screen time

Embed the processing logic before rendering the first screen, and use a timer to constantly check the image of the IMG node. Determine whether the picture is in the first screen and the loading is completed, find the time when the picture with the slowest loading time is completed in the first screen, and then calculate the first screen time. If there is no image on the first screen, if there is no image use domReady time. The statistical process is as follows:

First screen location Call API start statistics -> Bind the load event of all pictures in the first screen -> Determine whether the picture is in the first screen after the page is loaded and find the slowest loading one -> first screen timeCopy the code

This is simple statistical logic for synchronous loading, with a few additional points to note:

• If a page has an IFrame, you need to determine the load time
• GIF images may trigger load events repeatedly on Internet Explorer
• In the case of asynchronous rendering, the first screen should be calculated after the asynchronous fetch data is inserted
• CSS important background images can be counted by JS image URL request (browser will not reload)
• If there is no picture, JS execution time will be counted as the first screen, that is, the time when the text appears

This operation is optional

Domready time can be counted by default, since event operations are usually bound at this time. For JS with modular asynchronous loading, the loading time of important JS can be actively marked in the code, which is also the statistical way of product indicators.

performance.timing.domInteractive - performance.timing.navigationStart
Copy the code

Always download

Total Download Time By default, you can count the onload time to collect statistics about the time it takes to load all synchronized resources. If you have a lot of asynchronous rendering on your page, you can use the time it takes for the asynchronous rendering to complete as the total download time.

performance.timing.loadEventStart- performance.timing.navigationStart
Copy the code

Online abnormal

Interface Abnormal Monitoring

The method is very simple, in the case of interface error actively report errors.

JS code exception monitoring

• Try catch catch

This scenario requires developers to write code that uses try… Catch, to send exception information to the interface when an exception occurs:

Try {// possible exception code segment}catch(e){Copy the code

• Window. The onerror capture

This approach does not require the developer to write a lot of tries in the code… Catch, by adding onError listener to window, error information can be caught when JS exceptions occur. Syntax and running exceptions can be caught. But window.onerror must be placed before all js files to ensure that all exception messages are caught.

• Catch cross-domain JS file exceptions

At present, it can be said that basically all Web products set access-Control-allow-Origin: * response header for static resources such as JS/CSS /image on the server side, that is, Allow cross-domain request. In this environment, we simply add a Crossorigin attribute to the script tag that requests the cross-domain resource:

<script src="http://static.toutiao.com/test.js" crossorigin></script>
Copy the code

This way, when an exception occurs in the test.js file of a foreign domain, the onError listener of the current domain can catch the detailed exception information.

The resources

1. Front-end performance – Monitoring starts
2. Performance – Front-end Performance monitoring tool
3. Window. Performance Investigation (Transfer)
4. Research first screen time? You need to know these details first
5. Performance – Monitoring web page and application performance
6. Build front-end performance monitoring system in 7 days
7. DOMContentLoaded vs. Load