Design and package a front-end burying point reporting script, and gradually think about optimizing the process.

Main Contents:

  • Request: concise (fetch) efficient (head) | | gm (post)
  • Batch package and report
  • No network delay is reported
  • Better PV: visibilitychange
  • Better PV: Single page hash listener applied

Function:

  • If the statistics platform server provides only the reporting interface, how to encapsulate data and report can be used for reference
  • Using apis from third-party analytics platforms, you can think about optimizations and encapsulation
  • It’s not about norms, it’s about ideas

The final code: analytics. Js

Request: concise and efficient | | gm

Let’s implement the buried point reporting script in the most direct way. Create a file named analytics. Js and add a request to the script to wrap it up a bit:

export default function analytics (action = 'pageview') {
  var xhr = new XMLHttpRequest()
  let uploadUrl = `https://xxx/test_upload? action=${action}&timestamp=The ${Date.now()}`
  xhr.open('GET', uploadUrl, true)
  xhr.send()
}
Copy the code

This way we can submit a message to our statistics server and specify an action type by calling analytics(). If the amount of data we need to report is really small, such as’ action/event ‘, ‘time’, ‘user (ID)’, ‘platform environment’, etc., and the amount of data is within the url length limit supported by the browser, we can simplify the request by:

// The concise way
export default function analytics (action = 'pageview') {(new Image()).src = `https://xxx/test_upload? action=${action}&timestamp=The ${Date.now()}`
}
Copy the code

The English term of sending requests with IMG is: Image Beacon is mainly applied to the occasion where only log data needs to be sent to the server, and the server does not need to have a message body response. Such as collecting statistics on visitors. The difference between this and an Ajax request is: 1. It can only be a GET request, so the amount of data that can be sent is limited. 2. Only care whether the data is sent to the server, the server does not need to respond to the message body. And typically the client does not need to respond. 3. Achieved cross-domain

Or we could just upload using the new standard fetch method

// The concise way
export default function analytics (action = 'pageview') {
  fetch(`https://www.baidu.com?action=${action}&timestamp=The ${Date.now()}`, {method: 'get'})}Copy the code

Considering the reporting process we don’t care about the return value, we just need to know whether the reporting is successful or not, we can use Head request to implement our reporting process more efficiently:

// The efficient way
export default function analytics (action = 'pageview') {
 fetch(`https://www.baidu.com?action=${action}&timestamp=The ${Date.now()}`, {method: 'head'})}Copy the code

Head requests and parameter passing are the same as GET requests and are browser-constrained, but are much more efficient than GET because it does not require a response entity to be returned. The simple request in the above example alone performs approximately 20ms better in Chrome.

If a large amount of data is to be uploaded, the URL length after concatenating parameters exceeds the limit of the browser, causing a request failure. Then we use post:

// Generic way (fetch can be used, but fetch does not have cookies by default, there may be authentication problems)
export default function analytics (action = 'pageview', params) {
  let xhr = new XMLHttpRequest()
  let data = new FormData()
  data.append('action', action)
  for (let obj in params) {
    data.append(obj, params[obj])
  }
  xhr.open('POST'.'https://xxx/test_upload')
  xhr.send(data)
}
Copy the code

Batch package and report

Regardless of the amount of data in a single buried point, it is assumed that the page has multiple buried points for user behavior analysis, and frequent reporting may have certain impact on users’ access to normal functions. The most straightforward way to solve this problem is to reduce the number of reported requests. Therefore, we will implement a feature uploaded in batches. A simple idea is to pack and report every 10 pieces of data collected:

// Every 10 pieces of data are packaged
let logs = []
/** * @params {array
function upload (logs) {
  console.log('send logs', logs)
  let xhr = new XMLHttpRequest()
  let data = new FormData()
  data.append('logs', logs)
  xhr.open('POST'.this.url)
  xhr.send(data)
}

export default function analytics (action = 'pageview', params) {
  logs.push(Object.assign({
    action,
    timeStamp: Date.now()
  }, params))
  if (logs.length >= 10) {
    upload(logs)
    logs = []
  }
}
Copy the code

At the buried point, we perform dozens of times to see

import analy from '@/vendor/analytics1.js'
for (let i = 33; i--;) {
    analy1('pv')}Copy the code

Ok, normally the report should be successful and each request contains 10 data. However, the problem was soon exposed. The behavior of collecting N data and then sending it in a unified manner would be faulted. If the user closed the page when it did not collect N data, or if it was more than N times but could not gather N, the data would be lost if it was not processed. A straightforward solution is to listen for the page-beforeUnload event and upload less than N remaining logs before the page leaves. So, we add a beforeUnload event and, incidentally, tidy up the code to wrap it into a class:

export default class eventTrack {
  constructor (option) {
    this.option = Object.assign({
      url: 'https://www.baidu.com'.maxLogNum: 10
    }, option)
    this.url = this.option.url
    this.maxLogNum = this.option.maxLogNum
    this.logs = []
    // Listen for the unload event,
    window.addEventListener('beforeunload'.this.uploadLog.bind(this), false)}@param {string} embedded data * @param {object} embedded data */
  analytics (action = 'pageview', params) {
    this.logs.push(Object.assign({
      action,
      timeStamp: Date.now()
    }, params))
    if (this.logs.length >= this.maxLogNum) {
      this.send(this.logs)
      this.logs = []
    }
  }
  // Report an array of logs
  send (logs, sync) {
    let xhr = new XMLHttpRequest()
    let data = new FormData()
    for (var i = logs.length; i--;) {
      data.append('logs'.JSON.stringify(logs[i]))
    }
    xhr.open('POST'.this.url, ! sync) xhr.send(data) }// Use synchronous XHR requests
  uploadLog () {
    this.send(this.logs, true)}}Copy the code

So far, we have preliminarily realized the function. Before further adding features, we should continue to optimize the current code. In combination with the previous process, we can consider optimizing the following points:

  1. The request reporting method is optional: The call form is as followsanalytics.head(single report),analytics.post(the default)
  2. Use better sendBeacon for page unload and backward compatibility

In the case of sendBeacon, this method can asynchronously transfer a small amount of data to the Web server. In the uploadLog method of the above code, we used a synchronous XHR request in case the final log could not be reported if the page was closed or switched and the script was not executed. Beforeunload synchronizes features of XHR and sendBeacon in scenarios

  • Sync XHR: Block the script for a while while leaving the page to make sure the log is sent
  • SendBeacon: Make an asynchronous request when leaving the page without blocking and ensure that the log is issued. There are browser compatibility issues

It is worth mentioning that in a single page application, route switching does not have much impact on missed reporting, as long as the reporting script is mounted globally and handles page closure and jump to other domain names. In summary, based on these two optimizations, we need to improve the code before adding new features:

export default class eventTrack {
  constructor (option) {
    this.option = Object.assign({
      url: 'https://www.baidu.com'.maxLogNum: 10
    }, option)
    this.url = this.option.url
    this.maxLogNum = this.option.maxLogNum
    this.logs = []

    // Extend Analytics to allow single reporting
    this.analytics['head'] = (action, params) = > {
      return this.sendByHead(action, params)
    }
    this.analytics['post'] = (action, params) = > {
      return this.sendByPost(action, params)
    }
    // Listen for the unload event,
    window.addEventListener('beforeunload'.this.unloadHandler.bind(this), false)}@param {string} embedded data * @param {object} embedded data */
  analytics (action = 'pageview', params) {
    this.logs.push(JSON.stringify(Object.assign({
      action,
      timeStamp: Date.now()
    }, params)))
    if (this.logs.length >= this.maxLogNum) {
      this.sendInPack(this.logs)
      this.logs = []
    }
  }

  @param {array} logs array * @param {Boolean} sync Whether to synchronize */
  sendInPack (logs, sync) {
    let xhr = new XMLHttpRequest()
    let data = new FormData()
    for (var i = logs.length; i--;) {
      data.append('logs', logs[i])
    }
    xhr.open('POST'.this.url, ! sync) xhr.send(data) }@param {string} Buried point event @param {object} Buried point additional parameter */
  sendByPost (action, params) {
    let xhr = new XMLHttpRequest()
    let data = new FormData()
    data.append('action', action)
    for (let obj in params) {
      data.append(obj, params[obj])
    }
    xhr.open('POST'.this.url)
    xhr.send(data)
  }

  @param {string} Buried point event @param {object} Buried point additional parameter */
  sendByHead (action, params) {
    let str = ' '
    for (let key in params) {
      str += ` &${key}=${params[key]}`
    }
    fetch(`https://www.baidu.com?action=${action}&timestamp=The ${Date.now()}${str}`, {method: 'head'})}/** * Reports events executed when unload events are triggered */
  unloadHandler () {
    if (navigator.sendBeacon) {
      let data = new FormData()
      for (var i = this.logs.length; i--;) {
        data.append('logs'.this.logs[i])
      }
      navigator.sendBeacon(this.url, data)
    } else {
      this.sendInPack(this.logs, true)}}}Copy the code

No network delay is reported

Consider this question: what if our page is offline (i.e. the signal is bad), the user does something during that time, and we want to collect that data?

  1. If the outage is brief, the script continues to execute and no package upload is triggered. Because the log is still stored in the memory, the execution continues until the number of uploads is triggered, and the network is recovered.
  2. The network disconnection period is long and the report is triggered several times. Network errors may cause the report failure. The network is restored and subsequent logs are reported normally. In this case, the data during the disconnection period is lost.
  3. Logs cannot be reported when the network disconnection starts at a certain point and continues until the user closes the page.

We could try adding a “failed retransmission” feature, which is more of a stable error caused by a problem than by network instability, and retransmission does not solve such problems. Imagine that we collect data on the client, which can be easily recorded in the log file. Therefore, with the same consideration, we can also temporarily store the data on localstorage and continue to report it in the network environment. Therefore, the solution to this problem can be summarized as follows:

  1. Reporting data,navigator.onLineJudging network Status
  2. Network transmission is normal
  3. Enter when there is no networklocalstorage, delay reporting

Let’s modify sendInPack and add corresponding methods

sendInPack (logs, sync) {
    if (navigator.onLine) {
      this.sendMultiData(logs, sync)
      this.sendStorageData()
    } else {
      this.storageData(logs)
    }
  }
  sendMultiData (logs, sync) {
    console.log('sendMultiData', logs)
    let xhr = new XMLHttpRequest()
    let data = new FormData()
    for (var i = logs.length; i--;) {
      data.append('logs', logs[i])
    }
    xhr.open('POST'.this.url, ! sync) xhr.send(data) } storageData (logs) {console.log('storageData', logs)
    let data = JSON.stringify(logs)
    let before = localStorage['analytics_logs']
    if (before) {
      data = before.replace('] '.', ') + data.replace('['.' ')
    }
    localStorage.setItem('analytics_logs', data)
  }
  sendStorageData () {
    let data = localStorage['analytics_logs']
    if(! data)return
    data = JSON.parse(data)
    this.sendMultiData(data)
    localStorage['analytics_logs'] = ' '
  }
Copy the code

Note the problem with navigator.onLine in different browser development environments. For example, when accessing localhost in Chrome, the value of navigator.onLine is always false

Better PV: visibilitychange

PV is an important part of log reporting. So far we have basically achieved the reporting, now back to the business level. What is the purpose of PV, and how best to achieve our purpose? Recommended reading this article on PV: Why your PV statistics are Wrong

In most cases, our PV reporting assumes that each Page View corresponds to a Page Load, and some statistical code is run after each Page Load. However, this situation is problematic especially for single-page applications

  1. Should a user open a Page once and then use it hundreds of times over the next few days without refreshing the Page
  2. If two users visit the Page exactly the same number of times per day, but one refreshes every time while the other keeps the Page running in the background, should the Page View statistics for the two usage patterns be significantly different
  3. …

To follow better PV, we can add the following handling to the script:

  1. When the Page loads, if the Page’s visibilityState is visible, send Page View statistics;
  2. When the Page is loaded, if the visibilityState of the Page is hidden, it listens to the visiBilityChange event, and sends the Page View statistics when the visibilityState becomes visible.
  3. If the visibilityState changes from hidden to visible, and “sufficient time” has elapsed since the last user interaction, a new Page View statistic is sent;
  4. If the URL changes (only the Pathname or search part sends the change, the hash part should be ignored, because it is used to mark in-page jumps) send the new Page View statistics; Add the following fragment to our constructor:
this.option = Object.assign({
  url: 'https://baidu.com/api/test'.maxLogNum: 10.stayTime: 2000.// ms, the page goes from hidden to visible, and it has been long enough since the last user interaction to be considered the time interval for the new PV
  timeout: 6000   // Page switching interval, less than the number of ms does not count interval
}, option)
this.hiddenTime = Date. Now... ()// Listen for page visibility
document.addEventListener('visibilitychange', () = > {console.log(document.visibilityState, Date.now(), this.hiddenTime)
  if (document.visibilityState === 'visible' && (Date.now() - this.hiddenTime > this.option.stayTime)) {
    this.analytics('re-open')
    console.log('send pv visible')}else if (document.visibilityState === 'hidden') {
    this.hiddenTime = Date. Now... ()}})Copy the code

Better PV: hash jump

Consider that we are a single page application in hash mode, where route jumps are identified by ending with ‘#’ plus route. If we want to track every route switch, one way is to listen on every route component, or we can directly deal with it in the report file:

window.addEventListener('hashchange', () => {
  this.analytics()
})
Copy the code

But there’s a problem with that. How do you know that the current hash jump is a valid jump? For example, if the page has redirection logic, the user enters from page A (deprecated page), our code redirects it to page B, so that PV is sent twice, while the actual valid browsing is only on page B once. Or maybe the user just looks at page A briefly and then jumps to page B. Should page A be used as A valid PV? A better approach would be to set the effective interval, such as a view of less than 5s does not count as a valid PV, so the logic from this is that we need to adjust our Analytics method:

// encapsulate sendPV to sendPV
constructor(option) {...this.sendPV = this.delay((args) = > {
    this.analytics({action: 'pageview'. args}) })window.addEventListener('hashchange', () = > {this.sendPV()
  })
  this.sendPV()
···
}

delay (func, time) {
    let t = 0
    let self = this
    return function (. args) {
      clearTimeout(t)
      t = setTimeout(func.bind(this, args), time || self.option.timeout)
    }
}
Copy the code

In analytics. Js, we add some call tests. Considering different business scenarios, we still have more space to fill