preface

I still remember in my last company, one of the big guys built a surveillance system and wanted to study how he did it. Of course, we are not the people who pat their heads on the head and do what others do. Here’s a look at the benefits of such a platform.

background

Front end system first of all, why should we do for you, look at this table below, can clearly see that the front end performance increase the value of the product, or pretty helpful, but if we can real-time collected these information, and to monitor implementation, make the whole product in the product line has been to maintain efficient operation, this is our purpose.

performance earnings
Google’s delay400ms Searches are down0.59%
Bing delay2s Drop in revenue4.3%
Yahoo delay400ms Flow down5-9%
Mozilla pages open less2.2 s Download increase15.4%
Netflix open Gzip Improved performance by 13.25% Reduced bandwidth50%

Secondly, it is also good for the products we release, so that we can find our mistakes in time. If a product is in a new iteration, an unspeakable error occurs.

Right! It’s just indescribable. We can’t wait for users to complain. By then, the day lilies are cold.

start

Based on the above, we began to build a front-end monitoring and simple control platform. (Although there are many such systems on the market, such as ELK, I can’t help myself.)

It has to be easy.

Guys, forgive me, this is as far as I can get you.

Here we go.

These are some of the things we need to do.

To collect information

To make a surveillance system, first we have to have an object. The object of our surveillance! Object! Object! Object.

I wrote a page in my system that looked like this,

<body>
    <div>2</div>
    <div>2</div>
    <div>2</div>
    <div>2</div>
    <div>2</div>
    <div>2</div> 
</body>
Copy the code

Yes, this is the page we’re monitoring. This… I’m not lazy.

And then I designed three pieces of data

  • Page load time
  • Collect statistics about devices used by users
  • Error statistics

Page load time

window.logInfo = {};  // Count page load time
window.logInfo.openTime = performance.timing.navigationStart;
window.logInfo.whiteScreenTime = +new Date() - window.logInfo.openTime;
document.addEventListener('DOMContentLoaded'.function (event) {
  window.logInfo.readyTime = +new Date() - window.logInfo.openTime;
});
window.onload = function () {
  window.logInfo.allloadTime = +new Date() - window.logInfo.openTime;
  window.logInfo.nowTime = new Date().getTime();
  var timname = {
    whiteScreenTime: 'White screen time'.readyTime: 'User available time'.allloadTime: 'Total Download time'.mobile: 'Use equipment'.nowTime: 'time'};var logStr = ' ';
  for (var i in timname) {
    console.warn(timname[i] + ':' + window.logInfo[i] + 'ms');
    if (i === 'mobile') {
      logStr += '&' + i + '=' + window.logInfo[i];
    } else {
      logStr += '&' + i + '=' + window.logInfo[i]; }} (new Image()).src = '/action? ' + logStr;
};
Copy the code

Collect statistics about devices used by users

window.logInfo.mobile = mobileType();
function mobileType() {
  var u = navigator.userAgent, app = navigator.appVersion;
  var type =  {// Mobile terminal browser version information
    ios:!!!!! u.match(/\(i[^;] +; ( U;) ? CPU.+Mac OS X/), / / ios terminal
    iPad: u.indexOf('iPad') > -1./ / whether the device
    android: u.indexOf('Android') > -1 || u.indexOf('Linux') > -1.// An Android terminal or uc browser
    iPhone: u.indexOf('iPhone') > -1 || u.indexOf('Mac') > -1.// Whether the browser is iPhone or QQHD
    trident: u.indexOf('Trident') > -1./ / IE kernel
    presto: u.indexOf('Presto') > -1./ / opera kernel
    webKit: u.indexOf('AppleWebKit') > -1.// Apple, Google kernel
    gecko: u.indexOf('Gecko') > -1 && u.indexOf('KHTML') = = -1.// The Firefox kernel
    mobile:!!!!! u.match(/AppleWebKit.*Mobile/i) | |!!!!! u.match(/MIDP|SymbianOS|NOKIA|SAMSUNG|LG|NEC|TCL|Alcatel|BIRD|DBTEL|Dopod|PHILIPS|HAIER|LENOVO|MOT-|Nokia|SonyEricsson|SIE-|Amoi |ZTE/), // Whether it is a mobile terminal
    webApp: u.indexOf('Safari') = = -1 // Whether the web should be a program, without a header and a bottom
  };
  var lists = Object.keys(type);
  for(var i = 0; i < lists.length; i++) {
    if(type[lists[i]]) {
      returnlists[i]; }}}Copy the code

Error statistics

window.onload = function () {
        window.logInfo.allloadTime = +new Date() - window.logInfo.openTime;
        window.logInfo.nowTime = new Date().getTime();
        var timname = {
            whiteScreenTime: 'White screen time'.readyTime: 'User available time'.allloadTime: 'Total Download time'.mobile: 'Use equipment'.nowTime: 'time'};var logStr = ' ';
        for (var i in timname) {
            console.warn(timname[i] + ':' + window.logInfo[i] + 'ms');
            if (i === 'mobile') {
                logStr += '&' + i + '=' + window.logInfo[i];
            } else {
                logStr += '&' + i + '=' + window.logInfo[i]; }} (new Image()).src = '/action? ' + logStr;
    };
      
    var defaults = {
        msg:' '.// Error specific information
        url:' '.// The url where the error occurred
        line:' '.// The error line
        col:' '.// The column where the error occurred
        nowTime: ' './ / time
    };
    window.onerror = function(msg,url,line,col,error) {
        col = col || (window.event && window.event.errorCharacter) || 0;

        defaults.url = url;
        defaults.line = line;
        defaults.col =  col;
        defaults.nowTime = new Date().getTime();

        if (error && error.stack){
            // If the browser has stack information, use it directly
            defaults.msg = error.stack.toString();

        }else if (arguments.callee){
            // Try to fetch stack information through callee
            var ext = [];
            var fn = arguments.callee.caller;
            var floor = 3;  
            while (fn && (--floor>0)) {
                ext.push(fn.toString());
                if (fn  === fn.caller) {
                    break;
                }
                fn = fn.caller;
            }
            ext = ext.join(",");
            defaults.msg = error.stack.toString();
        }
        var str = ' '
        for(var i in defaults) {
            // console.log(i,defaults[i]);
            if(defaults[i] === null || defaults[i] === undefined) {
                defaults[i] = 'null'; 
            }
            str += '&'+ i + '=' + defaults[i].toString();
        }
        srt = str.replace('&'.' ').replace('\n'.' ').replace(/\s/g.' ');
        (new Image()).src = '/error? ' + srt;
    }
Copy the code

That’s all you need to do to collect data, either by sending /action requests or by sending /error requests, which can be customized, but I’m just talking about how the process works.

Then I processed and recorded all the requests through one of my background express.js, and the data after recording is like this.

User_ip = 127.0.0.1 & whiteScreenTime = 185 & readyTime = 192 & allloadTime mobile = webKit&nowTime = = 208 & 1513071388941Copy the code

The data processing

Here I wrote a script to parse, parse.js, here is not specific, look at the source code. Let me show you the parsed data.

I store the data in CVS format, and I also support json format export for the following diagrams, but you need to configure the visual interface later.

Here’s the data.

charts/csvData/2017-12-16time.csv

Time, bad time, the user can operation time, total download time, 1513427051482137137153, 1513427065080470471507, 1513427080040127127143 1513428714345274275323 1513428733583267268317 1513428743167268268317 1513428754796276276328Copy the code

The data show

I’m using highcharts.js here

I will not explain the specific configuration, you can go to the official website to check.

Below is a visual chart showing the information for each time of day.

The interface may not be particularly beautiful, please forgive me.

The environment

The node > = 6.0.0

Redis > = server

Here I would like to explain that if this online environment is deployed, if every record is recorded, it will consume a lot of memory, so I set up a layer of Redis, in order to prevent the impact of heavy traffic, and then can be stored at intervals.

const express = require('express');
const performance = require('./lib/performance.js');
const app = express();
const router = express.Router();
router.get('/'.function (req, res, next) {
  req.url = './index.html';
  next();
});
app.use(router);
app.use(performance({
    time: 10.// The unit is seconds
    originalDir: './originalData'.// Directory of data
    errorDir: './errorData' // The directory where the error was reported
}))
app.use(express.static('/'));
const server = app.listen(3000)
Copy the code

You can set the default time here, but I’m going to do it in 10 seconds for demo purposes. I usually do it once a minute.

Code address: github.com/hua1995116/…

If you have good suggestions and optimization plan, please also put forward to me in Issues.

Advanced (an actual combat chestnut using monitoring platform)

I used this platform to monitor one of my projects. If you just play, please just read the original system address above, you can ignore my paragraph, after all, my system is not perfect.

Online demo: www.qiufengh.com/#/

Monitor demo:qiufengh.com :8080/

Project: github.com/hua1995116/…

Here I have set logging every 1 minute.

// Monitor import
app.use(performance({
    time: 60.// The unit is seconds
    originalDir: './originalData'.// Directory of data
    errorDir: './errorData' // The directory where the error was reported
}))
Copy the code

And parsing every 10 minutes.

function setPrase() {
    setInterval(function(){
        parseData();
      }, 1000 * 60 * 10);
}
Copy the code

2017-12-20

This article only provides an idea, if you want professional, you can use professional statistical analysis tools such as ELK.