Set up front-end anomaly monitoring system
Involves skills
- Collect front-end errors (native, React, Vue)
- Write error reporting logic
- Write an error log collection service using egg.js
- Write a Webpack plug-in to automatically upload Sourcemap
- Restore the compressed source code location using Sourcemap
- Use Jest for unit testing
The working process
- Collect wrong
- Report the error
- The code goes live and the Sourcemap file is uploaded to the error monitoring server
- The monitoring server receives and logs errors when they occur
- Error analysis is performed based on sourcemAP and error log content
Abnormal collection
Let’s start by looking at how to catch exceptions.
JS abnormal
Js exceptions do not cause the JS engine to crash, but only terminate the currently executing task. For example, a page has two buttons. If an abnormal page occurs when you click the button, the page will not crash, but the function of this button will be invalid, and other buttons will be effective.
setTimeout(() = > {
console.log('1->begin')
error
console.log('1->end')})setTimeout(() = > {
console.log('2->begin')
console.log('2->end')})Copy the code
In the example above we started two tasks with setTimeout, even though the first task executed the wrong method. The execution of the program stopped. But the other task was not affected.
In fact, if you don’t open the console you won’t even see the error. It’s as if the mistake happened in silence.
Let’s look at how such errors can be collected.
try-catch
Our first thought was to use try-catch to collect.
setTimeout(() = > {
try {
console.log('1->begin')
error
console.log('1->end')}catch (e) {
console.log('catch',e)
}
})
Copy the code
If the error is not caught in a function, the error is thrown.
function fun1() {
console.log('1->begin')
error
console.log('1->end')}setTimeout(() = > {
try {
fun1()
} catch (e) {
console.log('catch',e)
}
})
Copy the code
The console prints error messages and error stacks, respectively.
Now, you might be thinking, well, why don’t you just make a try-catch error at the bottom. But the ideal is full, the reality is very skinny. Let’s look at the next example.
function fun1() {
console.log('1->begin')
error
console.log('1->end')}try {
setTimeout(() = > {
fun1()
})
} catch (e) {
console.log('catch', e)
}
Copy the code
Notice that the exception is not caught.
And that’s because JS has very limited try-catch capabilities and it doesn’t work very well with asynchrony. You can’t add a try-catch to all asynchrony to collect errors.
window.onerror
The greatest benefit of window.onerror is that it can catch both synchronous and asynchronous tasks.
function fun1() {
console.log('1->begin')
error
console.log('1->end')}window.onerror = (. args) = > {
console.log('onerror:',args)
}
setTimeout(() = > {
fun1()
})
Copy the code
-
Onerror return value
One other problem with onError is that if you return true you don’t get thrown up. Otherwise you’ll see the error log in the console.
Listening for Error Events
Window. The addEventListener (‘ error ‘() = > {})
Well, onError is great but there’s still a class of exceptions that you can’t catch. This is a network exception error. Take the following example.
<img src="./xxxxx.png">
Copy the code
Imagine if the image we wanted to display on the page suddenly stopped showing and we didn’t even know it was a problem.
AddEventListener is
window.addEventListener('error'.args= > {
console.log(
'error event:', args
);
return true;
},
true // Use the capture method
);
Copy the code
Promise exception catching
Promise came along primarily to allow us to address the issue of callback geography. It’s basically standard for our program development. Although we advocate es7 async/await syntax for writing, we do not rule out that many ancestral codes still have Promise writing.
new Promise((resolve, reject) = > {
abcxxx()
});
Copy the code
Neither onError nor listening error events can be caught in this case
new Promise((resolve, reject) = > {
error()
})
// Add exception catching
.catch((err) = > {
console.log('promise catch:',err)
});
Copy the code
Unless each Promise adds a catch method. But obviously you can’t do that.
window.addEventListener("unhandledrejection".e= > {
console.log('unhandledrejection',e)
});
Copy the code
We can consider unhandledrejection event capture error throw and let the error event handle
window.addEventListener("unhandledrejection".e= > {
throw e.reason
});
Copy the code
Async /await exception capture
const asyncFunc = () = > new Promise(resolve= > {
error
})
setTimeout(async() = > {try {
await asyncFun()
} catch (e) {
console.log('catch:',e)
}
})
Copy the code
In fact the async/await syntax is essentially a Promise syntax. The difference is that async methods can be caught by a superimposed try/catch.
If not, it will be captured with an unhandledrejection event, just like a Promise. In this case, we just need to add unHandlerejection globally.
summary
Exception types | Synchronized methods | Asynchronous methods | Resource to load | Promise | async/await |
---|---|---|---|---|---|
try/catch | ✔ ️ | ✔ ️ | |||
onerror | ✔ ️ | ✔ ️ | |||
Error event listening | ✔ ️ | ✔ ️ | ✔ ️ | ||
Unhandledrejection event listener | ✔ ️ | ✔ ️ |
In fact, we can throw the exception thrown by the unhandledrejection event again and we can handle it with the error event.
The final code is as follows:
window.addEventListener("unhandledrejection".e= > {
throw e.reason
});
window.addEventListener('error'.args= > {
console.log(
'error event:', args
);
return true;
}, true);
Copy the code
Webpack engineering
Now is the era of front-end engineering, engineering exported code is generally compressed and confused.
Such as:
setTimeout(() = > {
xxx(1223)},1000)
Copy the code
The error code points to the compressed JS file.
If you want to associate errors with the original code, you need the sourcemap file to help.
What is sourceMap
Simply put, sourceMap is a file that stores location information.
And, to be more careful, what this file contains is the position of the code after the transformation, and the corresponding position before the transformation.
How to use sourceMap to restore the location of the exception code will be covered in the exception analysis section.
Vue
Create a project
Create a project directly using vue-CLI tools.
Create a project vue create vue-sample CD vue-sample NPM I// Start the application
npm run serve
Copy the code
We’re going to turn off ESLint temporarily for testing purposes and it’s recommended that you turn esLint on at all times
Configure it in vue.config.js
module.exports = {
// Disable the ESLint rule
devServer: {
overlay: {
warnings: true.errors: true}},lintOnSave:false
}
Copy the code
We intentionally in SRC/components/HelloWorld. Vue
<script>
export default {
name: "HelloWorld".props: {
msg: String
},
mounted() {
// make an error
abc()
}
};
</script>Javascript window.addeventListener ('error', args => {console.log('error', error)})Copy the code
At this point the error is printed in the console, but the error event is not listened for.
handleError
To uniformly report Vue exceptions, use the handleError handle provided by Vue. This method is called whenever a Vue exception occurs.
We are in the SRC/main. Js
Vue.config.errorHandler = function (err, vm, info) {
console.log('errorHandle:', err)
}
Copy the code
React
npx create-react-app react-sample
cd react-sample
yarn start
Copy the code
We make an error using useEffect hooks
import React ,{useEffect} from 'react';
import logo from './logo.svg';
import './App.css';
function App() {
useEffect(() = > {
// An exception occurred
error()
});
return (
<div className="App">/ /... A little...</div>
);
}
export default App;
Copy the code
Add error event listening logic to SRC /index.js
window.addEventListener('error'.args= > {
console.log('error', error)
})
Copy the code
But from the run result, although the output error log is still captured by the service.
ErrorBoundary label
Error bounds can only catch errors of its children. Error boundaries cannot catch errors of their own. If an error boundary cannot render the error message, the error bubbles up to the nearest error boundary. This is also similar to how catch {} works in JavaScript.
Create the ErrorBoundary component
import React from 'react';
export default class ErrorBoundary extends React.Component {
constructor(props) {
super(props);
}
componentDidCatch(error, info) {
// Print an error when an exception occurs
console.log('componentDidCatch',error)
}
render() {
return this.props.children; }}Copy the code
Wrap the App tag in SRC /index.js
import ErrorBoundary from './ErrorBoundary'
ReactDOM.render(
<ErrorBoundary>
<App />
</ErrorBoundary>
, document.getElementById('root'));
Copy the code
In the last article we focused on how to collect JS errors. In this article we will talk about how exceptions are reported and analyzed.
Exception reporting
Select a communication mode
Dynamically create an IMG tag
The purpose of reporting is to send the captured exception information to the back end. The most common method is to create a label dynamically. There is no need to load any communication libraries and the page does not need to be refreshed. Basically including Baidu statistics Google statistics are based on this principle to do the buried point.
new Image().src = 'http://localhost:7001/monitor/error'+ '? info=xxxxxx'
Copy the code
By dynamically creating an IMG, the browser sends a GET request to the server. You can report errors to the server by placing the error data you need to report in a QueryString string.
Ajax report
We can actually use Ajax to report errors, just like we would in a business application. I won’t repeat it here.
What data to report
Let’s first look at the error event parameters:
The attribute name | meaning | type |
---|---|---|
message | The error message | string |
filename | Abnormal resource URL | string |
lineno | Abnormal line number | int |
colno | Abnormal column number | int |
error | Error object | object |
error.message | The error message | string |
error.stack | The error message | string |
One of the core should be the error stack, in fact, we locate the error is the most important error stack.
The error stack contains most debugging information. It includes the exception position (row number, column number), exception information
Reported data serialization
Since the communication can only be transmitted as strings, we need to serialize the object.
There are basically three steps:
-
The exception data is deconstructed from the attributes and stored in a JSON object
-
Convert a JSON object to a string
-
Convert the string to Base64
And of course you have to do the opposite on the back end and we’ll talk about that later.
window.addEventListener('error'.args= > {
console.log(
'error event:', args
);
uploadError(args)
return true;
}, true);
function uploadError({ lineno, colno, error: { stack }, timeStamp, message, filename }) {
/ / filter
const info = {
lineno,
colno,
stack,
timeStamp,
message,
filename
}
// const str = new Buffer(JSON.stringify(info)).toString("base64");
const str = window.btoa(JSON.stringify(info))
const host = 'http://localhost:7001/monitor/error'
new Image().src = `${host}? info=${str}`
}
Copy the code
Abnormal collection
The abnormal data must be received by a back-end service.
Take eggJS, a popular open source framework, as an example
Set up the EGGJS project
# create backend project egg-init backend --type=simple CD backend NPM I # Start project NPM run devCopy the code
Write the error upload interface
Start by adding a new route to app/router.js
module.exports = app= > {
const { router, controller } = app;
router.get('/', controller.home.index);
// Create a new route
router.get('/monitor/error', controller.monitor.index);
};
Copy the code
Create a new controller (app/ Controller /monitor)
'use strict';
const Controller = require('egg').Controller;
const { getOriginSource } = require('.. /utils/sourcemap')
const fs = require('fs')
const path = require('path')
class MonitorController extends Controller {
async index() {
const { ctx } = this;
const { info } = ctx.query
const json = JSON.parse(Buffer.from(info, 'base64').toString('utf-8'))
console.log('fronterror:', json)
ctx.body = ' '; }}module.exports = MonitorController;
Copy the code
Log file
The next step is to log errors. The implementation of the method can be written in FS, or with the help of log4JS such a mature log library.
Of course eggJS supports custom logging, so you can use this feature to customize a front-end error log.
In the/config/config. Default. Add a custom js logging configuration
// Define the front-end error log
config.customLogger = {
frontendLogger : {
file: path.join(appInfo.root, 'logs/frontend.log')}}Copy the code
In the/app/controller/monitor. Js add logging
async index() {
const { ctx } = this;
const { info } = ctx.query
const json = JSON.parse(Buffer.from(info, 'base64').toString('utf-8'))
console.log('fronterror:', json)
// Log the error
this.ctx.getLogger('frontendLogger').error(json)
ctx.body = ' ';
}
Copy the code
Abnormal analysis
When it comes to exception analysis, the most important work is actually to restore the webPack obfuscated compression code.
The Webpack plug-in implements SourceMap uploads
The Sourcemap file is generated during webpack packaging and needs to be uploaded to the exception monitoring server. This function we use the Webpack plug-in to complete.
Create the WebPack plug-in
/source-map/plugin
const fs = require('fs')
var http = require('http');
class UploadSourceMapWebpackPlugin {
constructor(options) {
this.options = options
}
apply(compiler) {
// Execute after packing
compiler.hooks.done.tap("upload-sourcemap-plugin".status= > {
console.log('webpack runing')}); }}module.exports = UploadSourceMapWebpackPlugin;
Copy the code
Load the WebPack plug-in
webpack.config.js
// Automatically upload the Map
UploadSourceMapWebpackPlugin = require('./plugin/uploadSourceMapWebPackPlugin')
plugins: [
// Add automatic upload plugin
new UploadSourceMapWebpackPlugin({
uploadUrl:'http://localhost:7001/monitor/sourcemap'.apiKey: 'xxx'})].Copy the code
Add read sourcemAP read logic
Add logic to read sourcemap files in apply
/plugin/uploadSourceMapWebPlugin.js
const glob = require('glob')
const path = require('path')
apply(compiler) {
console.log('UploadSourceMapWebPackPlugin apply')
// Definitions are executed after packaging
compiler.hooks.done.tap('upload-sourecemap-plugin'.async status => {
// Read the sourcemap file
const list = glob.sync(path.join(status.compilation.outputOptions.path, `./**/*.{js.map,}`))
for (let filename of list) {
await this.upload(this.options.uploadUrl, filename)
}
})
}
Copy the code
Implement the HTTP upload function
upload(url, file) {
return new Promise(resolve= > {
console.log('uploadMap:', file)
const req = http.request(
`${url}? name=${path.basename(file)}`,
{
method: 'POST'.headers: {
'Content-Type': 'application/octet-stream'.Connection: "keep-alive"."Transfer-Encoding": "chunked"
}
}
)
fs.createReadStream(file)
.on("data".chunk= > {
req.write(chunk);
})
.on("end".() = >{ req.end(); resolve() }); })}Copy the code
Add an upload interface on the server
/backend/app/router.js
module.exports = app= > {
const { router, controller } = app;
router.get('/', controller.home.index);
router.get('/monitor/error', controller.monitor.index);
// Add an upload route
router.post('/monitor/sourcemap',controller.monitor.upload)
};
Copy the code
Add the SourcemAP upload interface
/backend/app/controller/monitor.js
async upload() {
const { ctx } = this
const stream = ctx.req
const filename = ctx.query.name
const dir = path.join(this.config.baseDir, 'uploads')
// Determine whether the upload directory exists
if(! fs.existsSync(dir)) { fs.mkdirSync(dir) }const target = path.join(dir, filename)
const writeStream = fs.createWriteStream(target)
stream.pipe(writeStream)
}
Copy the code
The call plug-in Sourcemap is uploaded to the server when performing webPack packaging.
Parsing ErrorStack
Considering that this feature requires a lot of logic, we are going to develop it as a stand-alone function and use Jest for unit testing
Let’s look at our requirements
The input | A stack of errors | ReferenceError: xxx is not defined\n’ + ‘ at http://localhost:7001/public/bundle.e7877aa7bc4f04f5c33b.js:1:1392’ |
---|---|---|
SourceMap | slightly | |
The output | Source error stack | { source: ‘webpack:///src/index.js’, line: 24, column: 4, name: ‘xxx’ } |
Build the Jest framework
Start by creating a /utils/ stackParser.js file
module.exports = class StackPaser {
constructor(sourceMapDir) {
this.consumers = {}
this.sourceMapDir = sourceMapDir
}
}
Copy the code
Create the test file stackParser.spec.js in the sibling directory
The above requirements are expressed by Jest
const StackParser = require('.. /stackparser')
const { resolve } = require('path')
const error = {
stack: 'ReferenceError: xxx is not defined\n' +
' at http://localhost:7001/public/bundle.e7877aa7bc4f04f5c33b.js:1:1392'.message: 'Uncaught ReferenceError: xxx is not defined'.filename: 'http://localhost:7001/public/bundle.e7877aa7bc4f04f5c33b.js'
}
it('stackparser on-the-fly'.async() = > {const stackParser = new StackParser(__dirname)
/ / assertions
expect(originStack[0]).toMatchObject(
{
source: 'webpack:///src/index.js'.line: 24.column: 4.name: 'xxx'})})Copy the code
Now let’s run Jest
npx jest stackparser --watch
Copy the code
The display fails simply because we haven’t implemented it yet. So let’s implement this method.
Antisequence Error object
First create a new Error object to set the Error stack to Error, then use the error-stack-parser NPM library to convert to stackFrame
const ErrorStackParser = require('error-stack-parser')
/** * error stack deserialization *@param {*} Stack Error stack */
parseStackTrack(stack, message) {
const error = new Error(message)
error.stack = stack
const stackFrame = ErrorStackParser.parse(error)
return stackFrame
}
Copy the code
Parsing ErrorStack
Next we convert the code location in the error stack to the source location
const { SourceMapConsumer } = require("source-map");
async getOriginalErrorStack(stackFrame) {
const origin = []
for (let v of stackFrame) {
origin.push(await this.getOriginPosition(v))
}
// Destroy all consumers
Object.keys(this.consumers).forEach(key= > {
console.log('key:',key)
this.consumers[key].destroy()
})
return origin
}
async getOriginPosition(stackFrame) {
let { columnNumber, lineNumber, fileName } = stackFrame
fileName = path.basename(fileName)
console.log('filebasename',fileName)
// Check whether it exists
let consumer = this.consumers[fileName]
if (consumer === undefined) {
/ / read sourcemap
const sourceMapPath = path.resolve(this.sourceMapDir, fileName + '.map')
// Check whether the directory exists
if(! fs.existsSync(sourceMapPath)){return stackFrame
}
const content = fs.readFileSync(sourceMapPath, 'utf8')
consumer = await new SourceMapConsumer(content, null);
this.consumers[fileName] = consumer
}
const parseData = consumer.originalPositionFor({ line:lineNumber, column:columnNumber })
return parseData
}
Copy the code
Let’s test that out with Jest
it('stackparser on-the-fly'.async() = > {const stackParser = new StackParser(__dirname)
console.log('Stack:',error.stack)
const stackFrame = stackParser.parseStackTrack(error.stack, error.message)
stackFrame.map(v= > {
console.log('stackFrame', v)
})
const originStack = await stackParser.getOriginalErrorStack(stackFrame)
/ / assertions
expect(originStack[0]).toMatchObject(
{
source: 'webpack:///src/index.js'.line: 24.column: 4.name: 'xxx'})})Copy the code
Log the source location
async index() {
console.log
const { ctx } = this;
const { info } = ctx.query
const json = JSON.parse(Buffer.from(info, 'base64').toString('utf-8'))
console.log('fronterror:', json)
// Convert to source location
const stackParser = new StackParser(path.join(this.config.baseDir, 'uploads'))
const stackFrame = stackParser.parseStackTrack(json.stack, json.message)
const originStack = await stackParser.getOriginalErrorStack(stackFrame)
this.ctx.getLogger('frontendLogger').error(json,originStack)
ctx.body = ' ';
}
Copy the code
Open source framework
Fundebug
Fundebug focuses on real-time BUG monitoring for JavaScript, wechat applets, wechat games, Alipay applets, React Native, Node.js and Java online applications. Since its official launch on November 11, 2016, Fundebug has handled over 1 billion error events in total, and paid customers include Sunshine Insurance, Lychee FM, Zhangmen 1-on-1, Walnut Programming, Weomai and many other brand enterprises. Welcome free trial!
Sentry
Sentry is an open source real-time error tracking system that helps developers monitor and fix exceptions in real time. It focuses on continuous integration, improving efficiency, and improving the user experience. Sentry is divided into server and client SDK. The former can directly use the online services provided by its home, or can be built locally. The latter provides support for many major languages and frameworks, including React, Angular, Node, Django, RoR, PHP, Laravel, Android,.NET, JAVA, and more. It also offers solutions to integrate with other popular services, such as GitHub, GitLab, Bitbuck, Heroku, Slack, Trello, and more. The company’s current projects are also phasing in Sentry for error log management.
conclusion
So far, we’ve taken the basic functionality of front-end exception monitoring as an MVP(minimum viable Product). There is much more to be done, and ELK can be used for analysis and visualization of error logs. Docker can be used for publishing and deploying. Add permission control to upload and report EggJS.
Reference code location: github.com/su37josephx…