Production js error collection and source location

Now the front-end production environment code is basically compressed, if you need to know the source location of the compression code error, we can use the sourcemap file to achieve. But for the front end, if you put Sourcemap out there, you almost expose the source code. To solve this problem, I put the sourcemap file on the server, and then sent the browser error message to the server, which used the error message to locate the source code. Having used Sentry before, it seemed to support it, but I had some additional needs, so I built a simple wheel.

background

In fact, my initial idea is that I only need to monitor JS errors and push the error information to the enterprise wechat through the enterprise wechat robot.

Later, the error messages received by the enterprise wechat are all compressed code error messages, and the specific error location cannot be quickly located (or may not be found). Therefore, sourcemap is used to resolve the error information on the server side.

Check the specific location of the error and to find the corresponding file according to the information received by the enterprise wechat, well, too much trouble, and will report the error location directly to the GitLab source link, click the link in the message can jump to the source location.

Later, the wechat of the enterprise received too many messages, which was not conducive to management, so the error message was automatically created into the issue of GitLab.

Please check the official documents for the implementation of enterprise wechat push and automatic creation of GitLab Issue. This article introduces the implementation of the following points, as follows:

The browser collects errors and sends them to the server
Upload the sourcemap file to the server when the build is complete. Delete the local Sourcemap file after the upload, and the sourcemap URL is not required at the end of the packaged JS file
The server receives uploaded files
The server receives the error message and uses the source-Map tool to locate the error message to the source code location

The browser collects errors and sends them to the server

Since vuejs is used, errors can be directly collected in vue.config. errorHandler (errors elsewhere are skipped here). When using errorHandler, it is found that the row and column where the code reported the error cannot be obtained from the error returned by errorHandler. It can only be retrieved from error. Stack, so the entire stack must be passed to the server for processing.

Static/SRC /main.js

Vue.config.errorHandler = function(err, vm, info) {
  const data = {
    message: err.message,
    stack: err.stack,
    info,
    href: location.href
  }
  fetch('http://localhost:3000/log/loading.gif', {
    method: 'POST'.headers: {
      'content-type': 'application/json'
    },
    body: JSON.stringify(data)
  })
}
Copy the code

Upload Sourcemap when the build is complete

Two things need to happen here:

Build packaging requires the sourcemap file to be generated, and the sourcemap URL is not required at the end of the packaged JS file

This can be done using the SourceMapDevToolPlugin for WebPack

new webpack.SourceMapDevToolPlugin({
  filename: 'sourcemap/[file].map'.// Change the path to generate the sourcemap file (corresponding to dist/sourcemap)
  append: false // Do not add sourcemapUrl to the end of the file
})
Copy the code

Upload all sourcemap files to the server when the build is complete

Here I wrote a WebPack plug-in of my own. Since axios was used in the project, I also used Axios directly for uploading files in the plugin. FormData objects exist on the browser side and do not exist in native NodeJS, so FormData dependencies are introduced instead.

The idea of implementation: In the afterEmit hook of the Webpack Compiler, read all the.js.map files in the build output directory and upload them one by one (or multiple concurrently). Delete all the Sourcemap files once the upload is complete (the SourceMapDevToolPlugin above has set all the sourcemap files to the sourcemap directory, so you only need to delete this directory).

The static/UploadSourceMapPlugin js view the source file

const Path = require('path')
const Fs = require('fs')
const Axios = require('axios')
const FormData = require('form-data')
const PLUGIN_NAME = 'UploadSourceMapPlugin'

class UploadSourceMapPlugin {
  // Read all.js.map files in the directory
  async getAssets(distDir) {
    const files = await Fs.promises.readdir(distDir)
    return files.filter(el= > /\.js\.map$/i.test(el)).map(el= > Path.join(distDir, el))
  }

  // Upload the file to the server
  async upload(filepath) {
    const stream = Fs.createReadStream(filepath)
    const formData = new FormData()
    formData.append('file', stream)
    return Axios.default({
      url: 'http://localhost:3000/upload'.method: 'put'.headers: formData.getHeaders(),
      timeout: 10000.data: formData
    }).then().catch((err) = > {
      console.error(Path.basename(filepath), err.message)
    })
  }

  apply(compiler) {
    // The path must be the same as where the SourceMapDevToolPlugin stored the Sourcemap file
    const sourcemapDir = Path.join(compiler.options.output.path, 'sourcemap')
    compiler.hooks.afterEmit.tapPromise(PLUGIN_NAME, async() = > {console.log('Uploading sourcemap files... ')
      const files = await this.getAssets(Path.join(sourcemapDir, 'js')) // Upload only js sourcemap files
      for (const file of files) {
        await this.upload(file)
      }
      // Note: node < 14.14.0 can be replaced with fs.promises. Rmdir
      await Fs.promises.rm(sourcemapDir, { recursive: true}}})})Copy the code

The project was built using the Vue CLI, so the final vue.config.js file configuration is as follows:

Static /vue.config.js View the source file

const webpack = require('webpack')
const UploadSourceMapPlugin = require('./uploadSourceMapPlugin')
module.exports = {
  publicPath: '/'.configureWebpack(config) {
    if (process.env.NODE_ENV === 'production') {
      config.plugins.push(
        new webpack.SourceMapDevToolPlugin({
          filename: 'sourcemap/[file].map'.// Change the path to generate the sourcemap file (corresponding to dist/sourcemap)
          append: false // Do not add sourcemapUrl to the end of the file
        }),
        new UploadSourceMapPlugin()
      )
    }
  }
}
Copy the code

At this point, the construction part of the project has been basically completed.

The web service

The server uses @hapi/hapi as sever. The code is as follows:

Server/SRC /app.ts

import * as  Hapi from '@hapi/hapi'
import inert from '@hapi/inert'
import * as routes from './router'

async function start() {
  const server = Hapi.server({ host: '127.0.0.1'.port: 3001 })
  await server.register(inert)
  server.route(Object.values(routes))
  await server.start()
  console.log(`Server running at: ${server.info.uri}`)
}

start()
Copy the code

Interface for uploading files

Since the Sourcemap files contain hash values to ensure that each file name is unique, I put all uploaded Sourcemap files directly into the Uploads directory. If there is more code, only part of the code is displayed

Server/SRC /router.ts

/** Interface for uploading files */
export const upload = <Hapi.ServerRoute>{
  method: 'put'.path: '/upload'.options: {
    payload: {
      multipart: { output: 'stream' },
      allow: ['application/json'.'multipart/form-data'],}},async handler(request, h) {
    const { file } = request.payload as { file: MyFile }
    const dir = Path.join(process.cwd(), 'uploads')
    await Fs.ensureDir(dir)
    return new Promise((resolve) = > {
      const ws = Fs.createWriteStream(Path.join(dir, file.hapi.filename))
      file.pipe(ws)

      file.on('end'.() = > {
        resolve(h.response({ status: true }).code(200))
      })

      file.on('error'.() = > {
        resolve(h.response({ status: false }).code(500))})})}}Copy the code

The server receives the reported error and parses it through source-map

Error resolution for a simple package, mainly do the following steps:

Take the stack information, process it line by line, and get from each line the corresponding Sourcemap file name, the row and column where the error occurred (in the corresponding code)stackMethods)
Go to the uploads directory according to the sourcemap file name to find the corresponding file and read the contents (corresponding to the coderawSourceMapMethods)
The source-Map plug-in parses the file contents, lines, and columns to locate the source codeparseMethods)
Concatenation of the resulting source-map data during line-by-line processing returns a new error message (in the corresponding code)stackMethods)

Server/SRC /parseError. Ts This file code is more simplified, complete version please directly view the source code

import * as Path from 'path'
import * as Fs from 'fs-extra'
import { SourceMapConsumer } from 'source-map'

const uploadDir = Path.join(process.cwd(), 'uploads')
export default class ParseError {
  /** Read the sourcemap file contents */
  private async rawSourceMap(filepath: string) {
    return Fs.readJSON(filepath, { throws: false})}public async stack(stack: string) {
    const lines = stack.split('\n')
    const newLines: string[] = [lines[0]]
    // line by line
    for (const item of lines) {
      if (/ +at.+.js:\d+:\d+\)$/) {
        const arr = item.match(/\((https? :\/\/.+):(\d+):(\d+)\)$/i) | | []if (arr.length === 4) {
          const url = arr[1]
          const line = Number(arr[2])
          const column = Number(arr[3])
          const filename = (url.match(/ [^ /] + $/) | | [' '[])0]

          const res = await this.parse(filename + '.map', line, column)
          if (res && res.source) {
            const content = `    at ${res.name} (${[res.source, res.line, res.column].join(':')}) `
            newLines.push(content)
          } else {
            // The original error message is used if the parsing fails
            newLines.push(item)
          }
        }
      }
    }
    return newLines.join('\n')}/** Locate the source from sourcemap by row and column */
  private async parse(filename: string, line: number, column: number) {
    const raw = await this.rawSourceMap(filename)
    const consumer = await SourceMapConsumer.with(raw, null.consumer= > consumer)
    return consumer.originalPositionFor({ line, column })
  }
}
Copy the code

Receive the error message and return the parsed error message

File upload and error resolution have been implemented above, basically everything is ready, only to receive the client upload error message and parse, finally can do what you want to do.

Server/SRC /router.ts

/** accepts the error and returns */
export const jsError = <Hapi.ServerRoute>{
  method: 'post'.path: '/api/js/error'.async handler(req) {
    const data = <{ stack: string }>req.payload
    const parser = new ParseError()
    const result = await parser.stack(data.stack)
    parser.destroy() // Destroy cconsumer after parsing

    // After you get the result, you can do some operations you want, such as push, save data, etc
    // Return the parsed result directly

    return result
  }
}
Copy the code

At this point, you have completed the process from building and uploading the Sourcemap file, to server side file storage, to uploading the error message from the browser side, to server side parsing and locating the source code where the error occurred.

The source is available at GitHub github.com/satrong/par…

Original articles, reprint please indicate the source www.xiaoboy.com/topic/serve…

Production js error collection and source location

background

The browser collects errors and sends them to the server

Upload Sourcemap when the build is complete

The web service

Interface for uploading files

The server receives the reported error and parses it through source-map

Receive the error message and return the parsed error message

Related Posts

Contains hides elements outside the location of the click element

Cross – end development framework depth review

Echarts– Draw bubble map