In Web front-end development, code in production environment is usually compressed and obfuscated to reduce code size and improve source code security. However, when JS errors occur in production environments, compression and obfuscated code also make it more difficult to locate errors.

Let’s first look at an error log detected by iQiyi intelligent front-end anomaly monitoring platform (Hawkeye) :

Figure 1. Sample JS errors collected by Hawkeye

As you can see, the compressed error line number and column number have been lost, and the function name has been changed, so it is difficult to determine the true file name of the error. In this way, it is difficult to locate the error according to the error message of the compressed code, but the online code must be compressed and uploaded.

How to balance? SourceMap is a good way to solve this contradiction. This paper introduces the basic concept of Source Map and its application, design and practice in front-end anomaly monitoring.

I. Introduction to Source Map

A Source Map is an information file that stores the mapping between Source code and compiled code, including location information before and after code conversion. Source Map generation is now supported by various major packaging tools, such as Uglify JS, Grunt, Gulp, Webpack, etc. The generation of Source Map is generally carried out in the compilation stage of project packaging. The packaging tool first parses the Source code out of AST (Abstract Syntax Tree), and generates the Source Map using the Source information of nodes in AST during the generation of compiled code, that is, in the process of Generate in FIG. 2.

Figure 2 compiles and generates the Source Map

JS error collection and Source Map parsing errors

JS errors can be collected via window.addeventListener, and frame errors can be collected using hook functions exposed by the framework, such as vue.config.errorHandler. The Error stack can be resolved from error.prototype. stack.

After obtaining the error stack information and Source Map file contents, you can use Mozilla’s Source-Map library to locate errors. The SourceMapConsumer instance in the library represents information about a parsed source map, and we can query the generated source for information about the location of the original file by providing the file location and file content for this instance.

At present, all major monitoring platforms have Source Map parsing function for error monitoring. For example, Sentry supports map files uploaded by the Webpack plug-in and CLI tools, and Fundebug supports map files uploaded by the front-end UI, CLI, and interfaces. However, these platforms also have some disadvantages, such as some do not support users to select files for association matching, and some scenarios are not friendly when users fail to select map matching. Inspired by this, we developed the Source Map function based on hawk-Eye monitoring system, which has the following advantages:

  • Support simpler configuration to upload Source Map file through Webpack Plugin, and upload abnormal process does not affect the overall project compilation process, code invasion is low.

  • More ways to upload and select Source Map files are supported for more business error monitoring scenarios, with clearer prompts and more friendly interactions during use.

  • Supports map file management. You can associate the project name with the version number. The new version can match the map file of the previous version.

The application of Source Map in monitoring system, function design and practice of Hawk-eye Source Map are described in detail below.

Iii. Application of Source Map in monitoring system

When applying Source Map to front-end anomaly monitoring system, the main problem to be solved is how users upload Source Map and how monitoring system associate Source Map with error information. For ease of use, hawk-eye SourceMap can be used in three ways: manually uploading SourceMap for a single JS error stack, automatically uploading associated JS error stack information for the integration plug-in, and selecting associations from the SourceMap list for a single JS error stack. See Figure 3.

Figure 3 Three functions of Hawkeye Source Map

Accordingly, the overall architecture is as follows:

FIG. 4 Overall structure

Here is an introduction to each module:

  • Source Map automatic upload plug-in (support Webpack3+) : Users can integrate this plug-in in the project packaging process, and the generated Source Map can be uploaded automatically. After Webpack prints the asset to the Output directory, this plug-in can fetch the packaged map file and perform the upload logic without affecting the project compilation process. In addition, you can choose to delete the map file after Webpack compilation and execution through parameter configuration, which does not affect the online process. Since historical projects use older versions of Webpack more, Hawk-Eye optimizes compatibility for different versions of Webpack.

  • Manual upload Source Map module: The user can upload a single Source Map for matching of a single JS error, and carry out the prompt of the same name detection. Besides self-upload Map, the prompt can also reduce the situation of incorrect matching.

  • Source Map self-service selection module: it provides the function of selecting maps for association through the Source Map management center. For projects that cannot upload Source Map files every time for each version, it can select Map files to be matched by list and search.

  • Source Map display module: display the real Source code corresponding to the JS error and its location.

  • Source Map Management Center: Centrally manages adding and deleting Source Map files.

The three usage methods mentioned above are complementary, among which the integration plug-in automatically uploads the associated JS error stack information is highly recommended for user access, and its advantage is that it can be easily integrated into the automated build and release process of the project. After a connection, when you receive a JS error, you can view the error in the source code. The realization process is shown in Figure 5:

  • SourceMap automatically uploads the SourceMap file generated when the plug-in pulls the project package.

  • Call the file upload center interface to upload files. The file upload center has an independent authentication mechanism and connects to network security services.

  • Obtain the SourceMap file storage information and submit it to the SourceMap Administrator along with the project and version. After committing, you can delete the Source Map files generated during the build process.

  • When users view JS errors in the front-end exception monitoring system, the monitoring system obtains the SourceMap file from SourceMap management center according to the project and version number of JS errors.

  • Call the Mozillasource- Map library for parsing associations to visualize source code information.

Figure 5 Process design of automatic upload plug-in for Hawkeye Source Map

Iv. In-project access

The methods and tips for adding Source Map automatic upload plug-in to the project are as follows:

  • When initializing the monitoring platform access code, you need to carry the versionNum field, which is the version number of the current project, for later error stack and project map file version management processing.

  • A map file is generated when the project is packaged (a separate map file is required). For Source Map generation configuration, Webpack suggests hidden-source-Map mode, which can generate a complete and independent Map file without disclosing the sourceMappingURL, but the disadvantage is that it will increase compilation time. Cheap -source-map and cheap-module-source-map do not accurately and completely restore the error location because they omit some content.

  • Integrate the Map upload plug-in when packaging the deployment project, install the plug-in in devDependencies, and add the plug-in configuration to the Webpack plugin configuration in the project.

  • If the version number and project name are correctly imported when the monitoring platform is connected, and the map file of the correct version and project name is uploaded through the Map upload plug-in when the project is packaged. When an error occurs, Hawk-Eye will automatically match the JS name with the map file to display the correct source address.

  • If the project cannot be matched automatically, you can select the Map file corresponding to the project and version from the Source Map Management center for matching, or manually upload the Map file for matching.

The usage mode of hawk-eye monitoring background is indicated in Figure 6:

Figure 6 Example of accessing the Hawkeye Source Map function

V. Summary and prospect

In the Source Map view of the Hawkeye management system, you can easily see the real file name and line number, as shown in Figure 7. Because JS error can be directly located to the source code location, access to this function of the project, JS error location screening efficiency has been greatly improved.

Figure 7. Hawkeye JS error SourceMap view

We also encountered some difficulties in generating Source Map for some projects and integrating and uploading the Source Map plug-in:

1. In some projects packaged with custom Webpack configuration, following the official general DevTool :hidden-source-map configuration will always result in the size of some packaged JS files. In addition to the source map file generated separately, Some SourceMap content is mixed in some compiled JS. By looking at the features and contents of these enlarged JS files in detail, we solved the problem by turning off the Source Map option of vue-loader and CSS-loader to specify CSS respectively.

2. Source Map association of micro-front-end projects. In some micro front-end architecture projects, the micro front-end is independently online with independent version control, while the Hawkeye JSSDK that collects errors only needs to be integrated in the base project. The version of the base cannot be obtained in the process of packaging and launching the micro front-end, resulting in the SourceMap file cannot be automatically associated and matched. In a relatively simple way, we support the function of passing in the micro-front-end version in the collection JSSDK. In the base project, the micro-front-end version is obtained by calculation according to the identification of the micro-front-end, etc. When error is detected, the micro-front-end version is delivered together. In this way, during background association, the micro front-end version number is matched first, followed by the base version number, and automatic association can be successful.

3. Complete and independent map files need to be uploaded due to accurate positioning errors. In some large projects, the compilation speed will be affected when Source map files are packaged and generated, and 80% compilation time will be increased in bad cases. Therefore, consider packaging and optimizing SourceMap generation modules independently for each project, rather than implementing its built-in generation process by specifying the options of the packaging tool, to minimize the impact on the overall project packaging time.

In addition, depending on the access project, the Hawkeye Source Map automatic upload plugin will support more packaged build tools such as rollup and gulp.