The introduction

In the daily development process, a common component is often modified. The common component may be referenced in multiple places, so the modification of the component may affect multiple business scenarios. However, as the project has just been taken over or the span of time increases, It is extremely difficult to find out all the scenarios in which this component is introduced by human resources. We can only search the component name constantly. Even the component that is introduced by this component may be introduced by multiple scenarios, which may lead to the failure of regression testing to cover all the scenarios, leading to online problems

The solution

The concept statement

First of all, let’s declare a few concepts to help you understand them:

1. Module: Module is a discrete function block, which provides a smaller contact surface than the complete program. Well-written modules provide solid boundaries of abstraction and encapsulation, allowing each module in an application to have a coherent design and a clear purpose. In simple terms, the code is not compiled before, we write a file is a module.

Chunk: This Webpack-specific term is used internally to manage the bundling process. Output bundles consist of blocks, of which there are several types (such as Entry and child). Typically, blocks correspond directly to output bundles, but some configurations do not produce one-to-one relationships. To put it simply, webpack generates chunk files according to the file reference relationship. Basically, an entry file corresponds to a chunk.

3. Bundles: Bundles are generated by many different modules and contain the final version of the source files that have been loaded and compiled. After the chunk files are processed by Webpack, the generated code that runs in the browser is called the bundle. It should be noted that in theory chunk and bundle correspond one by one, but when you configure code separation, code extraction, etc., a chunk will generate multiple bundles according to the configuration

The process of establishing dependencies

In the construction stage, the resources and resource dependencies were analyzed recursively from entry, and module sets and dependencies between modules were gradually built within compilation objects. The core process was as follows:

To clarify, the build phase starts with the entry file:

Call handleModuleCreate to build module subclasses based on the file type
The runLoaders translation Module content that calls the Loader-Runner repository is typically translated from various resource types to JavaScript text
Call Acorn to parse the JS text into an AST
Iterate through the AST, triggering various hooks
1. inHarmonyExportDependencyParserPluginPlug-in to monitorexportImportSpecifierHooks that interpret the resource dependencies corresponding to the JS text
2. callmoduleThe object’saddDependencyAdd the dependent object tomoduleIn the dependency list
After the AST traverses, it calls module.handleParseResult to handle module dependencies
For new dependencies on The Module, call handleModuleCreate to control the flow back to the first step
After all dependencies have been resolved, the build phase ends

Module => AST => Dependences => module This requires that the final result of loaders must be a standard JavaScript syntax that can be processed by Acorn. For example, for images, the image binary needs to be converted to something like export default “data:image/ PNG; Base64, XXX “or export default “http://xxx” url format.

Compilation works recursively through this process, gradually resolving the contents of each module and module dependencies so that output can be packaged around those contents.

Example: Hierarchy

Imagine a file dependency tree like the following:Among themindex.js 为 entryFiles, depending on a/ B files; A depends on c/ D files. After initializing the build environment,EntryPluginAccording to theentryConfiguration to findindex.jsFile, callcompilation.addEntryFunction triggers the build process, which generates internal data structures like this:

The contents of module[index.js] and dependence objects are obtained from dependence[A.js] and dependence[B.js]. A. js, B. js, continue to call the handleParseResult function of module[index.js] according to the logic of the above flow chart, continue to process the files of A. js and B. js, recurse the above flow, and further obtain modules A and B:

C. js/ D. js dependencies are resolved from the A. js module, so we call handleParseResult on the Module [A.js] again and recurse the above process:

Once you’ve parsed all the modules at this point and found no more new dependencies, you can move on.

From the build process, it is clear how WebPack retrieves all dependencies, and it also keeps a clear record of these relationships. Let’s talk about the StatA object.

Stats configuration

Stats: Controls how WebPack prints package results for development or production environments. These statistics not only help developers analyze dependency charts for applications, but also optimize compilation speed. The JSON file can be generated by using the following command:

webpack --profile --json > stats.json
Copy the code

The stats.json file contains information that can be configured in the configuration file. Here are its configuration items, each with its own default value:

module.exports={
  ...
  stats: {
  
    // Fallback value of stats option when no option is defined (priority is higher than webpack local default)
    all: undefined.// Add resource information
    assets: true.// Sorts resources by the specified field
    // You can use '! Field 'to reverse the sort.
    assetsSort: "field".// Add build date and build time information
    builtAt: true.// Add cached (but not built) module information
    cached: true.// Display cached resources (set it to 'false' to display only output files)
    cachedAssets: true.// Add children information
    children: true.// Add chunk information (set to 'false' to allow less verbose output)
    chunks: true.// Add the building module information to the chunk information
    chunkModules: true.// Add information about the source of chunk and Chunk merge
    chunkOrigins: true.// Sort chunks by the specified field
    // You can use '! Field 'to reverse the sort. The default is to sort by id.
    chunksSort: "field".// The context directory used to shorten the request
    context: ".. /src/".// 'webpack --colors' equals
    colors: false.// Display the distance of each module to the entry starting point
    depth: false.// Display the starting point of the entry through the corresponding bundle
    entrypoints: false.// Add --env information
    env: false.// Add an error message
    errors: true.// Add error details (just like parsing logs)
    errorDetails: true.// Exclude the case where the resource is displayed in stats
    // This can be done by using the String RegExp function to get the assetName
    // And returns a Boolean value or an array as described below.
    excludeAssets: "filter" | /filter/ | (assetName) = >.return true|false |
      ["filter""|"/filter/"|"(assetName) = >.return true|false].// Remove the module from stats
    // This can be done using the String, RegExp, moduleSource function
    // And returns a Boolean value or an array as described below.
    excludeModules: "filter" | /filter/ | (moduleSource) = >.return true|false |
      ["filter""|"/filter/"|"(moduleSource) = >.return true|false].// Same as excludeModules
    exclude: "filter" | /filter/ | (moduleSource) = >.return true|false |
      ["filter""|"/filter/"|"(moduleSource) = >.return true|false].// Add the hash of the compilation
    hash: true.// Sets the maximum number of modules to display
    maxModules: 15.// Add building module information
    modules: true.// Sort the modules by the specified field
    // You can use '! Field 'to reverse the sort. The default is to sort by id.
    modulesSort: "field".// Show warning/error dependencies and sources (starting with Webpack 2.5.0)
    moduleTrace: true.// Display performance prompts when the file size exceeds' performance.maxAssetSize '
    performance: true.// Display module export
    providedExports: false.// Add information about the public path
    publicPath: true.// Add the reason why the module was introduced
    reasons: true.// Add the source code for the module
    source: true.// Add time information
    timings: true.// Show which module export is used
    usedExports: false.// Add webPack version information
    version: true.// Add a warning
    warnings: true.// Filter warning display (starting with WebPack 2.4.0),
    // Can be String, Regexp, a function to get warning
    // And returns a Boolean value or a combination of the above. The First match is won.
    warningsFilter: "filter" | /filter/ | ["filter"./filter/] | (warning) = >.return true|false}};Copy the code

Stats. Json file

Structure (Structure).

The outermost output JSON file is easier to understand, but there is still a small portion of nested data that is not so easy to understand. Rest assured, each part of this is explained in more detail later.

{"version": "1.4.13", // The version of webpack to compile "hash": "11593e3b3AC85436984A ", // The hash to compile "time": 2469, // Compile time (ms) "filteredModules": 0, // Exclude 'toJson' to count the number of modules ignored "outputPath": "/", // path to webpack output directory path "assetsByChunkName": {// The name of the chunk used for the mapping "main": "web.js? h=11593e3b3ac85436984a", "named-chunk": "named-chunk.web.js", "other-chunk": ["other-chunk.js", "other-chunk. CSS "]}, "assets": [// Array of asset objects], "chunks": [// Array of Chunk objects], "modules": [// array of module objects], "errors": [// Array of error string], "warnings": [// array of warning string]}Copy the code

Asset Objects

Each assets object represents a compiled output file. Assets have a common structure:

{"chunkNames": [], // The emitted id of the chunk "chunks" that this asset contains: [10, 6], // The emitted ID of the chunk that this asset contains: "Name ": "10.web.js", // Output file name" size": 1058 // file size}Copy the code

Chunk Objects

Each chunk represents a group of modules called chunks. Each object satisfies the following structure.

{" Entry ": true, // indicates whether the chunk contains the webpack runtime "files": [// an array containing the chunk's file names], "filteredModules": 0, // See the structure "id": 0, // the id of the chunk "initial": true, // indicates whether the chunk is loaded from the beginning or lazily loading "modules": [// Module objects array "web.js? H = 11593e3b3AC85436984A "], "names": [// An array of names of chunks contained in this chunk], "origins": [// more on that below], "parents": [], // Ids of the parent chunk "Rendered ": "Size ": 188057 // Chunk size (byte)}Copy the code

The chunks object will also contain a “origin” to indicate where each chunk came from. The origins are in the following form

{"loc": "", // which line generates this chunk "module": "(webpack)\test\browsertest\lib\index.web.js", // location of the module" moduleId": 0, // Module ID "moduleIdentifier": "(webpack)\test\browsertest\lib\index.web.js", // module address "moduleName": "./lib/index.web.js", // module relative address "name": "main", // chunk name" reasons": [// module object 'reason' array]}Copy the code

Module Objects

Each module in the dependency chart can be expressed in the following form, which is the part we need to focus on. The dependency information between modules is in this part.

{"assets": [// an array of asset objects], "built": true, // indicates that this module will participate in Loaders, parse, and be compiled with "cacheable": "Errors ": 0, // Number of errors found in this module "failed": "Id ": 0, // This module's ID (similar to 'module.id') "identifier": "(webpack)\test\browsertest\lib\index.web.js", // the unique identifier used within webpack "name": "./lib/index.web.js", // The actual file address "optional": false, // every request to this module will be wrapped in 'try... // Whether this module can be prefetched" profile": {// build "module "dependencies" factory": < span class =" font-size: 14px; }, "reasons": [// see below description], "size": 3593, // "source": "// Should not break it... \r\nif(typeof..." // The string input "warnings": 0 // the number of warnings to handle the module}Copy the code

Each module contains a reasons object that describes the reason for the module to be included in the dependency diagram. Each reason is similar to the origins of chunk objects mentioned above:

{"loc": "33:24-93", // The number of lines of code causing this dependency icon to be added "module": "./lib/index.web.js", // the relative address of the module based on the context "moduleId": 0, // Module ID "moduleIdentifier": "(webpack)\test\browsertest\lib\index.web.js", // module address "moduleName": "./lib/index.web.js", // better readable module name (used for "pretty-printing ") "type": "Require. Context ", // Type of request "userRequest": ".. /.. Cases "// source string for 'import' or 'require'}Copy the code

Errors and Warnings

Errors and warnings contain an array of strings. Each string contains information and a stack trace:

. /cases/parsing/browserify/index.js Critical dependencies: 2:114-121 This seem to be a pre-built javascript file. Even while this is possible, it's not recommended. Try to require to orginal source to get better results. @ .. /cases/parsing/browserify/index.js 2:114-121Copy the code

Best practices

Give a summary, methodology, routine, let the audience have a strong sense of growth and gain.

To sum up, the implementation process is as follows:

Add stats configuration to the webpack.config.js file, such as:

module.exports={
  ...
  stats: {chunkModules: false.chunks: false.modules: true.children: false.exclude: [/.*node_modules\/.*/]}}Copy the code

Run: webpack –config webpack.config.js –profile –json > stats.json

–profile: Provides specific compilation times for each step

— JSON: Outputs the compilation information to WEN as a JSON file

This generates a stats.json file in the root directory of the project, the sibling of webpack.config.js.

Online analysis enables us to have a comprehensive analysis of the construction results

Webpack formally provides Webpack Analyse, an online Web application.

Open the Webpack Analyse page and a pop-up prompts you to upload the JSON file, which is the stats. JSON file mentioned above, as shown in the figure below:

Webpack Analyse doesn’t develop the stats.json file of your choice to the server, it is parsed locally in the browser and you don’t have to worry about your code being compromised. After selecting the file, you should immediately see the following image (see the Examples provided on this page) :

Modules: Shows all Modules, each module corresponds to a file. It also contains the dependency diagram, module path, module ID, Chunk of the module and module size of all modules.

Chunks: Shows all code blocks, including multiple modules in one code block. It also contains the code block ID, name, size, number of modules each code block contains, and a dependency diagram between code blocks;

Assets: Displays all output file resources, including.js,.cssPictures, etc. It also includes the file name, size, and block of code from which the file came;

Warnings: Displays all Warnings that occur during the build.

Errors: Displays all error messages that occur during the build.

Hints: Shows how long it takes to process each module.

Our focus will be on the Module functionality, where Module dependencies are shown:

Each node in the diagram represents a module, and the following list lists all modules, one by one. When we need to find the dependencies of a module, click the Module in the list, and the dependencies before and after will be marked in the diagram, for example:

Black nodes represent the current node, green represents the modules on which the current Module depends, and red represents the nodes on which the current node is dependent.

Of course, such a dependent query can only see the relationship between modules before and after. When we modify a component, we naturally want to start from the node and search up to the top component. This is what we need, but we need to handle it manually.

By manual parsing, you can retrieve the dependency chain of the component

We already know that the stats.json generated above contains the dependency information we need, so when we need to reverse derive our test scope, what we need to do is to extract some of the data from the file structure and build the data structure we need.

Focus on the Module attribute, and the Reasons attribute for each object in the Module: This property contains where the component is dependent, so the general idea is to build a tree structure, starting with the target module as the root node, and iterate over its reasons properties to find which components depend on it, and then find the Module in the Module array and iterate over its reasons properties. Until you finally find the top-level component. Such a tree structure is generated from the bottom up, which is not conducive to finding affected components, so we need to traverse the tree structure and get every path, so we can get all paths from the top down.

const fs = require('fs')
const loadsh=require('loadsh')

fs.readFile('.. /.. /nodeServerClient/stats.json'.'utf8' , (err, data) = > {
  if (err) {
    console.error(err)
    return
  }
  const fileObj=JSON.parse(data);// Convert the read file contents into JSON objects
  const moduleArr=fileObj.modules;// Get an array of module data
  const moduleObj={};
  moduleArr.forEach(ele= > {    // Convert the array to an attribute of the object. The key value is the nameForCondition attribute of each item in the array
    moduleObj[ele.nameForCondition]=ele;
  });
 let filename='/Users/bytedance/workSpace/personDir/vscode_workspace/nodeServerClient/src/controller/ser.js'
  const tree=[];
  // Retrieve the tree structure from the changed module
  function createTree(filename,tree){
    // Get the changed Module object
    /* const targetmodule=moduleObj[Object.keys(moduleObj).filter(ele=>{ return ele.includes(filename)||ele===filename; }) [0]]; * /

    const targetmodule=moduleObj[filename];
    if(! targetmodule){console.log('Not obtained:${filename}Module `);return;
    };
    // Check whether the module has been added to the current level, because a module that depends on another module will appear multiple times in the module's reasons
    let isHaveTarget=tree.filter(item= >{
        return item.name===targetmodule.nameForCondition
    });
    if(isHaveTarget&&isHaveTarget.length>0) {return;
    }
    // Put the module into the node
    tree.push({
        name:targetmodule.nameForCondition,
        children:[]
    });
    
    if(targetmodule.reasons&&targetmodule.reasons.length>0) {for(let item of targetmodule.reasons){
            // Determine the termination condition
            if(item.type! = ='entry'&& item.resolvedModuleIdentifier! ==tree[tree.length-1].name){
                createTree(item.resolvedModuleIdentifier,tree[tree.length-1].children); }}}else{
        return; }}// Create a tree structure for the root node with the modified component
  createTree(filename,tree);
  // Get which components depend on the component
  console.log('= = = = = ='.JSON.stringify(tree));
  const pathArr=[];// An array of all paths
  // Prints an array of all paths in the tree structure
  function getTreeAllPath(tree){
      
      function getData(tree,path){
        tree.forEach(ele= > {
            if(ele.children&&ele.children.length>0){
                path.push(ele.name);
                getData(ele.children,path);
                path.pop();
            }else{ path.push(ele.name); pathArr.push(loadsh.cloneDeep(path)); path.pop(); }}); } getData(tree,[]); } getTreeAllPath(tree);// Each entry in an array is also an array, reversing the order of the array
  pathArr.forEach(item= >{
      item.reverse();
  })
  console.log('+ + + + + +'.JSON.stringify(pathArr));
})
Copy the code

future

During yarn commit, determine whether to retrieve the function of the impact module. If yes, export the related service function of the impact module based on the submitted file, and determine a reasonable test scope. 2. When conducting Git MR, export related service functions that are affected according to the merged files, and perform the last step to check whether there are any missing scenarios in the test.

The resources

Segmentfault.com/a/119000003…

Gitmind. Cn/app/doc/fac…

Webpack. Wuhaolin. Cn / 4% BC E4% % 98%…

www.webpackjs.com/api/stats/#…

webpack.github.io/analyse/

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Analysis and application of front-end module dependency

Meta

Abstract

The outline

The audience gains

The introduction

The solution

The concept statement

The process of establishing dependencies

Example: Hierarchy

Stats configuration

Stats. Json file

Structure (Structure).

Asset Objects

Chunk Objects

Module Objects

Errors and Warnings

Best practices

Online analysis enables us to have a comprehensive analysis of the construction results

By manual parsing, you can retrieve the dependency chain of the component

future

The resources

Analysis and application of front-end module dependency

Meta

Abstract

The outline

The audience gains

The introduction

The solution

The concept statement

The process of establishing dependencies

Example: Hierarchy

Stats configuration

Stats. Json file

Structure (Structure).

Asset Objects

Chunk Objects

Module Objects

Errors and Warnings

Best practices

Online analysis enables us to have a comprehensive analysis of the construction results

By manual parsing, you can retrieve the dependency chain of the component

future

The resources

Related Posts

The common API of String

React 1

[react.js] How to automatically cancel asynchronous requests for component uninstallation