Issue an overview

A simple scenario is to record the page or component resources used in the project. One way is to write the logic to the Loader so that the resources can be counted by the Loader when compiled. However, in Watch mode, a problem occurs. After modifying a file, the file will go to Loader again. Other files that have not been modified will not go to Loader again, resulting in the loss of resources counted after this compilation.

Scene: the repetition

Demo address: github.com/kotori2000/…

For example, part of the Webpack configuration in demo uses the custom TestWebpackPlugin and _babel-loader.

// webpack.config.js
module.exports = {
  plugins: [
    new TestWebpackPlugin()
  ],
  module: {
    rules: [{test: /\.js$/,
        use: ['_babel-loader'}]}}Copy the code

In addition, the resolution path of the Loader is configured to access the customized Loader

resolveLoader: {
  modules: ['node_modules', path.resolve(__dirname, 'loaders')]},Copy the code

Webpack initializes the compiler. _testObj_ object in thisCompilation hook;

// plugins/TestWebpackPlugin.js
let testObj
if(! compilation.__testObj__) { testObj = compilation.__testObj__ = {modulesMap: {}}}Copy the code

Then the loader stage executes the _babel-loader.js logic, assigns the modulesMap initialized in the previous stage to the constant, and adds the path resolved by loader to the modulesMap object. Loader processes three files in turn.

// loaders/_babel-loader.js
const testObj = this._compilation.__testObj__
  
if(! testObj) {return source
}

const modulesMap = testObj.modulesMap
if(! modulesMap[this.resourcePath]) {
  modulesMap[this.resourcePath] = this.resourcePath
}
Copy the code

This. resourcePath Path to the resource file.

After processing, you go to the Emit hook (at this point modulesMap already has three values) : the key value is the corresponding file path

At this point, changing the contents of the file triggers the next compilation (I’m modifying the B.js file here) :

= >

Only modified files are processed again by _babel-loader.js.

Only the modified paths of B.js are collected.

Cause of the problem

The cache function is enabled by default, and each time the compilation process is triggered, a compilation object will be generated. In fact, the compilation object is the process and data center of each separate compilation, from the start of compilation, file output to the final log output. All associated with compilation. And whether the control is (cache) needs to be compiled code in webpack/lib/Compilation of js addModule method:

/ / webpack/lib/Compilation. Js addModule
addModule(module, cacheGroup) {
    const identifier = module.identifier();
    const alreadyAddedModule = this._modules.get(identifier);
    if (alreadyAddedModule) {
      return {
        module: alreadyAddedModule,
        issuer: false.build: false.dependencies: false
      };
    }
    const cacheName = (cacheGroup || "m") + identifier;
    if (this.cache && this.cache[cacheName]) {
      const cacheModule = this.cache[cacheName];
      if (typeof cacheModule.updateCacheModule === "function") {
        cacheModule.updateCacheModule(module);
      }
      let rebuild = true;
      if (this.fileTimestamps && this.contextTimestamps) {
        rebuild = cacheModule.needRebuild(
          this.fileTimestamps,
          this.contextTimestamps
        );
      }
      if(! rebuild) { cacheModule.disconnect();this._modules.set(identifier, cacheModule);
        this.modules.push(cacheModule);
        for (const err of cacheModule.errors) {
          this.errors.push(err);
        }
        for (const err of cacheModule.warnings) {
          this.warnings.push(err);
        }
        return {
          module: cacheModule,
          issuer: true.build: false.dependencies: true
        };
      }
      cacheModule.unbuild();
      module = cacheModule;
    }
    this._modules.set(identifier, module);
    if (this.cache) {
      this.cache[cacheName] = module;
    }
    this.modules.push(module);
    return {
      module: module.issuer: true.build: true.dependencies: true
    };
  }
Copy the code

There is a needrebuild to determine whether recompiling is required (this.fileTimestamps, this.contextTimestamps: records of last changes to the file stored in the first or previous compilation).

Solutions:

(1) Force loader not to use cache, simple and crude

This way all js files will be compiled again through _babel-laoder no matter how they are modified. After the breakpoint debugging, we can see that after only modifying the b.js file index.js,a.js and B.js are processed again by _babel-laoder.

(2) Rewrite the addModule method

Save the compiled file in _babel-laoder with this._module.buildInfo (each module holds the compiled tag which can be read in plugin when it is removed from the cache), Rewrite the addModule method in TestWebpackPlugin, retain the original logical judgment and add the judgment that the cache needs to be removed (when recompilation is not needed), and manually add the cache file.


const buildInfo = this._module.buildInfo
buildInfo.modulesMap = buildInfo.modulesMap || {}
buildInfo.modulesMap[this.resourcePath] = testObj.modulesMap[this.resourcePath] = this.resourcePath

Copy the code

This. _module is a hack. Used to access the currently loaded Module object.

const rawAddModule = compilation.addModule
      compilation.addModule = (. args) = > {
        const addModuleResult = rawAddModule.apply(compilation, args)
        if(! addModuleResult.build && addModuleResult.issuer) {const buildInfo = addModuleResult.module.buildInfo
          if (buildInfo.modulesMap) {
            Object.assign(testObj.modulesMap, buildInfo.modulesMap)
          }
        }
        return addModuleResult
      }
    })
Copy the code

BuildInfo caches the file path that has been processed. The file path is too long to show the file name. The file path is index.js, a.js, b.js.

When modifying the b.js file, the index.js and a.js files will use this logic to remove the markup stored in the module because of caching.

The emit hook will emit all modules successfully after _babel-loader.js processing:

conclusion

However, in this demo scenario, the loader also added other processing logic, such as collecting modulesMap, which caused the failure. Depending on the situation, this can be avoided by mounting the modulesMap directly into the Compiler object if the process of collecting the modulesMap has nothing to do with adding or removing dependencies.