Webpack series 1: Common loader source code analysis, and hands-on implementation of a MD2HTml-loader

Webpack Series 2: Discover how the WebPack plug-in works

Webpack series 3: Webpack main process source code reading and implementation of a Webpack

preface

Plugins allow you to extend WebPack and, at the right moment, change the output through the API provided by WebPack, enabling WebPack to perform a wider range of tasks and build capabilities. This article will try to explore the workflow of the WebPack plug-in to uncover how it works. It also requires some understanding of the underlying WebPack and build process.

To understand the mechanics of webPack plug-ins, here are a few things to understand:

  1. A simple plug-in composition
  2. webpackThe build process
  3. TapableHow are plugins strung together
  4. compilerAs well ascompilationObjects and their corresponding event hooks.

Basic plug-in structure

Plugins are objects that can be instantiated using their own stereotype method Apply. Apply is executed by the Webpack Compiler only once when the plug-in is installed. The Apply method passes a reference to the WebPCK Compiler to access the compiler callback.

A simple plug-in structure:

class HelloPlugin{
  // Get the configuration passed in by the user to the plug-in in the constructor
  constructor(options){
  }
  // Webpack calls the Apply method of the HelloPlugin instance and passes the Compiler object to the plug-in instance
 apply(compiler) {  // Insert hook functions in the EMIT phase to process additional logic at specific times;  compiler.hooks.emit.tap('HelloPlugin', (compilation) => {  // Call the webPack callback function when the functional flow is complete;  });  If the event is asynchronous, it takes two parameters. The second parameter is the callback function, which is called to notify WebPack when the plug-in has finished processing the task before moving on to the next processing flow.  compiler.plugin('emit'.function(compilation, callback) {  // Support processing logic  // Execute callback to notify Webpack  // If callback is not executed, the running process will remain stuck at this point  callback();  });  } }  module.exports = HelloPlugin; Copy the code

To install a plug-in, simply place an instance of it in the Webpack Config plugins array:

const HelloPlugin = require('./hello-plugin.js');
var webpackConfig = {
  plugins: [
    new HelloPlugin({options: true})
  ]
}; Copy the code

Let’s take a look at how the WebPack Plugin works

  1. The configuration is read firstnew HelloPlugin(options)Initialize oneHelloPluginGet an example.
  2. Initialize thecompilerObject after callHelloPlugin.apply(compiler)Pass in the plug-in instancecompilerObject.
  3. The plug-in instance is being retrievedcompilerAfter the object can be passedCompiler.plugin (event name, callback function)Listen for events broadcast by Webpack. And can be accessed throughcompilerObject to operateWebpack.

Webapck build process

Before writing the plug-in, you also need to understand the Webpack build process so that you can plug in the right plug-in logic at the right time.

The basic build process for Webpack is as follows:

  1. Verify configuration files: read command line incoming orwebpack.config.jsFile that initializes the configuration parameters for this build
  2. generateCompilerObject: Executes the plug-in instantiation statement in the configuration filenew MyWebpackPlugin()forwebpackEvent flow hang customhooks
  3. Enter theentryOptionPhase:webpackStart reading configurationEntries, recursively traverses all entry files
  4. run/watch: If run inwatchMode executeswatchMethod, otherwise executerunmethods
  5. compilation: createCompilationObject callbackcompilationRelated hooks to enter each entry file in turn (entry), use loader to compile the file. throughcompilationI can read itmoduletheresource(Resource path),loaders(Loader used). Then use the compiled file contentsacornParsing generates an AST static syntax tree. This process is then recursively and repeatedly executed after all modules and dependencies are analyzedcompilation 的 sealMethod To sort, optimize and encapsulate each chunk__webpack_require__To simulate modular operations.
  6. emit: All files have been compiled and converted, containing the final output resources that we can call back to in the incoming eventcompilation.assetsGet the required data, including the resources to be exported, chunks of code, and so on.
// Modify or add resources
compilation.assets['new-file.js'] = {
  source() {
    return 'var a=1';
  },
 size() {  return this.source().length;  } }; Copy the code
  1. afterEmit: The file has been written to the disk
  2. doneFinish compiling

Attached is a WebPack compilation flow chart of Didi Cloud blog. If you don’t like to see the text, you can see the flow chart to understand and memorize

WebPack compile flow chart The original image from: blog.didiyun.com/index.php/2…

If you still don’t understand the webpack construction process, you are advised to read the full text and come back to this paragraph. I believe you will have a much deeper understanding of the Webpack construction process.

Understand the event flow mechanism tapable

Webpack is essentially an event flow mechanism, and its workflow is to chain plug-ins together, with Tapable at the heart of it all.

The Tapable event flow mechanism of Webpack ensures the orderness of plug-ins. When all plug-ins are connected in series, Webpack will broadcast events during the running process. Plug-ins only need to listen to the events they care about, so they can join the Webapck mechanism to change the operation of Webapck. Make the whole system expansibility good.

Tapable is also a small library that is a core tool for Webpack. Similar to the Events library in Node, the core principle is a subscription publishing model. The purpose is to provide similar plug-in interfaces.

The Compiler at the core of Webpack and the Compilation that creates bundles are both instances of Tapable and can broadcast and listen for events directly on the Compiler and Compilation objects as follows:

/ * ** Broadcast events* event-name specifies the event name. Ensure that the name is different from that of an existing event* /
compiler.apply('event-name',params);
compilation.apply('event-name',params); / * ** Listen events* / compiler.plugin('event-name'.function(params){}); compilation.plugin('event-name'.function(params){}); Copy the code

The Tapable class exposes the TAP, tapAsync, and tapPromise methods, and you can choose a function injection logic based on how the hook is synchronous/asynchronous.

Tap Sync hook

compiler.hooks.compile.tap('MyPlugin', params => {
  console.log('touches the compile hook synchronously. ')
})
Copy the code

TapAsync asynchronous hook, which tells Webpack asynchronous execution is over via callback tapPromise Asynchronous hook, returns a Promise telling Webpack asynchronous execution is over

compiler.hooks.run.tapAsync('MyPlugin', (compiler, callback) => {
  console.log('touches the Run hook asynchronously. ')
  callback()
})

compiler.hooks.run.tapPromise('MyPlugin', compiler => {  return new Promise(resolve= > setTimeout(resolve, 1000)).then((a)= > {  console.log('Reach the Run hook asynchronously with delay')  }) }) Copy the code

Tapable usage

const {
 SyncHook,
 SyncBailHook,
 SyncWaterfallHook,
 SyncLoopHook,
 AsyncParallelHook,  AsyncParallelBailHook,  AsyncSeriesHook,  AsyncSeriesBailHook,  AsyncSeriesWaterfallHook  } = require("tapable"); Copy the code
tapable

Simply implement a SyncHook

class Hook{
    constructor(args){
        this.taps = []
        this.interceptors = [] // Use this in the back
        this._args = args 
 }  tap(name,fn){  this.taps.push({name,fn})  } } class SyncHook extends Hook{  call(name,fn){  try {  this.taps.forEach(tap= > tap.fn(name))  fn(null,name)  } catch (error) {  fn(error)  }   } } Copy the code

tapableHow willwebapck/webpackPlug-in associated?

Compiler.js

const { AsyncSeriesHook ,SyncHook } = require("tapable");
/ / create the class
class Compiler {
    constructor() {
        this.hooks = {
 run: new AsyncSeriesHook(["compiler"]), // Asynchronous hooks  compile: new SyncHook(["params"]),// Synchronize the hook  };  },  run(){  // Execute asynchronous hooks  this.hooks.run.callAsync(this, err => {  this.compile(onCompiled);  });  },  compile(){  // Execute the sync hook and pass the parameter  this.hooks.compile.call(params);  } } module.exports = Compiler Copy the code

MyPlugin.js

const Compiler = require('./Compiler')

class MyPlugin{
    apply(compiler){// Accept the compiler argument
        compiler.hooks.run.tap("MyPlugin", () = >console.log('Start compiling... '));
 compiler.hooks.complier.tapAsync('MyPlugin', (name, age) => {  setTimeout((a)= > {  console.log('Compiling... ')  }, 1000)  });  } }  This is similar to the plugins configuration for webpack.config.js // Pass a new instance to the plugins property  const myPlugin = new MyPlugin();  const options = {  plugins: [myPlugin] } let compiler = new Compiler(options) compiler.run() Copy the code

For an in-depth look at Tapable’s articles, check out this post:

Webpack4 tapable core module source code parsing: https://www.cnblogs.com/tugenhua0707/p/11317557.html

Understanding Compiler (responsible for compiling)

What do compiler and compilation objects do when developing plug-ins

The Compiler object contains the current configuration for running Webpack, including entry, output, loaders, etc. This object is instantiated when Webpack is started and is globally unique. Plugin can use this object to obtain Webpack configuration information for processing.

If you still don’t understand what compiler does, don’t be afraid to read on. Run NPM run build to print all the compiler information to console.log(Compiler) on the console.

compiler
// In order to make it easier to see the structure of compiler, ellipses are used in a lot of code. Instead.
Compiler {
  _pluginCompat: SyncBailHook {
.  },
 hooks: {  shouldEmit: SyncBailHook { . },  done: AsyncSeriesHook { . },  additionalPass: AsyncSeriesHook { . },  beforeRun: AsyncSeriesHook { . },  run: AsyncSeriesHook { . },  emit: AsyncSeriesHook { . },  assetEmitted: AsyncSeriesHook { . },  afterEmit: AsyncSeriesHook { . },  thisCompilation: SyncHook { . },  compilation: SyncHook { . },  normalModuleFactory: SyncHook { . },  contextModuleFactory: SyncHook { . },  beforeCompile: AsyncSeriesHook { . },  compile: SyncHook { . },  make: AsyncParallelHook { . },  afterCompile: AsyncSeriesHook { . },  watchRun: AsyncSeriesHook { . },  failed: SyncHook { . },  invalid: SyncHook { . },  watchClose: SyncHook { . },  infrastructureLog: SyncBailHook { . },  environment: SyncHook { . },  afterEnvironment: SyncHook { . },  afterPlugins: SyncHook { . },  afterResolvers: SyncHook { . },  entryOption: SyncBailHook { . },  infrastructurelog: SyncBailHook { . }  }, . outputPath: ' '.// Output directory  outputFileSystem: NodeOutputFileSystem { . },  inputFileSystem: CachedInputFileSystem { . }, . options: {  //Compiler object contains all configuration information for WebPack, including entry, Module, output, resolve, etc  entry: [  'babel-polyfill'. '/Users/frank/Desktop/fe/fe-blog/webpack-plugin/src/index.js' ]. devServer: { port: 3000 },  output: { . },  module: { . },  plugins: [ MyWebpackPlugin {} ],  mode: 'production'. context: '/Users/frank/Desktop/fe/fe-blog/webpack-plugin'. devtool: false.. performance: {  maxAssetSize: 250000. maxEntrypointSize: 250000. hints: 'warning'  },  optimization: { . },  resolve: { . },  resolveLoader: { . },  infrastructureLogging: { level: 'info'.debug: false }  },  context: '/Users/frank/Desktop/fe/fe-blog/webpack-plugin'.// Context, file directory  requestShortener: RequestShortener { . }, . watchFileSystem: NodeWatchFileSystem {  // Listen for file change list information . } } Copy the code

Compiler source condensed version code parsing

Source address line (948) : https://github.com/webpack/webpack/blob/master/lib/Compiler.js

const { SyncHook, SyncBailHook, AsyncSeriesHook } = require("tapable");
class Compiler {
  constructor() {
    // 1. Define the lifecycle hook
    this.hooks = Object.freeze({
 / /... Hook, hook, hook, hook, hook, hook, hook  done: new AsyncSeriesHook(["stats"]),// Execute after compiling  beforeRun: new AsyncSeriesHook(["compiler"]),  run: new AsyncSeriesHook(["compiler"]),Execute before the compiler starts reading records  emit: new AsyncSeriesHook(["compilation"]),// This is executed before generating files to the output directory, calling the compilation parameter  afterEmit: new AsyncSeriesHook(["compilation"]),// execute after generating files to the output directory  compilation: new SyncHook(["compilation"."params"]),// Execute plug-ins after a compilation creation  beforeCompile: new AsyncSeriesHook(["params"]),  compile: new SyncHook(["params"]),// Before a new compilation is created  make:new AsyncParallelHook(["compilation"]),// Execute before completing a compilation  afterCompile: new AsyncSeriesHook(["compilation"]),  watchRun: new AsyncSeriesHook(["compiler"]),  failed: new SyncHook(["error"]),  watchClose: new SyncHook([]),  afterPlugins: new SyncHook(["compiler"]),  entryOption: new SyncBailHook(["context"."entry"])  });  / /... Omit code  }  newCompilation() {  // Create a Compilation object to call back the Compilation related hooks  const compilation = new Compilation(this);  / /... Sequence of operations  this.hooks.compilation.call(compilation, params); // Compilation object creation is complete  return compilation  }  watch() {  // Execute the watch method if running in watch mode, otherwise execute the run method  if (this.running) {  return handler(new ConcurrentCompilationError());  }  this.running = true;  this.watchMode = true;  return new Watching(this, watchOptions, handler);  }  run(callback) {  if (this.running) {  return callback(new ConcurrentCompilationError());  }  this.running = true;  process.nextTick((a)= > {  this.emitAssets(compilation, err => {  if (err) {  // The failed event is emitted when an exception is encountered in the compilation and output process  this.hooks.failed.call(err)  };  if (compilation.hooks.needAdditionalPass.call()) {  // ...  // done: the compilation is complete  this.hooks.done.callAsync(stats, err => {  // Before creating the compilation object  this.compile(onCompiled);  });  }  this.emitRecords(err= > {  this.hooks.done.callAsync(stats, err => {   });  });  });  });   this.hooks.beforeRun.callAsync(this, err => {  this.hooks.run.callAsync(this, err => {  this.readRecords(err= > {  this.compile(onCompiled);  });  });  });   }  compile(callback) {  const params = this.newCompilationParams();  this.hooks.beforeCompile.callAsync(params, err => {  this.hooks.compile.call(params);  const compilation = this.newCompilation(params);  // Trigger the make event and call addEntry to find the entry js and proceed to the next step  this.hooks.make.callAsync(compilation, err => {  process.nextTick((a)= > {  compilation.finish(err= > {  // Encapsulate the build result (SEAL), and sort each module and chunk successively. Each chunk has an entry file  compilation.seal(err= > {  this.hooks.afterCompile.callAsync(compilation, err => {  // Asynchronous events need to call a callback function to inform Webpack to proceed to the next flow when the plug-in has finished processing the task.  // Otherwise the process will stay stuck  return callback(null, compilation);  });  });  });  });  });  });  }  emitAssets(compilation, callback) {  const emitFiles = (err) = > {  / /... Leave out a bunch of code  // afterEmit: The file has been written to disk  this.hooks.afterEmit.callAsync(compilation, err => {  if (err) return callback(err);  return callback();  });  }   When the EMIT event occurs, the final output resources, code blocks, modules and their dependencies can be read and modified (this is the last chance to modify the final file)  this.hooks.emit.callAsync(compilation, err => {  if (err) return callback(err);  outputPath = compilation.getPath(this.outputPath, {});  mkdirp(this.outputFileSystem, outputPath, emitFiles);  });  }  / /... Omit code } Copy the code

The general form for inserting hooks in the apply method is as follows:

// Compiler provides compiler.hooks that make the plug-in do different things at different times.
Compiler.hooks. Phase. Tap function ('Plug-in name', (phase callback parameter) => {
});
compiler.run(callback)
Copy the code

Understand Compilation (responsible for creating bundles)

The Compilation object represents a resource version build. When running the WebPack development environment middleware, each time a file change is detected, a new compilation is created, resulting in a new set of compilation resources. A Compilation object represents the current module resources, compile-generated resources, changing files, and state information about the dependencies being tracked, simply storing the compiled content in memory. The Compilation object also provides callbacks to plugins that need to customize their functions, so that plug-ins can choose to use extensions when doing their own customization.

In a nutshell,Compilation is about building modules and chunks and optimizing the build process with plug-ins.

As with Compiler, tapAsync and tapPromise can also be accessed on some hooks, depending on the hook type.

Console outputconsole.log(compilation)

Compiler objects can also be read from the Compilation.

Source code more than 2000 lines, can not see the move –, interested in their own look. https://github.com/webpack/webpack/blob/master/lib/Compilation.js

This section introduces several commonly used Compilation Hooks

hook type When to call
buildModule SyncHook Triggered before the module starts compiling and can be used to modify the module
succeedModule SyncHook This hook is executed when a module is successfully compiled
finishModules AsyncSeriesHook Called when all modules have compiled successfully
seal SyncHook When acompilationTriggered when it stops receiving new modules
optimizeDependencies SyncBailHook At the beginning of dependency optimization
optimize SyncHook At the beginning of the optimization phase
optimizeModules SyncBailHook When executed at the beginning of the module optimization phase, plugins can perform module optimization in this hook with the callback argument:modules
optimizeChunks SyncBailHook Executed at the beginning of the block optimization phase, the plugin can perform the block optimization in this hook with the callback argument:chunks
optimizeChunkAssets AsyncSeriesHook Optimize any code block resources that are storedcompilation.assetsOn. A chunk has a files property, which points to all files created by a chunk. Any additional chunk resources are storedcompilation.additionalChunkAssetsOn. Callback parameters:chunks
optimizeAssets AsyncSeriesHook Optimize all storescompilation.assetsOf all resources. Callback parameters:assets

Compiler differs from Compilation

Compiler represents the entire Webpack lifecycle from startup to shutdown, whereas Compilation simply represents a new Compilation and is recreated whenever a file changes.

Commonly used API

Plug-ins can be used to modify output files, add output files, even improve Webpack performance, and so on, but plug-ins can do a lot of things by calling the APIS provided by Webpack. Because Webpack provides a large number of apis, there are many that are rarely used, and space is limited, so here are some commonly used apis.

Read output resources, code blocks, modules and their dependencies

Some plug-ins may need to read the results of Webpack processing, such as output resources, code blocks, modules and their dependencies, for further processing. At the time of the EMIT event, the conversion and assembly of the source file has been completed and the final output resources, code blocks, modules and their dependencies can be read, and the contents of the output resources can be modified. The plug-in code is as follows:

class Plugin {
  apply(compiler) {
    compiler.plugin('emit'.function (compilation, callback) {
      // compilation.chunks is an array that stores all code blocks
      compilation.chunks.forEach(function (chunk) {
 // chunk represents a code block  // A code block consists of several modules, each of which can be read by chunk.forEachModule  chunk.forEachModule(function (module) {  // module represents a module  // module.fileDependencies An array of dependencies for the current module  module.fileDependencies.forEach(function (filepath) {  });  });   // Webpack will generate output file resources according to Chunk, each Chunk corresponds to one or more output files  // For example, if the Chunk contains CSS modules and uses ExtractTextPlugin,  // The Chunk generates.js and.css files  chunk.files.forEach(function (filename) {  // compilation. Assets store all the current output resources  // Call the source() method of an output resource to get the contents of the output resource  let source = compilation.assets[filename].source();  });  });   // This is an asynchronous event, remember to call Webpack callback to notify Webpack that the event listening process is finished.  // If you forget to call callback, Webpack will stay stuck and never execute later.  callback();  })  } }  Copy the code

Listening for file changes

Webpack starts with the configured entry module and finds all dependencies in turn. When the entry module or its dependencies change, a new Compilation is triggered.

When developing plug-ins, you often need to know which file changes caused the new Compilation, using code like this:

// The watch-run event is triggered when the dependent file changes
compiler.hooks.watchRun.tap('MyPlugin', (watching, callback) => {
  // Get the list of files that have changed
  const changedFiles = watching.compiler.watchFileSystem.watcher.mtimes;
  // changedFiles format is key-value pair, the key is the changed file path.
 if(changedFiles[filePath] ! = =undefined) {  // The file corresponding to filePath has changed  }  callback(); }); Copy the code

By default, Webpack only monitors whether the entry and its dependent modules have changed, and in some cases the project may need to introduce new files, such as an HTML file. Because JavaScript files do not import HTML files, Webpack does not listen for changes to the HTML files and does not trigger a new Compilation when the HTML files are edited. To listen for changes to the HTML file, we need to add the HTML file to the dependency list, using the following code:

compiler.hooks.afterCompile.tap('MyPlugin', (compilation, callback) => {
  // Add the HTML file to the file dependency list so that Webpack can listen for the HTML module file and restart the compilation if the HTML template file changes
  compilation.fileDependencies.push(filePath);
  callback();
});
Copy the code

Modifying output Resources

In some cases, the plugin needs to modify, add, or delete the output resources. To do this, you need to listen for the EMIT event. The emit event is the last time to modify the output resources of the Webpack because all module transformations and code blocks corresponding files have been generated when the emit event occurs.

Compilation. assets is a key-value pair. The key is the name of the file to be exported, and the value is the corresponding content of the file.

Set up the compilation.assets code as follows:

// Set the output resource named fileName
  compilation.assets[fileName] = {
    // Return the file contents
    source: (a)= > {
      // fileContent can be either a string representing a text file or a Buffer representing a binary file
 return fileContent;  },  // Returns the file size  size: (a)= > {  return Buffer.byteLength(fileContent, 'utf8');  }  };  callback(); Copy the code

Determine which plug-ins webPack uses

// The ExtractTextPlugin is used in the current configuration.
// Compiler parameters are those passed in by Webpack in Apply
function hasExtractTextPlugin(compiler) {
  // List of all plug-ins used in the current configuration
  const plugins = compiler.options.plugins;
 // Go to plugins to find instances of the ExtractTextPlugin  return plugins.find(plugin= >plugin.__proto__.constructor === ExtractTextPlugin) ! =null; } Copy the code

The above 4 methods derived from the article: [Webpack learning – the Plugin] : http://wushaobin.top/2019/03/15/webpackPlugin/

Manages Warnings and Errors

As an experiment, what happens if you insert throw new Error(“Message”) inside apply, the terminal prints Unhandled Rejection Error: Message. Webpack then interrupts execution. To issue warnings or error messages to the user during compilation without affecting the execution of webpack, use compilation.warnings and compilation.errors.

compilation.warnings.push("warning");
compilation.errors.push("error");
Copy the code

The case demo code in this article is shown

Github.com/6fedcom/fe-…

How to debug the WebPack packaging process or plug-in code?

  1. Under the current WebPack project project folder, execute the command line:
node --inspect-brk ./node_modules/webpack/bin/webpack.js --inline --progress
Copy the code

The –inspect-brk argument starts Node in debug mode:

The terminal will output:

The Debugger listening on the ws: / / 127.0.0.1:9229/1018 c03f nikon d60-7473-4 - b62c - 949 a6404c81d For help, see: https://nodejs.org/en/docs/inspectorCopy the code
  1. Enter Chrome ://inspect/# Devices
Click to inspect
  1. Then click “Continue” in the Chrome debugger, and the breakpoint is stored in the debugger breakpoint we set in the plugin.
debugger