preface

The previous two sections have introduced two core concepts in WebPack: Plugin and Loader.

Today, we will connect plugin and Loader in series and write a program architecture based on plug-in system from 0 to complete a mini imitation version of Webpack.

Target requirements:

  • implementationjsModule packaging of
  • Set uppluginSystem, allowing access developers to customizeplugin
  • Set uploaderSystem, allowing access developers to customizeloader

Mini-webpack with plugin and loader enabled, developers can write their own plug-ins to directly intervene in the packaging process. In addition to basic JS compilation, we can also add processing capabilities for images and CSS.

According to the execution process, we first develop the plug-in system of Mini-Webpack, so that it has the ability to access plug-ins.

After the plug-in system is built, HTML templates, CSS and images can be introduced into the project through custom plugin and Loader. Finally, all the dependent files are compiled and packaged to produce the dist directory output (the source code is pasted at the end).

Main program architecture

Create a configuration file webpack.config.js in the project root directory (code below).

The entry address is the index.js file in the SRC /js directory, and the generated JS code is packaged and placed in dist/bundle.js.

Webpack.config. js also adds the compilation of CSS and images, loading the relevant loader for processing respectively.

In the plugins section, a plugin is configured to handle HTML. Finally, mini-Webpack generates an index.html in the dist directory and inserts the compiled js path into the index.html.

// webpack.config.js
module.exports = {
  entry:path.join(__dirname,"./src/js/index.js"),
  output:{
     path:path.join(__dirname,"/dist"),
     filename:"bundle.js"
  },
  module:{
    rules:[{
       test:/\.js/,
       use:[babelLoader]
    },{
      test:/\.css/,
      use:[htmlLoader]
    },
    {
      test:/\.(jpg|png|gif)/,
      use:[{
         loader:fileLoader,
         options:{
            outputPath:"./image"
         }
      }]
    }
   ]
  },
  plugins:[
     new HtmlWebpackPlugin({
        template:path.join(__dirname,"./src/index.html"),
        filename:"index.html"
     })
  ]
}
Copy the code

After the configuration file is written, the entry file for creating mini-webpack under the project is mini-webpack.js(code is as follows).

The code for the entry file is simple: pass webpack.config.js into the Compiler class to generate the Compiler object, which executes the Run method to start the compilation process.

// mini-webpack.js const options = require(".. /webpack.config"); const Compiler = require("./Compiler"); const compiler = new Compiler(options); compiler.run(); // Start webpack compilationCopy the code

The compiler implementation

Compiler is the backbone engine of Mini-Webpack, which internally uses Tapable to define four key life cycle functions as follows (tapable is described in the previous section without further elaboration).

  • initialize: Triggered when initialization is complete
  • compile: Triggered before compilation
  • emit: Triggered before the package file is generated
  • done: Triggered when the build is complete

The Compiler. Js constructor defines four lifecycle hooks, followed by a call to the bindHook function to begin binding the plugin.

In the webpack.config.js configuration file written by the developer, there is a column of plugins that defines plugins. The following bindHook function takes out each of the plugins from the configuration file and connects them to the Mini-Webpack system for use.

// Compiler.js const { SyncHook } = require('tapable'); class Compiler { constructor(options){ this.options= options; this.hooks = { initialize:new SyncHook(["arg"]), compile:new SyncHook(["arg"]), emit:new SyncHook(["arg"]), done:new SyncHook(["arg"]) } this.outPutDir = this.options.output.path; // Output directory this.bindhook (); } /** * bindHook events */ bindHook(){const {plugins} = this.options; Plugins.foreach ((plugin)=>{plugins.apply.call (plugin,this); }) this.hooks.initialize.call(this.options); // Trigger the initialize hook. }Copy the code

Once the Compiler instance is created, calling the run method initiates the code compilation task (the code is shown below).

The run method doesn’t have much code, but it covers almost the entire build process. Analysis in order of execution is as follows:

  • The run method first triggers the plug-in defined under the Compile hook at a time node before the code is compiled

  • The Complilation object is then created, which does the real work by calling the buildModule function to compile the code

  • After the buildModule function executes, it wraps all the code to be packaged into a data object and returns it to this.assets

  • The this.emit function uses the file API provided by Nodejs to generate the corresponding file directory from this.assets.

class Compiler { ... async run(){ this.hooks.compile.call(this.options); This.plilation = new Complilation(this.options,this); This. / / generate build object assets = await this.com plilation. BuildModule (); // Start compiling this.hooks. Emit. Call (this.assets); This.emit () before printing static resources to the file directory; // Generate a package file this.links.done. Call (); // Build complete}}Copy the code

As you can see from the flow of the run method, the key part is that complilation.buildModule() returns this.assets, from which the final directory of files is generated.

What does the data structure of this.assets look like?

this.assets = { 'bundle.js': 'const name = "hello world"; console.log(name); '}Copy the code

The this.assets data structure is easy to understand. The key corresponds to the file name and the value corresponds to the code content of the file.

As a result, this structure creates a bundle.js file in the dist directory and fills it with the right side of the code.

If the generated this.assets data structure is as follows, the dist directory will generate two files, bundle.js and index. HTML, and the script path of bundle.js will be inserted into the index. HTML.

this.assets = { 'bundle.js': 'const name = "hello world"; console.log(name); ', 'index.html': '<! DOCTYPE html><html lang="en"><head>\n' + ' <meta charset="UTF-8">\n' + ' <meta http-equiv="X-UA-Compatible" content="IE=edge">\n' + ' <meta name="viewport" content="width=device-width, Initial - scale = 1.0 "> \ n '+' < title > Document < / title > \ n '+' < / head > \ n '+' < body > \ n '+' < div id =" root "> \ n '+' hello world\n' + ' </div>\n' + '\n' + '<script src="./bundle.js"></script></body></html>', }Copy the code

To sum up, as long as we can dynamically control the data structure of this.assets, we can decide the content of the generated file. Js, CSS, HTML, and images (which can be serialized into binary data) can all be packaged.

Now we write a plugin that packs mini-webpack to generate an index.html file and inject the JS path into it.

The configuration file introduces the plug-in HtmlWebpackPlugin and builds an instance object using the new keyword.

// webpack.config.js module.exports = { ... plugins:[ new HtmlWebpackPlugin({ template:path.join(__dirname,"./src/index.html"), // HTML page template filename:"index.html" // filename})]}Copy the code

HtmlWebpackPlugin is an exported class.

There is a core method apply in the class, which is implemented by calling the apply method of the plug-in when Compiler is connected to the plug-in.

The Apply method listens internally for the COMPILER’s EMIT hook, and as described above the EMIT event fires before generating the package file.

At the corresponding time of the EMIT event,this.assets will have been completed by complilation. BuildModule () and will be returned.

Now that this. Assets is available in the Apply method, the template string of THE HTML is obtained based on the configuration passed in. Using the API provided by jsDOM, the script path is inserted into the HTML and assigned to compiler.assets.

Through a plugin,this. Assets adds an index. HTML property, and the final package generates the corresponding HTML file.

// html-webpack-plugin.js
const path = require("path");
const fs = require("fs");
const { JSDOM } = require("jsdom");

class HtmlWebpackPlugin {

  constructor(options){
     this.template = options.template;
     this.filename = options.filename || "index.html";
  }

  
  apply(compiler){

    compiler.hooks.emit.tap("insertHtml",()=>{

      const { filename } = compiler.options.output;
       
      const js_url = `./${filename}`;

      const code = fs.readFileSync(this.template).toString();

      const dom = new JSDOM(code);

      const body = dom.window.document.querySelector("body");

      body.innerHTML = body.innerHTML + `<script src="${js_url}"></script>`;

      compiler.assets[this.filename] = dom.serialize(); 

    })

  }

}

module.exports = HtmlWebpackPlugin;

Copy the code

Complilation implementation

The Complilation instance does the real work, calling the buildModule method to compile and build the code, and finally returning this.assets.

After complilation. BuildModule is executed, it first gets the project’s entry file address via the configuration item webpack.config.js.

The first thing that complilation needs to do is to get a loader that processes the.js file according to the rules configuration in webpack.config.js.

// index.js const { add } = require("./other"); require(".. /css/global.css"); const img_url = require(".. /img/1.png"); The console. The log (add (1, 1));Copy the code

In general, the most common requirement for JS file processing is to convert es6 syntax to ES5. Complilation can convert the above ES6 syntax to ES5 by calling loader.

Now look at how loader converts es6 syntax to ES5 (code below).

// webpack.config.js module.exports = { module:{ rules:[{ test:/\.js/, Use: [babelLoader] / / all js file to be babelLoader processing again}}} -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- / / Babelloader.js const parser = require("@babel/parser"); const {transformFromAstSync} = require("@babel/core") Exports = function(content){const ast = parser.parse(content,{sourceType:"module"}) const {code} =  transformFromAstSync(ast,null,{ presets: ["@babel/preset-env"] }) return code; }Copy the code

BabelLoader is an exported function, and Content is the source code passed in.

This function first converts the source code into an AST syntax tree using the babel-related tools, and then sets the Presets to return the AST as ES5 code.

After processing by babelLoader, the source of the entry file index.js looks like this.

// index.js var _require = require("./other.js") require(".. /css/global.css"); var img_url = require(".. /img/1.png"); The console. The log (_require. Add (1, 1));Copy the code

Upon observation, all const keywords become VAR, and the code is indeed converted to ES5.

But throwing this code directly into the browser will definitely cause an error, because the browser doesn’t know what require is, and it won’t help you intelligently introduce CSS and images.

The second step of the complilation instance is to scan the require syntax in index.js above. If files with the. Js suffix are found, ignore them first.

But if it finds a file with another suffix, it looks for the appropriate loader in webpack.config.js to process it. For example, after the above code is processed using a loader that handles CSS and images, the code is converted to look like this.

require(“.. / CSS /global.css”) is replaced with a function that, when the browser finally executes it, adds the global.css style to the header of the page document.

Picture the require (“.. /img/1.png”) is replaced with an image path, and the image name is converted to a hash value.

var _require = require("./other.js") (function(){ var tag = document.createElement("STYLE"); // Create the style tag tag.innerhtml = "body {color: red; } "; / / global. CSS style contents var head = document. GetElementsByTagName (" head ") [0]; Head. AppendChild (tag); }) (); var img_url = "image/0e3c014db8b376a43cbf7cca8291a036a357b2937f4b6dfb03864d0ea2c9bf11.png"; The console. The log (_require. Add (1, 1));Copy the code

The CSS and image processing can be implemented by loader, the next look at loader how to handle.

HtmlLoader deals with CSS files and returns a string function.

The style function creates a style tag, inserts the contents of the CSS file, and then places it under the document’s head tag.

// htmlLoader.js module.exports = function(content){ return `(function(){ var tag = document.createElement("STYLE"); // Create the style tag tag.innerhtml = ${json.stringify (content)}; var head = document.getElementsByTagName("Head")[0]; Head. AppendChild (tag); `}) (); }Copy the code

FileLoader can be used to work with images or fonts, and it generates a hash name based on the file content. The compiler emits the hook binding to an event function.

The EMIT hook function is fired when mini-WebPack is ready to package the build file.

It adds the image under this.assets and returns the relative path of the image. Finally, the dist directory will also generate the corresponding image.

// fileLoader.js const path = require("path"); const sha256 = require("sha256"); module.exports = function(){ const ext = path.extname(this.filename); Const hash = sha256(this.raw); . / / this row is the binary data file, generate image hash name const outputPath = this. Query. The outputPath | | ". / ". Const img_relative = path.join(outputPath, '${hash}${ext}'); // User configured directory const img_relative = path.join(outputPath, '${hash}${ext}'); / / after joining together the packaging pictures of relative path enclosing context, hooks, emit. Tap (" imgResolve ", () = > {enclosing context. The assets [img_relative] = this. Raw; }) return json.stringify (img_relate.replace (/\\/g,"/")); // The relative path returns directly}Copy the code

How does a complilation instance perform code conversion for CSS and images by scanning all the require syntax in the entry file index.js?

The tools provided by Babel can be easily done by first converting the code into an AST syntax tree, and then finding all require syntax in the code according to the data characteristics of the syntax tree. If CSS files or images are found, the subsequent processing can be carried out.

const traverse = require("@babel/traverse").default; / /... traverse(ast, { CallExpression(path) { if (path.node.callee.type === "Identifier" && path.node.callee.name === "require"){ ... }},});Copy the code

After the complilation instance performs the above tasks, the code for the entry file is eventually transformed to look like this.

The following code does all the work with JS, CSS, and images, but it still won’t work when thrown into a browser. Browsers don’t know require, and it can’t intelligently introduce other JS.

var _require = require("./other.js"); (function(){ var tag = document.createElement("STYLE"); // Create the style tag tag.innerhtml = "body {color: red; } "; / / global. CSS style contents var head = document. GetElementsByTagName (" head ") [0]; Head. AppendChild (tag); }) (); var img_url = "image/0e3c014db8b376a43cbf7cca8291a036a357b2937f4b6dfb03864d0ea2c9bf11.png"; The console. The log (_require. Add (1, 1));Copy the code

Rely on the analysis of

The third step in the complilation instance is to start handling the situation where Require introduces JS, assuming the project source code is as follows.

There are three files in the project :index.js, other.js, and three.js. Index.js introduces the add method exported by other.js. Other. js introduces the multiple method of three.js.

Now how do you combine these three pieces of code and run them in a browser?

Var _require = require("./other.js"); (function(){ var tag = document.createElement("STYLE"); // Create the style tag tag.innerhtml = "body {color: red; } "; / / global. CSS style contents var head = document. GetElementsByTagName (" head ") [0]; Head. AppendChild (tag); }) (); var img_url = "image/0e3c014db8b376a43cbf7cca8291a036a357b2937f4b6dfb03864d0ea2c9bf11.png"; The console. The log (_require. Add (1, 1)); ---------------------- // other.js const { multiple } = require("./three"); exports.add = (a,b)=>{ return multiple(a+b); } --------------------- // three.js exports.multiple = (total)=>{ return total * 10; }Copy the code

If the above three pieces of code were converted to the following form, wouldn’t the code merge be complete?

// bundle.js var entry = "./src/js/index.js"; / / entry address var deps = {/ / dependence graph. / SRC/js/index. "js" : "var _require = the require (\". / other. Js \ "); (function(){var tag = document.createElement(\"STYLE\"); . ", "./other.js":"var _require = require(\"./three.js\"); exports.add = (a,b)=>{ return _require.multiple(a+b); }", "./three.js":"exports.multiple = (total)=>{ return total * 10; }" } (function(entry,modules){ function require(pathname){ var module = { exports:{} } ; (function(require,module,exports){ const code = modules[pathname]; try{ eval(code); }catch(error){ console.log(error); } })(require,module,module.exports); return module.exports; } require(entry); })(entry,deps)Copy the code

Deps is a data object. Key corresponds to the file name and value corresponds to the code processed by loader.

Once the code is thrown into the browser, the require function first loads the entry file./ SRC /js/index.js, relies on the graph to return the source code based on the address of the entry file, and then executes the code using eval.

If require(“./other.js”) is encountered during eval(code), the require function is recursively invoked until all dependent files have been executed.

Therefore, as long as the code of each file in the project is finally compiled into bundle.js, the whole compilation and construction task is completed. The most difficult part of this process is the generation of dependency graph. How to convert the code in each file into the above data structure of dependency graph DEPS?

Again, we need to use the Babel tool to achieve this goal (code below). AnalyseLib is an analysis dependent function, and initially analyseLib takes the source code and filename of the entry file and starts executing.

The source code is then converted into the AST syntax tree, and all js files dependent on the entry file are found and stored in the DEPS array by traversing the syntax tree.

As the process continues, the DEPS array iterates through the loop, starting a recursive call to the analyseLib function. After all the recursive calls are complete, all dependent files and code are assigned to this.modules.

With this.modules, the code for synthesizingbundle.js is very simple based on the format above.

The last step for the complilation instance is to add a bundle.js property and its value to this.assets and return its build result this.assets to compiler. This is the end of the mini-WebPack construction task.

const traverse = require("@babel/traverse").default; class Complilation { ... modules = {}; AnalyseLib (code,filename){const ext = path.extname(filename); // Get the filename suffix if(ext! == ".js"){// Only js files need dependency analysis return; } const ast = parser.parse(code,{// generate ast syntax tree sourceType:"module"}) const deps = []; traverse(ast, {// Traverse the AST syntax tree looking for the require statement CallExpression(PATH) {if (path.node.callee.type === "Identifier" && path.node.callee.name === "require") { deps.push(path.node.arguments[0].value); }}}); this.modules[filename] = code; for(let i = 0; i < deps.length ; i++){ const dep = deps[i]; this.analyseLib(this.getCode(dep),dep); // get the dependent file code and recursively call analyseLib function}}... }Copy the code

The source code

The source code