background

When I talk about build tools, I tend to use the word “automation” in front of them, because build tools are designed to free our hands from repetitive mechanical tasks.

What is front-end automation? Front-end engineers need to maintain very large and complex code, code maintenance, packaging, release and other processes become very complicated, and waste more and more time and energy, of course, with the increase of the process of human error increases the error rate.

Every team wants a tool that can help streamline processes, improve efficiency, and reduce error rates in development. There is an increasing discussion of automated deployment, and many large teams in the country have their own mature automated deployment tools.

Common build tools gulp, Webpack, Parcel, rollup, vite, fis, Grunt, etc

Over the years, Webpack has become the build tool of choice because:

  • Webpack can provide a one-stop solution for new projects that most teams use to keep abreast of The Times. These technologies are almost always “modular + new language + new framework”.

  • Webpack has a good ecology and maintenance team, which can provide good development experience and guarantee quality;

  • Webpack is used and validated by a large number of Web developers around the world to find tutorials and experience sharing at all levels.

The composition of webpack

The concept of bundles is very important in WebPack. Another important concept is the dependency diagram, which forms an important core of this wrapper.

Webpack has four core elements that are very relevant when we use webPack configuration:

  • Entry: Configuration entry file, that is, the entry to generate the dependency diagram
  • Output: indicates the output location of a file
  • Loader: compilation process of matching files
  • Plugins: Plug-in handling for the entire build packaging process (file compression, hot loading of the DEV environment)

The gateway determines where we need to package those files, the Loader takes over the compilation of the matching files here, and plugins are the operations for the entire build process, introducing plug-ins for what functionality is needed.

How does WebPack packaging work

Also called, how is WebPack modular

CommonJS is a synchronously loaded module commonly used for Nodes. Because the Node application runs on the server, the program can directly read the files of each module through the file system, which is characterized by fast response and will not block the operation of the program because of synchronization.

AMD is an asynchronous loading module, so it is commonly used in the front end. The front-end project runs in the browser, and each module has to load JS module files through HTTP requests. If it is affected by network and other factors, the browser will appear “suspended” (stuck), which affects the user experience.

ESModule aims to unify the front and back end modularity. Webpack converts ES6 modular code into CommonJS for browser compatibility.

Why webPack files can be used in browsers: Webpack will pack all JS modules into bundle.js (except asynchronously loaded modules, which will be covered later) and load them into memory instead of modules.

Webpack’s modular handling of CommonJS

For example:

Index.js file, importing test.js file

const test = require('./test');
console.log(test);
console.log('hello world');
Copy the code

Test. The js file

module.exports = {
  name: 'startdt',
  age: '5',
};
Copy the code

After we execute webpack, the packaging is complete and we can see the code inside bundle.js

// modules is an array of modules, each element of which is a function

Var installedModules = {}; function(modules) {// installedModules = {}; // Load a module from the array, Function __webpack_require__(moduleId) {// Function __webpack_require__(moduleId) {// If (installedModules[moduleId]) {return installedModules[moduleId].exports; if(installedModules[moduleId]. Var module = installedModules[moduleId] = {// Index of the module in the array I: ModuleId, // Whether the module is fully loaded l: false, // The exported value of the module, also called the module body content, is overridden by exports: {}}; // Get the function corresponding to the module whose index is moduleId from modules // call the function and pass in the parameters needed for the function. Modules [moduleId]. Call (Module. exports, module, module.exports, __webpack_require__); // Mark the module as loaded module.l = true; // exports returns the exported value of the module, i.e. the body of the module. } // Expose all modules __webpack_require__.m = modules; // Expose cached modules __webpack_require__.c = installedModules; . . __webpack_require__.p = ""; // Load entry module and return exports // Load entry module and return exports // Load entry module and return exports // // The module whose index is 0 is the index.js file. // __webpack_require__. S = 0; // __webpack_require__. }) /***** the gorgeous partition line on the top of the webpack initialization code, below is the module code we wrote *******/ / all modules are stored in an array, [/* module 0 corresponds to index.js */ (function(module, exports,) __webpack_require__) {// Import foo from the __webpack_require__ specification, Const test = __webpack_require__(1); console.log(test); console.log('hello world');}), Module. exports = {name: 'startdt', age: 0;} /* Module 1 corresponding to foo.js */ (function(module, exports) {// Exports = {name: 'startdt', age: '5'};})]);Copy the code

The above is an immediate function, which can be written simply:

Function (modules) {function __webpack_require__(index) {return [/* Return __webpack_require__(0); return __webpack_require__(0); })([/* Array of all modules */])Copy the code

Bundle.js runs directly in the browser because:

Webpack emulates module loading (similar to the Require statement in Node.js) via the _webpack_require_ function (which defines a loading function that can be executed in the browser) and mounts the defined module contents to module.exports.

The __webpack_require__ function is also optimized for module cache. The loaded module will not be executed a second time, and the execution result will be cached in memory. When a module is accessed a second time, it will directly read the cached value in memory.

The reason why separate module files are merged into a single bundle.js is that browsers cannot load module files locally as quickly as Node.js, but must load files that are not yet available through network requests. If there are many modules, the load time will be very long, so put all modules in an array and perform a network load.

For this part, I wrote a demo

The source address

Webpack’s handling of es6 Module modularity

For example,

Index.js file, importing test.js file

import test from './test';

console.log(test);
console.log('hello world');
Copy the code

Test. The js file

export default {
  name: 'startdt',
  age: '5',
};
Copy the code

Once packaged, the bundle.js code looks like this

(function(modules) { var installedModules = {}; function __webpack_require__(moduleId) { if(installedModules[moduleId]) { return installedModules[moduleId].exports; } var module = installedModules[moduleId] = { i: moduleId, l: false, exports: {} }; modules[moduleId].call(module.exports, module, module.exports, __webpack_require__); module.l = true; return module.exports; } __webpack_require__.m = modules; __webpack_require__.c = installedModules; __webpack_require__.d = function(exports, name, getter) { if(! __webpack_require__.o(exports, name)) { Object.defineProperty(exports, name, { configurable: false, enumerable: true, get: getter }); }}; __webpack_require__.n = function(module) { var getter = module && module.__esModule ? function getDefault() { return module['default']; } : function getModuleExports() { return module; }; __webpack_require__.d(getter, 'a', getter); return getter; }; __webpack_require__.o = function(object, property) { return Object.prototype.hasOwnProperty.call(object, property); }; __webpack_require__.p = ""; return __webpack_require__(__webpack_require__.s = 0); })([related module]);Copy the code

Packaged content is similar to the CommonJS modular approach

function(module, __webpack_exports__, __webpack_require__) { "use strict"; DefineProperty (__webpack_exports__, "__esModule", {value: __webpack_exports__); true }); var __WEBPACK_IMPORTED_MODULE_0__foo__ = __webpack_require__(1); console.log(__WEBPACK_IMPORTED_MODULE_0__foo__["a"]); console.log('hello world'); }, function(module, __webpack_exports__, __webpack_require__) { "use strict"; __webpack_exports__["a"] = ({ name: 'startdt', age: '5', }); }Copy the code

Different from CommonJS

First, the module.exports before the argument to the wrapper function becomes _webpack_exports_

Second, add the attribute __esModule to __webpack_exports__ where the ES6 module import syntax is used

The rest is similar to CommonJS

On-demand loading of Webpack files

Webpack packages all modules into the main file, so modules are loaded synchronously. But loading on demand (also known as lazy loading) is also one of the most commonly used optimization techniques in application development.

Loading on demand, generally speaking, means that the code is executed to the asynchronous module (the module content is in another JS file), the corresponding asynchronous module code is immediately loaded through the network request, and then the following process is continued.

The main js file

Window. The document. The getElementById (" BTN "). The addEventListener (' click ', function () {/ / when the button is clicked to load after the show. The js file, Import (/* webpackChunkName: "show" */ './show'). Then ((show) => {show('Webpack'); })});Copy the code

Show the js file

module.exports = function (content) {
  window.alert('Hello ' + content);
};
Copy the code

The most critical line in the code is import(/* webpackChunkName: “show”/”./show “). Webpack has built-in support for import() statements, which Webpack deals with when it encounters similar statements:

Create a Chunk with./show.js as the entry./show.js.

The file generated by Chunk is loaded only when the code executes to the import statement.

Import returns a Promise, which can be retrieved from the show.js export in the Promise’s then method when the file loads successfully.

Webpack has a require.ensure API syntax to mark asynchronous loading modules, and Webpack4 recommends using the new import() API (with the @babel/plugin-syntax-dynamic-import plug-in).

Since require. Ensure executes the following process via a callback function, and import() returns a promise, this means that async/await syntax can be used, making it possible to execute an asynchronous process as if writing synchronous code.

After the above content is packaged, two chunk files are generated, namely the main file execution entry file bundle.js and the asynchronous loading file 0.bundle.js.

// 0.bundle.js

// Asynchronous module // window["webpackJsonp"] is a bridge connecting multiple chunk files // window["webpackJsonp"]. Push = primary chunk file. webpackJsonpCallback = window (window [" webpackJsonp "] [" webpackJsonp] "| | []), push ([[0], / / logo chunkId asynchronous module, can determine whether an asynchronous code loaded success / / same as synchronization module, storing the {module path: } {"./ SRC /async.js": (function(module, __webpack_exports__, __webpack_require__) { __webpack_require__.r(__webpack_exports__); __webpack_exports__["default"] = (function () { return 'hello, aysnc module'; }); }) } ]);Copy the code

The source code of the asynchronous module is stored in the packaged file of the asynchronous module. In order to distinguish different asynchronous modules, the corresponding identifier of the asynchronous module is also saved: chunkId. The above code actively calls the window[” webpackJsonp “].push function, which is the key function that connects the asynchronous module to the main module and is defined in the main file. Actually window[” webpackJsonp “].push = webpackJsonpCallback

The implementation process of Webpack asynchronous loading module is basically the same as jSONP.

Now that we know the results of packaging, how does Webpack work

The core object of WebPack

Tapable: Controls the publishing and subscribing of hooks. Compiler and Compilation objects inherit from Tapable

Compiler 

Compiler inherits Tapable objects to broadcast and listen for Webpack events.

The Compiler object is the Compiler for WebPack, and only one Compiler object exists in the Webpack cycle.

The Compiler object creates an instance when WebPack is started. The Compiler instance contains the complete configuration of WebPack, including loaders and plugins.

Compilation

The Compilation inherits Tapable objects that broadcast and listen for Webpack events.

The Compilation instance represents only one Webpack build and the generation of compiled resources.

With the Watch option enabled in Webpack development mode, a new build is created each time a change in the entry file module is detected: Generate a new compilation resource and a new compilation object. This compilation object contains the currently compiled module resource, generated resources, changed files, and dependent states

The running flow of WebPack is a sequential process, from start to finish:

  1. Initialization parameters: read and merge parameters from configuration files and Shell statements to get the final parameters;

  2. Start compiling: initialize the Compiler object using the parameters obtained in the previous step (instantiate the Complier object), load all the configured plug-ins, execute the run method of the object to start compiling, and generate the Compilation object (instantiate the Compilation object).

  3. Determine the entry: according to the entry in the configuration, call AST engine (ACorn) to process the entry file, generate the abstract syntax tree AST, and build all the dependencies of the module according to AST;

  4. Module compilation: Starting from the entry file, call all configured Loader to translate the module, find out the module that the module depends on, and then recurse this step until all the entry dependent files have gone through this step;

  5. Complete module compilation: After using Loader to translate all modules in step 4, obtain the final content of each module after translation and the dependencies between them;

  6. Output resources: assemble chunks containing multiple modules one by one according to the dependency between the entry and modules, and then convert each Chunk into a separate file and add it to the output list. This step is the last chance to modify the output content.

  7. Output complete: After determining the output content, determine the output path and file name based on the configuration, and output the file content to the directory.

Preparation before compilation

Overview of this phase: Register project-configured plugins on the various hooks of compiler, and register the default webpack plugin ➡️ to register resolverFactory.hooks to provide parameter objects for the factory.createresolver method.

The event mechanism of Webpack is the event flow control based on tapable library. Various hooks are exposed in the whole compilation process, and Plugin registers to listen to some hooks. When this hook is triggered, the method bound in Plugin will be executed.

new NodeEnvironmentPlugin({
  infrastructureLogging: options.infrastructureLogging
}).apply(compiler);
Copy the code

Loader phase

Recursive compilation generates module instances

  • The Resolve phase returns an object that contains all the information for the current module

Overview of this phase: Use the enhanced resolve library to resolve the inline loader and its corresponding resource ➡️, and the loader of the project config. Then merge and sort all loaders ➡️ to obtain the parser and generator corresponding to module. Used for subsequent AST parsing and template generation ➡️ outputs a composite object containing all module information, including the current module context, loaders, absolute paths, dependencies, and so on, to provide a callback to afterResolve. This object will next be used to initialize the Module instance of the current file.

Such as

import Styles from style-loader! css-loader? modules! ./styles.cssCopy the code

Will be resolved into:

{
  "resource": "./styles.css",
  "elements": [
    {
      "loader": "style-loader"
    },
    {
      "loader": "css-loader",
      "options": "modules"
    }
  ]
}
Copy the code

Then each task of the parameter array is processed in parallel, and a Results list is returned after completion. The order of the list is the order of the parameter array, and has nothing to do with the execution order.

Results obtained:

Results: [[{"loader": "absolute path 1 of loader", "options": "loader parameter 1"}, {"loader":" absolute path 2 of loader", "options": "Loader parameter 2"}], {"resource":" module absolute path ", "resourceResolveData": "module basic information (enhanced-resolve execution result)"}]}Copy the code

Parse loaders in Config Module Rules and find corresponding loaders by recursive filtering:

{
  "result": [
    { "type": "type", "value": "javascript/auto" },
    { "type": "resolve", "value": {} },
    { "type": "use", "value": { "loader": "babel-loader" } }
  ]
}
Copy the code

Merge and sort loaders:

Next, inline Loader with prefix! ,!!!!! , -! And result with enforce parameter to determine whether to disable and sort the loader.

The resolve result (path information) is then sorted and merged in the callback, and the useLoadersPost, useLoadersPre and useLoaders obtained in the previous step are processed in parallel.

That is, the loaders configuration sequence is postLoader, inlineLoader, loader (Normal), and preLoader, and the execution sequence is reversed.

  • Execute loader phase, initialize module module, and translate loader in reverse order

Start the building Module process. New NormalModule(result) gets the initialized module ➡️ executes runLoaders processing source code during build, preemptively reads each loader and executes its pitch, The normal of each loader is executed in reverse order, resulting in a compiled string or Buffer.

The runLoaders method comes from loader-Runner and is used to execute various Loaders in a specified process, post-processing the module’s source code into JavaScript in either String or Buffer format (and possibly SourceMap).

  • The Parse phase collects dependencies

Call parser to convert the previous runLoaders compilation to ast using the Acorn library. The generated AST is divided into three parts: ImportDeclaration, Functional Declaration, and Variable Declaration. ➡️ iterates through the AST and triggers hook plugins to collect dependencies based on import/export and asynchronism. These dependencies are used to resolve recursive dependencies and template operations. ➡️ generates unique buildHash for each module based on relevant information.

  • Recursively handle the dependency phase (repeat the steps above)

Recursively resolves all dependent modules based on their interdependencies. Resolve ➡️ execute loader ➡️ parse ➡️ to collect and process the modules that the module depends on until all import dependent (direct or indirect) files have been processed by these steps. Finally, an entry module is returned.

Loader process induction

In this way, starting with the entry module, all modules are converted and compiled recursively based on their dependencies.

Return process.nexttick (callback) until all dependencies have been converted; Callback will be called before the next event loop tick,

Return an entry module:

{ "module": { //... / / synchronization module "dependencies" : [" HarmonyImportSideEffectDependency ", "HarmonyImportSpecifierDependency"], / / asynchronous module "blocks" : ["ImportDependenciesBlock"] } }Copy the code

Throw an error is not trigger compilation. Hooks: succeedEntry, until the end of this module to generate.

plugins

A webPack plug-in consists of the following aspects:

  • A non-anonymous JS function

  • Define the Apply method on its prototype object

  • Specifies the WebPack hook event that mounts itself

  • Manipulate specific data about what’s going on inside webpack

  • Method completes by invoking the webPack callback

The basic structure of the plug-in

A plug-in is an instantiated object with an Apply method in the prototype that is called by WebPack once when the plug-in is installed. The Apply method provides a reference to the currently active Webpack Compiler that allows access to the compiler’s callback

A simple Plugin case

function HelloWorldPlugin() { // }; HelloWorldPlugin.prototype.apply = function(compiler) { compiler.plugin('webpacksEventHook', function(compilation, callback) { console.log('Hello World! ') callback(); }); }; module.exports = HelloWorldPlugin; function HelloCompilationPlugin(options) {} HelloCompilationPlugin.prototype.apply = function(compiler) { // Setup callback for accessing a compilation: compiler.plugin("compilation", function(compilation) { // Now setup callbacks for accessing compilation steps: compilation.plugin("optimize", function() { console.log("Assets are being optimized."); }); }); }; module.exports = HelloCompilationPlugin;Copy the code

Use of plug-ins

const HelloWorldPlugin = require('hello-world');

const webpackConfig = {
  // ... config settings here ...
  plugins: [
    new HelloWorldPlugin({options: true})
  ]
};
Copy the code