Webpack4/5 packaging principle analysis

Recently, I wrote some webpack plug-ins in the company to deal with the code above the work, so I studied the packaging principle of Webpack. This paper summarizes the basic implementation principle of Webpack. Because the company uses WebPack 4.35.0 internally, we have a simple understanding of Webpack based on this version, and then upgrade to Webpack 5 for analysis. This series is divided into three parts

Webpack5 packaging analysis
Webpack loader implementation
Webpack’s event flow and plugin principles

The principle of Webpack is analyzed from the following directions

Main uses of Webpack
Main content analysis of webpack after packaging
Implement a simple Webpack
Write something simpleloader
Write something simpleThe plug-in

Main uses of Webpack

When the program function is more complex, sometimes we will pull out the code module, convenient for us to carry out module management, for example, we have a code below:

// Directory structure
/ * the test ├ ─ ─ a. s ├ ─ ─ b.j s ├ ─ ─ index. The HTML └ ─ ─ index. The HTML * /
// a.js 
import b from  './b.js'
export default 'a' + b
// b.js
export default 'b'
// index.js! [](https://p9-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/ecdf1b37323740fda5ae40b8d2293bb7~tplv-k3u1fbpfcp-watermark.image)
import str from  './a.js'
console.log(str)

// index.html
<body>
<script type="module" src="index.js"></script>
</body>
Copy the code

For this simple Es6 module reference, the browser will send multiple requests, which limits the requests under the same domain name. If the function is complex, the request congestion will occur and affect the performance.

Based on this example, summarize our main uses for WebPack

Package references to modules to reduce file requests
When we use third-party modules that are not updated for a long time, we can use Webpack to separate modules and then cache them to reduce user requests
Transform and compress for different resources and code. For example, TypeScript converts to Javascript and Stylus converts to CSS
File optimization, you can compress and merge some resources
Code segmentation, extract the common code of multiple pages, extract the code that does not need to execute part of the first screen and load it asynchronously.

Main content analysis of webpack after packaging

# # test/SRC directory structure ├ ─ ─ a. s ├ ─ ─ base ├ ─ ─ b.j s └ ─ ─ index, js// a.js
let b = require('./base/b.js');
module.exports = 'a' + b;

// b.js
module.exports = 'b';

// index.js
let str = require('./a.js');
console.log(str)

// webpack.config.js
const path = require("path");
module.exports = {
  mode: 'development'.entry: "./src/index.js".output: {
    filename: "bundle.js".path: path.resolve(__dirname, "dist"),}};Copy the code

When we use NPX webpack, we generate build.js in the dist directory, we remove some comments and some things we don’t care about so far, and that’s about it

(function (modules) { // webpackBootstrap
  // The module cache
  var installedModules = {}
  // The require function
  function __webpack_require__(moduleId) {
    // Check if module is in cache
    if (installedModules[moduleId]) {
      return installedModules[moduleId].exports
    }
    // Create a new module (and put it into the cache)
    var module = installedModules[moduleId] = {
      i: moduleId,
      l: false.exports: {}}// Execute the module function
    modules[moduleId].call(module.exports, module.module.exports, __webpack_require__)
    // Return the exports of the module
    return module.exports
  }
  // Load entry module and return exports
  return __webpack_require__(__webpack_require__.s = './src/index.js') ({})'./src/a.js':
      (function (module.exports, __webpack_require__) {
        eval('let b = __webpack_require__("./src/base/b.js"); \r\nmodule.exports = \'a\' + b; \r\n\n\n')}),'./src/base/b.js':
      (function (module.exports) {
        eval('module.exports = \'b\'; \r\n\n\n')}),'./src/index.js':
      (function (module.exports, __webpack_require__) {
        eval('let str = __webpack_require__("./src/a.js"); \r\nconsole.log(str)\r\n\r\n\n\n')})})Copy the code

Break up the code snippet

Using an immediate execution function, we put some of the source code and pathname mapping of some of our content modules, and the file content is wrapped through a function, passing in three variables, respectivelymodule.exports.__webpack_require__
Used for source filesrequireCode snippets, all replaced with__webpack_require__

  './src/a.js': (function (module.exports, __webpack_require__) {
        eval('let b = __webpack_require__(/*! ./base/b.js */ "./src/base/b.js"); \r\nmodule.exports = \'a\' + b; ')}),Copy the code

Then throughmodulesThe argument is passed inside the function and defines a cache object to cache modules that have already been loadedinstalledModules
Implemented a__webpack_require__The function, which takes a module ID as an argument, returns an internally defined variablemodule.exports

  // The module cache
  var installedModules = {}
  // The require function
  function __webpack_require__(moduleId) {
    // Check if module is in cache
    if (installedModules[moduleId]) {
      return installedModules[moduleId].exports
    }
    // Create a new module (and put it into the cache)
    var module = installedModules[moduleId] = {
      i: moduleId,
      exports: {}}// Execute the module function
    modules[moduleId].call(module.exports, module.module.exports, __webpack_require__)
    // Return the exports of the module
    return module.exports
  }
  // Load entry module and return exports
  return __webpack_require__('./src/index.js')
Copy the code

To pass on our entry filesmoduleId, the call__webpack_require__Function and return a value
Essentially this code snippet says this
- Custom functions are executed__webpack_require__And passed in an entry file./src/index.js
- Dependencies on other modules found in import files ("./src/a.js"), recursively called__webpack_require__And passed in the dependent path"./src/a.js"(modulesThe key)
- in"./src/a.js"A dependency was found in the code"./src/base/b.js"Recursively called__webpack_require__
- And then it comes back__webpack_require__The return value of the function

Implement a simple Webpack

We created two directories, one for the implementation of WebPack (webpack-write) and one for the use of webpack in the front-end development process (webpack-dev), and based on these two directories we started implementing a simple Webpack

Connect two projects

`webpack-write`

├─ ├─ hk-├.js ├─ package.json/ / package. In json
  "bin": {
    "hcc-webpack": "./bin/hcc-webpack.js"
  },
  
// hcc-webpack.js
console.log('hcc-webpack')
Copy the code

We perform the NPM link to link the execution file to the local NPM library

$ npm link
// C:\Users\chucaihuang\AppData\Roaming\npm\node_modules\webpack-write -> D:\study\hcc-webpack\blog\webpack-write
Copy the code

`webpack-dev`

├── b.├ ─ ├─ b.├.js ├─ webpack.config.jsCopy the code

// webpack.config.js
const path = require("path");
module.exports = {
  entry: "./src/index.js".output: {
    filename: "bundle.js".path: path.resolve(__dirname, "dist"),}};Copy the code

We performed NPM Link Webpack-write above in the development environment

D:\study\hcc-webpack\blog\webpack-dev\node_modules\webpack-write -> 
C:\Users\chucaihuang\AppData\Roaming\npm\node_modules\webpack-write -> 
D:\study\hcc-webpack\blog\webpack-write
Copy the code

Then we execute NPX hCC-webpack to synchronize our updates in webpack-write in real time, so that we can easily adjust whether the webpack we wrote is good or not.

Overall requirements analysis and code implementation

Based on the packaged code analysis above, we need to identify several points

Id of the entry file for the modulemoduleId
Mapping the file path to the file contents
Generate a package file from the template

1. `hcc-webpack.js`file

We need to do a few things in hCC-webpack.js in the bin folder in webpack-write

Get user’swebpack.config.jsConfiguration options for
Webpack documentcompilerThe instance runs through the packaging process, so we need to create a new onecompilerThe instance
Start compiling

/ / bin/HCC - webpack. Js file

#!/usr/bin/env node
// console.log('hcc-webpack-3')

const path = require('path')
1. Obtain the configuration in webpack-dev
let config = require(path.resolve('webpack.config.js'))

// Create Compile instance
let Compiler = require('.. /lib/Compiler.js');
let compiler = new Compiler(config)

// Start packing
compiler.run()
Copy the code

2. Lib files`Compiler.js`file

We need to get the entry id and the content of the file to map the production-dependent file to the file content

// Get the contents of the entry file
class Compiler {
  constructor(config = {}) {
    // Store the configuration
    this.config = config
    // Determine the entry file
    this.entryId = config.entry
    // You need to obtain the file resource through the absolute path. You need to obtain the working path
    this.root = process.cwd()
  }
  getSource(modulePath) {
    let source = fs.readFileSync(modulePath, {
      encoding: 'utf8'
    })
    return source
  }
  buildModuleSource(modulePath) {
    let source = this.getSource(modulePath)
    // console.log(source)
  }
  run() {
    // console.log(' run ', this.config)
    // console.log(' run ', this.entryid)
    // 1. Obtain the contents of the import file and determine the dependencies of the file
    this.buildModuleSource(path.join(this.root, this.entryId))
  }
}
Copy the code

Handles the content of the source code

To obtain the source code for transformation, therequireChange to the implementation of their own__webpack_require__And generate the entry file dependencies we pass throughastSyntax tree for source code modification
1. Babylon primarily converts the source code to AST
2. @babel/traverse needs to traverse to the corresponding nodes
3. @babel/types replaces nodes traversed
4. @babel/generator needs to generate the result of the replacement

buildModuleSource(modulePath) { let source = this.getSource(modulePath) + let { sourceCode } = this.parse(source) console.log(sourceCode) } parse(source) { let ast = babylon.parse(source) traverse(ast, {CallExpression(p) {// function call // a() require() let {node} = p // get the call node if (node.callee.name)=== 'require') {Node.callee. name = '__webpack_require__' // replace require}}}) let sourceCode = Generator (ast). Code return {sourceCode} }Copy the code

We need to get toindex.jsAnd then recursivelyrequireThe replacement of
As we can see from the packaging, we areindex.jsUsed in therequire('./a.js'), but inmodulesThe key in the./src/a.jsSo we need to do some file path processing, based on the SRC directory to handle the dependency key

 buildModuleSource(modulePath) {
     let source = this.getSource(modulePath)
- let { sourceCode } = this.parse(source)
+ // Get the relative directory based on SRC
+ // Module ID relative path = modulePath - this.root ->./ SRC /index.js
+ let moduleName = './' + path.relative(this.root, modulePath)
+ let { sourceCode, dependencies } = this.parse(source, path.dirname(moduleName)) // ./src
    console.log(sourceCode)
}

parse(source) {
   let ast = babylon.parse(source)
+ let dependencies = [Traverse (ast, {CallExpression(p) {// function call // a() require() let {node} = p // get the call node if (node.callee.name=== 'require') {Node.callee. name = '__webpack_require__' // replace require+ let moduleName = node.arguments[0]. Value
ModuleName = './' + path.join(parentDir, moduleName)// Change./a.js to./ SRC /a.js based on SRC
          // console.log('1', moduleName, parentDir)
+ dependencies.push(moduleName)
+ node.arguments = [types.stringLiteral(moduleName)];
       }
     }
   })
   let sourceCode = generator(ast).code
   return {
     sourceCode,
+ dependencies}}Copy the code

Get module dependencies recursivelymodules

 buildModuleSource(modulePath) {
   let source = this.getSource(modulePath)
   let { sourceCode } = this.parse(source)
   let moduleName = './' + path.relative(this.root, modulePath)
   let { sourceCode, dependencies } = this.parse(source, path.dirname(moduleName)) // ./src
   
+ // match the relative path to the contents of the module
+ this.modules[moduleName] = sourceCode
+ // Get the module's dependencies recursively
+ if (dependencies && dependencies.length) {
+ dependencies.forEach(modulePath => {
+ console.log(modulePath)
+ this.buildModuleSource(path.join(this.root, modulePath))
+})
+}
+ console.log(this.modules)
}
Copy the code

conclusion

We have completed 2 of the above three points, now we just need to send the packaged file to the specified location according to the template

~~1. Id of the entry file of the modulemoduleId~~

~~2. Map the file path to the file content~~

Generate a package file from the template

Add the content of the templatemain.ejsEjs is used to produce packaged files

# /lib/main.ejs (function (modules) { var installedModules = {}; function __webpack_require__(moduleId) { if (installedModules[moduleId]) { return installedModules[moduleId].exports; } var module = installedModules[moduleId] = { i: moduleId, l: false, exports: {} }; modules[moduleId].call(module.exports, module, module.exports, __webpack_require__); module.l = true; return module.exports; } return __webpack_require__(__webpack_require__.s = '<%-entryId%>'); }) ({ <%for(let key in modules) {%> '<%-key%>': (function (module, exports, __webpack_require__) { eval(`<%-modules[key] %>`); }}), < % % >});Copy the code

Send a file

Run () {this.buildModulesource (path.join(this.root, this.entryid), true+ this.emitFile();} emitFile() { let dist = path.join(this.config.output.path, this.config.output.filename); let templateStr = this.getSource(path.join(__dirname, 'main.ejs')); let code = ejs.render(templateStr, { entryId: this.entryId, modules: this.modules }); // The path corresponds to this.assets = {}; this.assets[dist] = code; if (! fs.existsSync(this.config.output.path)){ fs.mkdirSync(this.config.output.path); } fs.writeFileSync(dist, this.assets[dist], { flag: 'a+' }); }Copy the code

After packaging, we found that the file name under modules did not correspond. Since the file separator under Linux and Windows is different, we need to modify entryId so that the entry file can correspond to the key under modules, so as to obtain the source code and recursively rely on it

+ buildModuleSource(modulePath, isEntry = false) {Let source = this.getSource(modulePath) // Get the relative directory based on SRC // Relative path of module ID = modulePath - this.root ->./ SRC /index.js let moduleName = './' + path.relative(this.root, modulePath)+ console.log(moduleName)
+ if (isEntry) {
+ this.entryId = moduleName;} let { sourceCode, dependencies } = this.parse(source, Path.dirname (moduleName)) //./ SRC // match the relative path to the contents of the module this.modules[moduleName] = sourceCode // Recursively obtain the module's dependency if (dependencies && dependencies.length) { dependencies.forEach(modulePath => { this.buildModuleSource(path.join(this.root,  modulePath)) }) } } run() { // 1. Gets the contents of the entry file and determines the dependencies of the file+ this.buildModuleSource(path.join(this.root, this.entryId), true)// Launch a file, the packaged file this.emitFile(); }Copy the code

The last

There are no changes to the packaging mechanism in webpack5, which removes module dependencies from the entry file after packaging

(() = > { // webpackBootstrap
  var __webpack_modules__ = ({
    './src/a.js':
        ((module, __unused_webpack_exports, __webpack_require__) = > {

          eval('let b = __webpack_require__(/*! ./base/b.js */ "./src/base/b.js"); \r\nmodule.exports = \'a\' + b; \r\n\n\n')}),'./src/base/b.js':
        ((module) = > {
          eval('module.exports = \'b\'; \r\n\n\n')})})var __webpack_module_cache__ = {}

  // The require function
  function __webpack_require__(moduleId) {
    // Check if module is in cache
    if (__webpack_module_cache__[moduleId]) {
      return __webpack_module_cache__[moduleId].exports
    }
    // Create a new module (and put it into the cache)
    var module = __webpack_module_cache__[moduleId] = {
      // no module.id needed
      // no module.loaded needed
      exports: {}}// Execute the module function
    __webpack_modules__[moduleId](module.module.exports, __webpack_require__)

    // Return the exports of the module
    return module.exports
  }
  (() = > {
    eval('let str = __webpack_require__(/*! ./a.js */ "./src/a.js"); \r\nconsole.log(str)\r\n\n\n')
  })()
})()

Copy the code

Due to the length of the article, we will explain some webpack loader and Webpack plugins mechanism and principles.

Webpack4/5 packaging principle analysis

Main uses of Webpack

Main content analysis of webpack after packaging

Break up the code snippet

Implement a simple Webpack

Connect two projects

webpack-write

webpack-dev

Overall requirements analysis and code implementation

1. hcc-webpack.jsfile

2. Lib filesCompiler.jsfile

conclusion

The last

Related Posts

Parent to child component learning in VUE

Save and Restore instances in canvas

Canvas small program to achieve picture compression

`webpack-write`

`webpack-dev`

1. `hcc-webpack.js`file

2. Lib files`Compiler.js`file