You don't know webpack static file generation process

Author: Cui Jing

This article requires you to have a certain understanding of Webpack, if you are more interested, you can refer to our previous Webpack source code parsing series: Webpack series – overview.

Some concepts

The Compilation initializes the following variables:

this.mainTemplate = new MainTemplate(...)
this.chunkTemplate = new ChunkTemplate(...)
this.runtimeTemplate = new RuntimeTemplate
this.moduleTemplates = {
   javascript: new ModuleTemplate(this.runtimeTemplate, "javascript"),
   webassembly: new ModuleTemplate(this.runtimeTemplate, "webassembly")}this.hotUpdateChunkTemplate // Not yet
Copy the code

MainTemplate: Generates code to execute the main process, including the webpack startup code and so on. ChunkTemplate: The resulting code is loaded via JsonP.

Here’s an example: We have an entry file:

// main.js
import { Vue } from 'vue'
new Vue(...)
Copy the code

Such files are packaged to produce an app.js, a chunk-vendor.js.

The structure of app.js is as follows:

 (function(modules) { // webpackBootstrap
   // Start function for webpack
   // Webpack built-in methods{{})moduleId: (function(module.exports, __webpack_require__) {
      // We write js code in each module
   },
   // ...
 })
Copy the code

Chunk-vendors. Js is structured as follows:

(window["webpackJsonp"] = window["webpackJsonp"] || []).push([["chunk-vendors"] and {moduleId: (function(module.exports, __webpack_require__) {
     // ...
   },
   // ...
})
Copy the code

App.js contains the webpack bootstrap code, which is the overall framework of the mainTemplate.

App.js loads chunk-vendor.js via jonsP, the framework of the JS code is placed in the chunkTemplate.

The code generation process for each module in app.js and Chunk-vendors. Js is in ModuleTempalte.

Code generation main process

Chunk code is generated in seal phase. Start from the Compilation. CreateChunkAssets.

The main flow diagram is as follows

** Note 1: ** The Render function is identified in the JavascriptModulePlugin. This render function is later called in createChunkAssets.

The moduleTemplate is generated during initialization during Compilation
this.moduleTemplates = {
	javascript: new ModuleTemplate(this.runtimeTemplate, "javascript"),
	webassembly: new ModuleTemplate(this.runtimeTemplate, "webassembly")};Copy the code
RenderManifest is the function registered in JavascriptModulesPlugin because it is mainTemplate. And this determines the template used by the module inside for moduleTemplates. Javascript
compilation.mainTemplate.hooks.renderManifest.tap(
  "JavascriptModulesPlugin".(result, options) = > {
    / /...
    result.push({
      render: () = >
      compilation.mainTemplate.render(
        hash,
        chunk,
        moduleTemplates.javascript,
        dependencyTemplates
      ),
      / /...
    });
    returnresult; });Copy the code
Note 3: The module-source process is appended at the end

First determine whether the current structure uses mainTemplate or chunkTemplate. The two Tempaltes will have their own render flow. Let’s take mainTempalte as an example to see the process of Render.

The main structure code is generated in the Render main flow, which is the framework part of the code generated in our app.js demo. The code for each moulde is then generated. The process is accomplished by a function in the ModuleTemplate.

When module is generated, hook. Content, hook. Module, hook. Render, hook. Package are called. After each hook gets the result, it is passed to the next hook. Hook. Module After the hook is executed, the code for the Module is returned. Then in hook. Render, wrap this code into a function. If we had configured output.pathinfo=true (configuration description) in webpack.config.js, we would have added some path and tree-shaking related comments to the resulting code in hook.package. So we can read the code.

Once you have all the Module code, wrap it in arrays or objects.

Modify the code

Add additional content to one of the modules using the hooks generated by the above files

BannerPlugin is the addition of additional content at the beginning of the chunk file. What if we just want to add content to a module? Review the above code generation, flow chart, module code generation there are several key hook, such as hook. The content, hook. The module, the hook. The render. You can register functions in these hooks to make changes. A simple demo is shown below

const { ConcatSource } = require("webpack-sources");
class AddExternalPlugin {
  constructor(options) {
    // Plugin is initialized. This handles some parameter formatting and so on
    this.content = options.content // Get the content to add
  }
  apply(compiler) {
    const content = this.content
    compiler.hooks.compilation.tap('AddExternal'.compilation= > {
      compilation.moduleTemplates.javascript.hooks.render.tap('AddExternal'.(
        moduleSource,
        module ) = > {
          // The module argument is passed, which we can configure to execute the following logic in a module
          // ConcatSource means that the code we add to it will be concatenated at the end of the processing.
          const source = new ConcatSource()
          // Insert what we want to add at the beginning
          source.add(content)
          // Insert source code
          source.add(moduleSource)
          // return the new source code
          return source
      })
    })
  }
}
Copy the code

Wrap an extra layer of logic around the chunk execution code.

We have configured the umD mode, or the output.library parameter. After configuring these two things, the resulting code structure is different from the original app.js demo. Library =’someLibName’, for example, would look like this

var someLibName =
(function(modules){
// webpackBootstrap}) ([/ /... Each module
])
Copy the code

The implementation of this is to modify the code generated by mainTemplate in the hooks. RenderWithEntry section above.

If we want to wrap up some of our own logic in some cases. We can do it right here. I’ll give you a simple demo

const { ConcatSource } = require("webpack-sources");
class MyWrapPlugin {
  constructor(options){}apply(compiler) {
    const onRenderWithEntry = (source, chunk, hash) = > {
      const newSource = new ConcatSource()
      newSource.add(`var myLib =`)
      newSource.add(source)
      newSource.add(`\nconsole.log(myLib)`)
      return newSource
    }
    compiler.hooks.compilation.tap('MyWrapPlugin'.compilation= > {
      const { mainTemplate } = compilation
      mainTemplate.hooks.renderWithEntry.tap(
        "MyWrapPlugin",
        onRenderWithEntry
      )
      // If we support configuration of some variables, then we need to write our configuration information to the hash. Otherwise, the hash value will not change when you modify the configuration.
      // mainTemplate.hooks.hash.tap("SetVarMainTemplatePlugin", hash => {
      // hash.update()
      // });}}})module.exports = MyWrapPlugin
Copy the code

Webpack compiled results

var myLib =/ * * * * * * / (function(modules) {
/ /... Webpack bootstrap code
/ * * * * * * /  return __webpack_require__(__webpack_require__.s = 0);
/ * * * * * * / })
/ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * /
/ * * * * * * / ([
/* 0 */
/ * * * / (function(module.exports) {
// ...
/ * * * / })
/ * * * * * * / ])
console.log(myLib);
Copy the code

BannerPlugin

Similar to the built-in BannerPlugin. After the above chunk file is generated, that is, after the createChunkAssets execution is completed, the entire chunk file contents are modified. For example, bannerPlugin is in optimizaChunkAssets hook

In this hook you can get chunks of a parameter and then add additional content there.

The contents of chunkAssets are modified

After createChunkAssets is executed, the contents of the file can be retrieved from other hooks for modification.

AfterOptimizeChunkAssets hook, webpack generates sourcemap. If you make code changes after this, such as optimizeAssets or later emit hooks, you will find that sourcemap is incorrect. Like the following example

compiler.hooks.compilation.tap('AddExternal'.compilation= > {
  compilation.hooks.optimizeAssets.tap('AddExternal'.assets= > {
    let main = assets["main.js"]
    main = main.children.unshift('//test\n//test\n')})})Copy the code

Impact on hash. When the chunk generation is complete, the hash will be generated. Changes made to the code in the hook after the hash is generated, such as additions, do not affect the result of the hash. Take the example above of modifying chunk code. If our plugin is updated, the changes change, but the generated hash does not change with it. Therefore, the content of the plugin needs to be written into the hash in the hash-generating hooks.

The module – source generated

Each dependency generated in the Parser phase is handled in the module-source process, and the source code is converted from dependency.Template. Here we look at module-source in conjunction with the original parser. Take the following demo as an example:

// main.js
import { test } from './b.js'
function some() {
  test()
}
some()

// b.js
export function test() {
  console.log('b2')}Copy the code

AST converted from main.js parser:

Parser on the AST

if (this.hooks.program.call(ast, comments) === undefined) {
  this.detectMode(ast.body);
  this.prewalkStatements(ast.body);
  this.blockPrewalkStatements(ast.body);
  this.walkStatements(ast.body);
}
Copy the code

program

Testing have used the import/export, would increase HarmonyCompatibilityDependency HarmonyInitDependency (described later)
detectMode

Check if there is use Strict and use ASM at the beginning, to make sure that the use strict written at the beginning of our code is still at the beginning
prewalkStatements

Iterates through all variable definitions in the current scope. Import {test} from ‘./b.js’ is also in the current scope, so import is handled here (see javasjavascript -parser for details). For the import will be adding additional ConstDependency and HarmonyImportSideEffectDependency
blockPrewalk

Handles let/const (var only in prewalk), class name, export, and export default in the current scope
walkStatements

Start digging into each node for processing. Here will find all using the test code, and then add HarmonyImportSpecifierDependency

After the manager goes through this, he will join the demo above

HarmonyCompatibilityDependency

HarmonyInitDependency

ConstDependency

HarmonyImportSideEffectDependency

HarmonyImportSpecifierDependency

These fall into two categories:

ModuleDependency: there is a corresponding denpendencyFactory, in the process of processModuleDependencies will to deal with the dependency, get the corresponding module

HarmonyImportSideEffectDependency --> NormalModuleFactory

HarmonyImportSpecifierDependency --> NormalModuleFactory

Both refer to the same module(./b.js), so they will be de-duplicated. Then Webpack along the dependency, handles b.js… Until all moduleDependency is handled
Used to generate code only when files are generated

module.source

Get the source code first, then work on each dependency

HarmonyCompatibilityDependency

Insert __webpack_require__.r(__webpack_exports__) at the beginning; Identifies this as an esModule
HarmonyInitDependency

Iterating through all dependencies, responsible for generating code to introduce the ‘./b.js’ module for import {test} from ‘./b.js’

/* harmony import */ var _b_js__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(0);
ConstDependency

In the HarmonyInitDependency phase, the import statement was inserted, so the import {test} from ‘./b.js’ needs to be removed from the source code. ConstDependency replaces this with empty, delete
HarmonyImportSideEffectDependency

The action phase is in HarmonyInitDependency

HarmonyImportSpecifierDependency

Dependencies generated by test() in the code. It replaces test in the code
- Gets the variable name _b_js__WEBPACK_IMPORTED_MODULE_0__ corresponding to the ‘./b.js’ module
- Get the name of the property that test corresponds to in B.js. (Because of webpack compilation, to simplify the code, export test in B.js may be changed to export a = test.)
  
  Object(_b_js__WEBPACK_IMPORTED_MODULE_0__[/* test */ "a"])
  If it is called, it will follow a logic.
```
if (isCall) { if (callContext === false && asiSafe) { return `(0,${access})`; } else if (callContext === false) { return `Object(${access})`; }}Copy the code
```
- Then replace test in the code

After all the dependency:

Once we know this process, if we need to make some simple changes to the source code, we can use the various hooks of the Parser stage to do so. One advantage of making changes here is that you don’t have to worry about corrupting sourcemap and affecting hash generation.

A demo of the code inserted into the Parser

For example, when we use a plug-in, we need to write this

import MainFunction from './a.js'
import { test } from './b.js'
MainFunction.use(test)
Copy the code

In practice, the Webpack plug-in is used to automatically insert the test when it detects an introduction

import MainFunction from './a.js'
MainFunction.use(test)
Copy the code

The key that the above mentioned HarmonyImportSideEffectDependency, HarmonyImportSpecifierDependency and ConstDependency

The following code

const path = require('path')
const ConstDependency = require("webpack/lib/dependencies/ConstDependency");
const HarmonyImportSideEffectDependency = require("webpack/lib/dependencies/HarmonyImportSideEffectDependency")
const HarmonyImportSpecifierDependency = require("webpack/lib/dependencies/HarmonyImportSpecifierDependency")
const NullFactory = require("webpack/lib/NullFactory");

// The path to introduce a.js. This path is followed by Webpack's resolve
const externalJSPath = `${path.join(__dirname, './a.js')}`

class ProvidePlugin {
	constructor(){}apply(compiler) {
		compiler.hooks.compilation.tap(
			"InjectPlugin".(compilation, { normalModuleFactory }) = > {
				const handler = (parser, parserOptions) = > {
          // When parser processes import statements
          parser.hooks.import.tap('InjectPlugin'.(statement, source) = > {
            parser.state.lastHarmonyImportOrder = (parser.state.lastHarmonyImportOrder || 0) + 1;
            // Create a dependency of './a.js'
            const sideEffectDep = new HarmonyImportSideEffectDependency(
              externalJSPath,
              parser.state.module,
              parser.state.lastHarmonyImportOrder,
              parser.state.harmonyParserScope
            );
            // Set a location for dependency. This is set to the same location as import {test} from './b.js', where the modification will be inserted when the code inserts.
            sideEffectDep.loc = {
              start: statement.start,
              end: statement.end
            }
            // Set renames to indicate that mainFunction is imported from outside
            parser.scope.renames.set('mainFunction'."imported var");
            // Add this dependency to module dependencies
            parser.state.module.addDependency(sideEffectDep);
            
            / / -- -- -- -- -- -- -- -- -- -- -- -- - deal with insert mainFunction. Use (test) -- -- -- -- -- -- -- -- -- -- -- --
            if(! parser.state.harmonySpecifier) { parser.state.harmonySpecifier =new Map()
            }
            parser.state.harmonySpecifier.set('mainFunction', {
              source: externalJSPath,
              id: 'default'.sourceOrder: parser.state.lastHarmonyImportOrder
            })
            // For mainFunction in mainfunction. use
            const mainFunction = new HarmonyImportSpecifierDependency(
              externalJSPath,
              parser.state.module,
              -1,
              parser.state.harmonyParserScope,
              'default'.'mainFunction'[-1, -1].// Insert it at the beginning of the code
              false
            )
            parser.state.module.addDependency(mainFunction)
            
            // Insert the code snippet. Use (
            const constDep1 = new ConstDependency(
              '.use(',
              -1.true
            )
            parser.state.module.addDependency(constDep1)
            
            // Insert code snippet test
            const useArgument = new HarmonyImportSpecifierDependency(
              source,
              parser.state.module,
              -1,
              parser.state.harmonyParserScope,
              'test'.'test'[-1, -1].false
            )
            parser.state.module.addDependency(useArgument)
            
            // Insert code snippet)
            const constDep2 = new ConstDependency(
              ')\n',
              -1.true
            )
            parser.state.module.addDependency(constDep2)
          });
        }
				normalModuleFactory.hooks.parser
					.for("javascript/auto")
					.tap("ProvidePlugin", handler);
				normalModuleFactory.hooks.parser
					.for("javascript/dynamic")
					.tap("ProvidePlugin", handler); }); }}module.exports = ProvidePlugin;

Copy the code

The generated code is as follows

/ * 1 * /
/ * * * / (function(module, __webpack_exports__, __webpack_require__) {

"use strict";
const mainFunction = function () {
  console.log('mainFunction')
}

mainFunction.use = function(name) {
  console.log('load something')}/* harmony default export */ __webpack_exports__["a"] = (mainFunction);

/ * * * / }),
/ * 2 * /
/ * * * / (function(module, __webpack_exports__, __webpack_require__) {

"use strict";
__webpack_require__.r(__webpack_exports__);
/* harmony import */ var _Users_didi_Documents_learn_webpack_4_demo_banner_demo_a_js__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(1);
/* harmony import */ var _b_js__WEBPACK_IMPORTED_MODULE_1__ = __webpack_require__(0);
_Users_didi_Documents_learn_webpack_4_demo_banner_demo_a_js__WEBPACK_IMPORTED_MODULE_0__[/* default */ "a"].use(_b_js__WEBPACK_IMPORTED_MODULE_1__[/* test */ "a"])

Object(_b_js__WEBPACK_IMPORTED_MODULE_1__[/* test */ "a") ()/ * * * / })
Copy the code

DefinePlugin

DefinePlugin introduction

You can use this plugin to replace constants at compile time, for example:

Commonly used in js code according toprocess.env.NODE_ENVTo distinguish between dev and Production environments. So as to realize different branch logic in different environment.

new DefinePlugin({
  'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV)
})
Copy the code

You can configure the API URL

new DefinePlugin({
  API_DOMAIN: process.env.NODE_ENV === 'dev' ? '" / / 10.96.95.200 "' : '"//api.didi.cn"'
})
Copy the code

Implementation of dev and Production API request domain name switch.

A brief introduction to some principles: a simple example

new DefinePlugin({
  'TEST': "'test'"
})
Copy the code

The const a = TEST is used in the code. Parser iterates to the right of the = sign, triggering the expression parsing hook

/ / the key is the TEST
parser.hooks.expression.for(key).tap("DefinePlugin".expr= > {
  const strCode = toCode(code, parser); // Result is set to 'test'
  if (/__webpack_require__/.test(strCode)) {
    // If __webpack_require__ is used, the generated ConstantDependency requireWebpackRequire=true
    Function (module, exports){} __webpack_require__, function(module, exports){} As a function module, exports, __webpack_require__) {}
    return ParserHelpers.toConstantDependencyWithWebpackRequire(
      parser,
      strCode
    )(expr);
  } else {
    / / ParserHelpers toConstantDependency generates a ConstDependency, and added to the current module
    / / ConstDependency expression = "" test", "where is the position that corresponds to the test in our code
    returnParserHelpers.toConstantDependency( parser, strCode )(expr); }});Copy the code

As mentioned earlier, ConstDependency replaces the source code counterpart. So do the following in the later code generation phase

ConstDependency.Template = class ConstDependencyTemplate {
	apply(dep, source) {
    // if range is a number, insert; If it is an interval, it is a replacement
		if (typeof dep.range === "number") {
			source.insert(dep.range, dep.expression);
			return;
		}
		Dep. Expression = "test"
		source.replace(dep.range[0], dep.range[1] - 1, dep.expression); }};Copy the code

So this is the replacement of TEST in the source code.

conclusion

I believe that through the above detailed process analysis and some corresponding demo practice, for Webpack is how to generate static files in the whole process have been understood. Hopefully, in the future, you’ll be able to do it yourself if you encounter a similar scenario where the existing eco-plugin doesn’t meet your needs.

One of the biggest motivations for us to dig into this detail was our need to use many of these static file generation applications in our open source applets framework MPX. If you are interested, you are welcome to know, to use, to build.

In addition, the team number of Didi front-end technology team has also been online, and we have synchronized certain recruitment information. We will continue to add more positions, so interested students can chat with us.

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

You don’t know webpack static file generation process

Some concepts

Code generation main process

Modify the code

The contents of chunkAssets are modified

The module – source generated

module.source

conclusion

You don’t know webpack static file generation process

Some concepts

Code generation main process

Modify the code

The contents of chunkAssets are modified

The module – source generated

module.source

conclusion

Related Posts

DD weekly front seven questions detailed explanation – fifth issue

Isn’t it amazing to use GitHub Issue instead of Dodo?

Build a VUe-CLI project and initialization project from 0.