The full text is 6000 words, we talk about packing loop, welcome thumb up follow forward.

To recap, as discussed in the previous article “A little hard on the Webpack: Depth parsing of the Dependency Graph,” Webpack resolved this after the make phase:

  • modulecontent
  • module 与 moduleDependency diagrams between

After entering the generation (SEAL) stage, Webpack first calculates the Chunk Graph based on module dependencies, module features, entry configuration, etc., to determine the quantity and content of the final product. This part of the principle is described in the previous article “A little difficult knowledge: The Webpack Chunk Subcontracting Rules are also described in more detail.

After this article continues to talk about the Chunk Graph, the module translation process begins to merge into the module packaging process. The general process is as follows:

To facilitate understanding, I have horizontally divided the packaging process into three stages:

  • The entrance: Refers from Webpack startup to invocationcompilation.codeGenerationAll of the preceding operations
  • Module translation: traversalmodulesArray, complete the translation operation of all modules, and store the results tocompilation.codeGenerationResultsobject
  • Module merge and package: In a specific context framework, business modules and Runtime modules are combined, packaged into bundles, and calledcompilation.emitAssetThe output product

The business module here refers to the project code written by the developer; The Runtime module is the runtime code that is dynamically injected to support various features after Webpack analyzes the business module. It was explained in detail in the previous article Webpack Principles Series 6: Understanding the Webpack Runtime thoroughly, so I won’t repeat it here.

As you can see, Webpack first translates modules one by one into module products — module translation — and then splices module products into bundles — module merging and packaging. We will discuss the principle of these two processes separately in the following logic.

First, module translation principle

1.1 introduction

Let’s review the Webpack artifact:

The above example consists of two business files, index.js/name.js. The corresponding Webpack configuration is shown in the lower left corner of the figure above. The Webpack build product is shown in the main.js file on the right, which contains three pieces of content, respectively from top to bottom:

  • name.jsThe translation product corresponding to the module, the functional form
  • Runtime code injected by Webpack on demand
  • index.jsThe translation product corresponding to the module, the IIFE(Immediate Execution Function) form

The role of the runtime code and the generation logic was explained in detail in the previous article, Webpack Principles Series 6: Understanding the Webpack Runtime thoroughly. The other two are the products of name.js and index.js respectively. You can see that the product and the source code have the same semantics and functions, but the form of expression has changed greatly, such as the content of index.js before and after compilation:

On the right of the figure above is the Webpack build artifact, with the following changes from the source on the left:

  • The entire module is wrapped in IIFE (Instant Execute Function)
  • add__webpack_require__.r(__webpack_exports__);Statement for adapting the ESM specification
  • In the sourceimportStatements are translated to__webpack_require__A function call
  • The source codeconsoleStatementsnameThe variable is translated to_name__WEBPACK_IMPORTED_MODULE_0__.default
  • Add comments

So how do these transformations work in Webpack?

1.2 Core Process

Module translation operation starts from module.codeGeneration call and corresponds to the above flow chart:

Summarize the key steps:

  • Generate method of an object called JavascriptGenerator. Inside the method:

    • Traversal of the moduledependencies 与 presentationalDependenciesAn array of
    • Executes each array itemdependenyThe corresponding to an objecttemplate.applyMethods,applyModify the module code, or update itinitFragmentsAn array of
  • After the traversal is complete, callInitFragment.addToSourceStatic method that will be generated by the previous stepsourceObject and theinitFragmentsArrays are merged into module artifacts

In short, you iterate through the dependencies, modify the module code in the dependent objects, and then merge all changes into the final product. Here’s the key:

  • inTemplate.applyFunction, how to update the module code
  • inInitFragment.addToSourceIn a static method, how to putTemplate.applyThe resulting side effects are combined into the final product

The logic of these two parts is complicated, and they will be explained separately below.

1.3 the Template. The apply function

The JavaScriptGenerator class is definitely a C-bit player in the above process, but it doesn’t directly modify the contents of the module. Instead, it delegates the implementation to the Template type after a few layers.

Webpack 5 source, JavascriptGenerator. Generate function will traverse module dependencies array, called dependent objects corresponding Template subclasses apply method update module content, it is a little round, the original code more spare, Therefore, I extracted the important steps as the following pseudocode:

class JavascriptGenerator { generate(module, GenerateContext) {const source = new ReplaceSource(module.originalSource()); const { dependencies, presentationalDependencies } = module; const initFragments = []; for (const dependency of [...dependencies, ... presentationalDependencies]) {/ / find the dependency of the corresponding template const template = generateContext.dependencyTemplates.get(dependency.constructor); // Apply the template.apply function by passing in the source and initFragments. Template.apply (dependency, source, {initFragments})} Return initFragment.addToSource (source, initFragment.addToSource, initFragment.addToSource, initFragment.addToSource) generateContext); } // Dependency subclass class xxxDependency extends Dependency {} // Dependency subclass Template defines const xxxDependency.Template = class xxxDependencyTemplate extends Template { apply(dep, source, {initFragments}) { // 1. Source. Replace (dep.range[0], dep.range[1] -1, 'some thing') // 2. InitFragment.push (new XXXinitFragment ())}}

Can be seen from the pseudo code above, JavascriptGenerator. Generate function’s logic is relatively curing:

  1. Initializes a series of variables
  2. traversemoduleObject dependent array, find eachdependencyThe correspondingtemplateObject, calltemplate.applyFunction modifies the contents of the module
  3. callInitFragment.addToSourceMethod, mergesource 与 initFragmentsArray to generate the final result

The key here is JavascriptGenerator. Generate function does not operate the module source code, it only provides an implementation framework, the real processing module content translation logic are xxxDependencyTemplate object the apply function, Lines 24-28 in the pseudocode above.

Each Dependency subclass maps to a unique Template subclass, and usually both classes are written in the same file, such as constDependency and constDependencyTemplate; NullDependency NullDependencyTemplate. In the Webpack Make phase, dependencies between modules are documented through the Dependency subclass. Go to the build (SEAL) phase and modify the Module code through the Template subclass.

Module, JavaScriptGenerator, Dependency, and Template form the following interaction:

The Template object can update the module’s code in two ways:

  • Direct manipulationsourceObject that modifies the module code directly. The initial content of the object is equal to the source code of the module, passing through multipleTemplate.applyFunctions are gradually replaced with new code forms as they flow
  • operationinitFragmentsArray to insert supplementary snippets outside of the module source code

The side effect from both of these operations is eventually passed to the initFragment.addtoSource function to synthesize the final result. The details are briefly added below.

1.3.1 Change the code using Source

Source is a set of tools for editing strings in Webpack. It provides a range of string manipulation methods, including:

  • String merge, replace, insert, etc
  • Module code caching, sourceMap mapping, hash calculation, etc

Many plugins and loaders within Webpack and in the community use the Source library to edit the code content, including the Template.Apply system described above. Logically, when starting the module code generation process, Webpack will first initialize the Source object with the original contents of the module, i.e. :

const source = new ReplaceSource(module.originalSource());

After that, the Source content is changed by the Dependency subclasses in order and on demand, such as the core code in the ConstDependencyTemplate:

ConstDependency.Template = class ConstDependencyTemplate extends ( NullDependency.Template ) { apply(dependency, source, templateContext) { // ... if (typeof dep.range === "number") { source.insert(dep.range, dep.expression); return; } source.replace(dep.range[0], dep.range[1] - 1, dep.expression); }};

In the constDependencyTemplate above, the apply function calls source.insert to insert a code based on the parameter condition, or calls source.replace to replace a code.

1.3.2 Update the code with InitFragment

In addition to manipulating the source directly, you can modify the artifacts of the module by manipulating the array of InitFragments in Template.Apply. The InitFragments array entries are typically subclass instances of InitFragment, and they typically take two functions, getContent and getEndContent, to fetch the header and tail of the code fragment, respectively.

For example, the HarmonyImportDependencyTemplate in the apply function:

HarmonyImportDependency.Template = class HarmonyImportDependencyTemplate extends (
  ModuleDependency.Template
) {
  apply(dependency, source, templateContext) {
    // ...
    templateContext.initFragments.push(
        new ConditionalInitFragment(
          importStatement[0] + importStatement[1],
          InitFragment.STAGE_HARMONY_IMPORTS,
          dep.sourceOrder,
          key,
          runtimeCondition
        )
      );
    //...
  }
 }

1.4 Code Merging

After the above template.apply process is completed, the translated source object and the array of code fragment initFragments are generated, and then the initFragment.addtoSource function is called to merge the two into the module artifact.

The core code for AddToSource is as follows:

class InitFragment { static addToSource(source, initFragments, GenerateContext) {// Sequence const SortedFragments = InitFragments. Map (extractFragmentIndex); .sort(sortFragmentWithIndex); / /... const concatSource = new ConcatSource(); const endContents = []; For (const Fragment of SortedFragments) {// merge Fragment.getContent to fetch the fragment content concatSource.add(fragment.getContent(generateContext)); const endContent = fragment.getEndContent(generateContext); if (endContent) { endContents.push(endContent); }} // merge source concatSource.add(source); // merge Fragment.GetEndContent for (const Content of EndContents.Reverse ()) {ConcatSource.add (content); } return concatSource; }}

As you can see, the logic of the addToSource function is:

  • traverseinitFragmentsArrays, merged in orderfragment.getContent()The product of
  • mergesourceobject
  • traverseinitFragmentsArrays, merged in orderfragment.getEndContent()The product of

As a result, the module code merge operation mainly involves wrapping the module code source layer by layer with an array of InitFragments, both of which are maintained at the Template.Apply level.

1.5 example: Custom Banner plug-in

After the Template. Apply translation is combined with the InitFragment.AddToSource translation, the module has completed the transformation from the user code form to the production form. To better understand the above module translation process, we will try to develop a Banner plug-in. To automatically insert a string before each module.

Implementally, the plug-in mainly involves Dependency, Template, hooks objects, code:

const { Dependency, Template } = require("webpack"); class DemoDependency extends Dependency { constructor() { super(); } } DemoDependency.Template = class DemoDependencyTemplate extends Template { apply(dependency, source) { const today = new Date().toLocaleDateString(); source.insert(0, `/* Author: Tecvan */ /* Date: ${today} */ `); }}; module.exports = class DemoPlugin { apply(compiler) { compiler.hooks.thisCompilation.tap("DemoPlugin", (Compilation) => {// Call DependencyTemplates, Register the Dependency to the compilation of the Template mapping. DependencyTemplates. Set (DemoDependency, new DemoDependency Template ()); Compilation. Hooks. SucceedModule. Tap (" DemoPlugin ", (module) = > {/ / module after the completion of the building, Insert DemoDependency object module.addDependency(new DemoDependency()); }); }); }};

Key steps for the example plug-in:

  • writeDemoDependency 与 DemoDependencyTemplateClass,DemoDependencyFor example only, no actual function;DemoDependencyTemplateIn itsapplyIn the callsource.insertInsert a string, as in lines 10-14 of the sample code
  • usecompilation.dependencyTemplatesregisteredDemoDependency 与 DemoDependencyTemplateThe mapping relation of
  • usethisCompilationHook to achievecompilationobject
  • usesucceedModuleHook a subscriptionmoduleThe event is built and calledmodule.addDependencyMethod to addDemoDependencyRely on

The product of the completion of operation, the module object in the generation process is invoked to DemoDependencyTemplate. The apply function, we defined a, string effect such as:

Interested readers can also read the following files from the Webpack 5 repository directly to learn more use cases:

  • Lib/dependencies/ConstDependency js, a simple example, can learnsourceMore ways to do it
  • Lib/dependencies/HarmonyExportSpecifierDependencyTemplate js, a simple example, can learninitFragmentsMore uses of arrays
  • Lib/dependencies/HarmonyImportDependencyTemplate js, use a more complex but the sample, the comprehensive studysource,initFragmentsUse of Arrays

Second, module merger and packaging principle

2.1 introduction

After describing the process of translating individual modules, let’s go back to the flowchart:

Flow chart, the compilation. CodeGeneration translation stage – that is, module function has been completed, after the completion of the module of the translation results will be saved to the compilation. CodeGenerationResults object, A new execution process, module merge packaging, is then started.

The module merging and packaging process will push the corresponding modules and runtimeModule into the template framework according to the rules, and eventually merge the modules and output into a complete bundle file, such as in the above example:

In the Bundle file on the right side of the example, the parts that are red framed are the user code files and the products generated by the runtime module. The rest of the bundles hold up a run-time framework in the form of IIFE, which is the template framework.

(() => { // webpackBootstrap "use strict"; var __webpack_modules__ = ({ "module-a": ((__unused_webpack_module, __webpack_exports__, __webpack_require__) => { // ! The module code,}), "the module - b" : ((__unused_webpack_module __webpack_exports__, __webpack_require__) = > {/ /! Module code,})}); // The module cache var __webpack_module_cache__ = {}; // The require function function __webpack_require__(moduleId) { // ! Webpack CMD achieve} / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * / / /! A variety of runtime / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * / var __webpack_exports__ = {}; // This entry need to be wrapped in an IIFE because it need to be isolated against other modules in the chunk. (() => { / /! Entry module}) (); }) ();

Looking at the logic here, the runtime framework contains the following key parts:

  • The outermost layer is wrapped by an IIFE
  • One records divisionentryExternal to other module code__webpack_modules__Object, whose key is the module identifier; Value is the translated code of the module
  • An extremely simplified CMD implementation:__webpack_require__function
  • Finally, a package arrivedentryThe IIFE function of the code

Module translation is the translation of modules into code that can run in a host environment, such as a browser; Modules merge in tandem to make the modules conform to development expectations and run the application logic normally. Next, let’s reveal how this part of the code is generated.

2.2 Core Process

In compilation. CodeGeneration is performed, that is, all user code module is implemented with the runtime module after the translation operation, seal function call compilation. CreateChunkAssets function, The javascriptModulesPlugin listens for this hook message and starts assembling the bundle. Pseucode:

// Webpack 5 // lib/ Programer.js class Programer.js {seal() {// The Webpack 5 // lib/ Programer.js class Programer.js {seal() { Ready for this. CodeGenerationResults = this. CodeGeneration (enclosing modules). // 1. Call createChunkAssets this.createChunkAssets(); } createChunkAssets() {// Run the render operation for each chunk (const chunk of this.chunks) {// 2. Trigger renderManifest hook const res = this. Hooks. RenderManifest. Call ([] and {the chunk, codeGenerationResults: this.codeGenerationResults, ... others, }); / / submit this assembly results. EmitAsset (res) render (),... others); } } } // lib/javascript/JavascriptModulesPlugin.js class JavascriptModulesPlugin { apply() { compiler.hooks.compilation.tap("JavascriptModulesPlugin", (compilation) => { compilation.hooks.renderManifest.tap("JavascriptModulesPlugin", (result, Options) => {// JavascriptModuulesPlugin returns the assembly function render const render = () => // render inside according to chunk Content, choose to use the template 'renderMain' or 'renderChunk' // 3. Listen for hooks and return the packaging function this.renderMain(options); result.push({ render /* arguments */ }); return result; }); }); } renderMain() {/* */} renderChunk() {/* */} }

The core logic here is that Compilation publishes bundles as RenderManifest hooks; The JavaScriptModulesPlugin listens to this hook and calls different packaging functions depending on the content nature of the chunk.

The above is for Webpack 5 only. In Webpack 4, the packaging logic is centered on
MainTemplateTo complete.

The built-in packaging functions of the JavaScriptModulesPlugin are:

  • renderMain: Used when packaging the main chunk
  • renderChunk: Used when typing the chunk, such as the asynchronous chunk module

The logic of the two packaging functions is close to that of piecing together the modules in order. Here is a brief introduction to the implementation of RenderMain.

2.3 renderMainfunction

The renderMain function involves a lot of scene determination. The original code is long and winding, but I’ve highlighted a few steps:

class JavascriptModulesPlugin { renderMain(renderContext, hooks, compilation) { const { chunk, chunkGraph, runtimeTemplate } = renderContext; const source = new ConcatSource(); / /... // - "var __webpack_module_cache__ = {};" // - "__webpack_require__" function const bootstrap = this.renderBootstrap(renderContext, hooks); / / 2. Calculate the current chunk, in addition to the entry of other modules code const chunkModules = Template. RenderChunkModules (renderContext, inlinedModules? allModules.filter((m) => ! inlinedModules.has(m)) : allModules, (module) => this.renderModule( module, renderContext, hooks, allStrict ? "strict" : true ), prefix ); / / 3. Calculate the shipment line module code const runtimeModules = renderContext. ChunkGraph. GetChunkRuntimeModulesInOrder (the chunk); // 4.1 First, merge the core CMD implementation. That is, above bootstrap code const beforeStartup = template.asString (bootstrap.beforeStartup) + "\n"; source.add( new PrefixSource( prefix, useSourceMap ? new OriginalSource(beforeStartup, "webpack/before-startup") : new RawSource(beforeStartup) ) ); If (const module of runtimeModules. Length > 0) {for (const module of runtimeModules) { compilation.codeGeneratedModules.add(module); For (const m of chunkModules) {const renderedModule = this.renderModule(m, renderContext, renderContext, renderContext, renderContext, renderContext, renderContext, renderContext hooks, false); Source. Add (renderedModule)} // 4.4 Merges the entry module code if (hasEntryModules && runtimeRequirements.has(RuntimeGlobals.returnExportsFromRuntime) ) { source.add(`${prefix}return __webpack_exports__; \n`); } return source; }}

The core logic is:

  • First calculate the bundle CMD code, i.e__webpack_require__function
  • Calculate the current chunk, except for the entry module codechunkModules
  • Calculate the module code at runtime

  • Begin the merge operation with the following substeps:

    • Merge CMD code
    • Merge the Runtime module code
    • traversechunkModulesVariables that merge module code other than entry
    • Merge the entry module code
  • Returns the result

Summary: First, the product forms of different components are calculated, and then the product is assembled and packaged sequentially, and the merged version is output.

At this point, Webpack completes the Bundle translation and packaging process, and then calls Compilation. emitAsset, and outputs the product to FS according to the context, and the Webpack single compilation and packaging process is over.

Third, summary

This article delves into the Webpack source code and discusses in detail the second half of the packaging process, from the generation of the Chunk Graph to the final output. The key points are:

  • It starts by traversing all the modules in the Chunk, performing translation operations for each module, and producing module level artifacts
  • According to the type of Chunk, different structural frameworks are selected, module products are assembled in sequence, and packaged into the final bundle

To recap, we:

  • In the article “Understanding the Core Principles of Webpack”, a highly summarized discussion of the workflow of Webpack from front to back, to help readers to have a more abstract understanding of the implementation principle of Webpack;
  • The implementation principle of Webpack plug-in mechanism is introduced in detail in “[source interpretation] Webpack plug-in architecture in-depth explanation”, which helps readers to deeply understand the design of Webpack architecture and hooks.
  • In Slight Webpack Knowledge: Dependency Graph Dependency Graph, we introduce the vague concept of module Dependency graphs to help readers understand the process of Dependency discovery and Dependency building in Webpack
  • In “A Little Hard Points: Detailed Webpack Chunk Subcontracting Rules”, the basic logic and implementation methods of Chunk Subcontracting are introduced in detail to help readers understand the principle of product sharding
  • In Principles of the Webpack Series 6: Understanding the Webpack Runtime, we explain in detail the origin and function of the run-time code in bundles, other than business modules, to help you understand the running logic of the product
  • Finally, the module translation and merge packaging logic is introduced in this article

At this point, the main process of Webpack compilation and packaging has been able to be well connected, I believe that readers along the context of this article, carefully control the source code to learn patiently, must have a deep understanding of front-end packaging and engineering, mutual encourage.