Core principles of handwriting WebPack

  • I. Core packaging principles
    • 1.1 The packaging process is as follows
    • 1.2 Details
  • 2. Basic preparations
  • Third, obtain the module content
  • Iv. Analysis module
  • Collect dependencies
  • 6. ES6 to ES5 (AST)
  • Get all dependencies recursively
  • Handle two keywords

I. Core packaging principles

1.1 The packaging process is as follows

  1. You need to read the contents of the entry file.
  2. Analyze entry files, recursively read the contents of files that the module depends on, and generate an AST syntax tree.
  3. Generate code that the browser can run from the AST syntax tree

1.2 Details

  1. Gets the main module contents
  2. Analysis module
    • Install @babel/ Parser package (go to AST)
  3. Process the module content
    • Mount @babel/ Traverse packs
    • Install @babel/core and @babel/preset-env (ES6 to ES5)
  4. Recurse all modules
  5. Generate the final code

2. Basic preparations

Let’s build a project

The project directory is as follows for now:

Has put the project on the github:https://github.com/Sunny-lucking/howToBuildMyWebpack can humble to a star

We created the add.js file and minus.js file, and then imported the index.js file into the index.html.

The code is as follows:

add.js

export default (a,b)=>{
  return a+b;
}
Copy the code

minus.js

export const minus = (a,b) = >{
    return a-b
}
Copy the code

index.js

import add from "./add.js"
import {minus} from "./minus.js";

const sum = add(1.2);
const division = minus(2.1);
 console.log(sum); console.log(division); Copy the code

index.html

<! DOCTYPE html><html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head> <body> <script src="./src/index.js"></script> </body> </html> Copy the code

Now let’s open index. HTML. Guess what?? Obviously an error will occur because the browser does not yet recognize ES6 syntax such as import

But that’s okay, because we’re here to solve these problems.

Third, obtain the module content

Ok, let’s start with the core packaging principles above. The first step is to implement fetching the main module contents.

Let’s create a bundle.js file.

// Get the main entry file
const fs = require('fs')
const getModuleInfo = (file) = >{
    const body = fs.readFileSync(file,'utf-8')
    console.log(body);
} getModuleInfo("./src/index.js")  Copy the code

The current project directory is as follows

Let’s execute bundle.js and see if we succeeded in obtaining the entry file contents

Wow, a success as expected. Everything is under control. Ok, I’ve done the first step, but let me see what the second step is.

Oh? It’s the analysis module

Iv. Analysis module

The main task of the analysis module is to parse the obtained module contents into an AST syntax tree, which requires a dependency package @babel/ Parser

npm install @babel/parser
Copy the code

Ok, the installation is complete and we’ll introduce @babel/ Parser into bundle.js,

// Get the main entry file
const fs = require('fs')
const parser = require('@babel/parser')
const getModuleInfo = (file) = >{
    const body = fs.readFileSync(file,'utf-8')
 // Add code  const ast = parser.parse(body,{  sourceType:'module' // specifies the ES module to parse  });  console.log(ast); } getModuleInfo("./src/index.js") Copy the code

Take a look at @babel/ Parser’s documentation:

Visibility provides three apis, and we currently use parse.

Its main function is to parses the provided code as an entire ECMAScript program, which is an AST that parses the provided code into complete ECMAScript code.

Take a look at the parameters provided by the API

For the moment, we’re using the sourceType, which specifies what module we’re parsing.

Ok, now let’s execute bundle.js and see if the AST was generated successfully.

Success. Another expected success.

However, it is important to know that we are currently parsing not only the contents of the index.js file, but also other information about the file. And its contents are actually in the body of its property program. As is shown in

We can print ast.program.body instead

// Get the main entry file
const fs = require('fs')
const parser = require('@babel/parser')
const getModuleInfo = (file) = >{
    const body = fs.readFileSync(file,'utf-8')
 const ast = parser.parse(body,{  sourceType:'module' // specifies the ES module to parse  });  console.log(ast.program.body); } getModuleInfo("./src/index.js" Copy the code

perform

Now the printout is the contents of the index.js file (the code we wrote in index.js).

Collect dependencies

Now we need to walk through the AST and gather the dependencies we need. What does that mean? This is a collection of file paths that were imported with the import statement. We put the collected paths into dePS.

As mentioned earlier, traversing the AST uses @babel/traverse dependencies

npm install @babel/traverse
Copy the code

Now, let’s introduce.

const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
const traverse = require('@babel/traverse').default
const getModuleInfo = (file) = >{
 const body = fs.readFileSync(file,'utf-8')  const ast = parser.parse(body,{  sourceType:'module' // specifies the ES module to parse  });   // Add code  const deps = {}  traverse(ast,{  ImportDeclaration({node}){  const dirname = path.dirname(file)  const abspath = '/' + path.join(dirname,node.source.value)  deps[node.source.value] = abspath  }  })  console.log(deps);   } getModuleInfo("./src/index.js") Copy the code

Let’s take a look at the official description of @babel/ Traverse

All right, so brief

But it’s not hard to see that the first parameter is AST. The second parameter is the configuration object

So let’s look at the code we wrote

traverse(ast,{
    ImportDeclaration({node}){
        const dirname = path.dirname(file)
        const abspath = '/' + path.join(dirname,node.source.value)
        deps[node.source.value] = abspath
 } }) Copy the code

In the configuration object, we’ve configured the ImportDeclaration method, what does that mean? Let’s look at the AST that we printed earlier.

The ImportDeclaration method represents the handling of a node of type ImportDeclaration.

Here we get the value of the source in the node, which is node.source.value,

What do I mean by value? This is actually the value of import, which you can see in our index.js code.

import add from "./add.js"
import {minus} from "./minus.js";

const sum = add(1.2);
const division = minus(2.1);
 console.log(sum); console.log(division); Copy the code

‘./add.js’ and ‘./minus. Js’ after import

We then add the file directory path to the obtained value and save it in DEps, which is called collecting dependencies.

Ok, so that’s the end of the operation, let’s see if the collection was successful.

Oh my god. It worked again.

6. ES6 to ES5 (AST)

Now we need to convert the ACQUIRED AST from ES6 to ES5. As mentioned earlier, this step requires two dependency packages

npm install @babel/core @babel/preset-env
Copy the code

We will now introduce dependencies and use them

const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
const traverse = require('@babel/traverse').default
const babel = require('@babel/core')
const getModuleInfo = (file) = >{  const body = fs.readFileSync(file,'utf-8')  const ast = parser.parse(body,{  sourceType:'module' // specifies the ES module to parse  });  const deps = {}  traverse(ast,{  ImportDeclaration({node}){  const dirname = path.dirname(file)  const abspath = ". /" + path.join(dirname,node.source.value)  deps[node.source.value] = abspath  }  })  The new code const {code} = babel.transformFromAst(ast,null, { presets: ["@babel/preset-env"]  })  console.log(code);  } getModuleInfo("./src/index.js") Copy the code

Let’s take a look at @Babel/Core’s transformFromAst documentation

Harm, is as always brief…

In a nutshell, this simply translates the AST we pass into the module type we set in the third parameter.

All right, now let’s do it and see what happens

Oh, my god. Success as always. So it turns us from const to var.

Ok, so that’s the end of this step. Gee, you might be wondering why the collection dependency from the previous step doesn’t matter here, and it does. Dependencies are collected for the following recursive operations.

Get all dependencies recursively

We now know that getModuleInfo is used to retrieve the contents of a module, but we haven’t returned the contents yet, so change the getModuleInfo method

const getModuleInfo = (file) = >{
    const body = fs.readFileSync(file,'utf-8')
    const ast = parser.parse(body,{
        sourceType:'module' // specifies the ES module to parse
    });
 const deps = {}  traverse(ast,{  ImportDeclaration({node}){  const dirname = path.dirname(file)  const abspath = ". /" + path.join(dirname,node.source.value)  deps[node.source.value] = abspath  }  })  const {code} = babel.transformFromAst(ast,null, { presets: ["@babel/preset-env"]  })  // Add code  const moduleInfo = {file,deps,code}  return moduleInfo }  Copy the code

We return an object that contains the module’s path (file), the module’s dependencies (deps), and the module converts to ES5 code

This method can only retrieve information about a module, but how can we retrieve information about dependent modules within a module?

That’s right, look at the title, you should have thought about recursion.

Now let’s write a recursive method that recursively gets dependencies

const parseModules = (file) = >{
    const entry =  getModuleInfo(file)
    const temp = [entry]
    for (let i = 0; i<temp.length; i++){        const deps = temp[i].deps
 if (deps){  for (const key in deps){  if (deps.hasOwnProperty(key)){  temp.push(getModuleInfo(deps[key]))  }  }  }  }  console.log(temp) } Copy the code

Explain parseModules method:

  1. We first pass in the main module path
  2. Put the obtained module information into the Temp array.
  3. The outer loop traverses the Temp array, where the temp array only has the main module
  4. Loop to get the dependency deps of the main module
  5. Through dePs, push the obtained dependency module information into the Temp array by calling getModuleInfo.

Current bundle.js file:

const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
const traverse = require('@babel/traverse').default
const babel = require('@babel/core')
const getModuleInfo = (file) = >{  const body = fs.readFileSync(file,'utf-8')  const ast = parser.parse(body,{  sourceType:'module' // specifies the ES module to parse  });  const deps = {}  traverse(ast,{  ImportDeclaration({node}){  const dirname = path.dirname(file)  const abspath = ". /" + path.join(dirname,node.source.value)  deps[node.source.value] = abspath  }  })  const {code} = babel.transformFromAst(ast,null, { presets: ["@babel/preset-env"]  })  const moduleInfo = {file,deps,code}  return moduleInfo }  // Add code const parseModules = (file) = >{  const entry = getModuleInfo(file)  const temp = [entry]  for (let i = 0; i<temp.length; i++){ const deps = temp[i].deps  if (deps){  for (const key in deps){  if (deps.hasOwnProperty(key)){  temp.push(getModuleInfo(deps[key]))  }  }  }  }  console.log(temp) } parseModules("./src/index.js") Copy the code

According to the current execution of our project, the three modules of index. Js,add.js and minus. Js should be stored in temp. , execution see.

Awesome!! That’s true.

However, the format of the objects in the temp array is not good for later operations. We want to store the files in the form of key, {code, deps} value. So, let’s create a new object, depsGraph.

const parseModules = (file) = >{
    const entry =  getModuleInfo(file)
    const temp = [entry] 
    const depsGraph = {} // Add code
    for (let i = 0; i<temp.length; i++){ const deps = temp[i].deps  if (deps){  for (const key in deps){  if (deps.hasOwnProperty(key)){  temp.push(getModuleInfo(deps[key]))  }  }  }  }  // Add code  temp.forEach(moduleInfo= >{  depsGraph[moduleInfo.file] = {  deps:moduleInfo.deps,  code:moduleInfo.code  }  })  console.log(depsGraph)  return depsGraph } Copy the code

Ok, now that’s the format

Handle two keywords

Our goal now is to generate a bundle.js file, which is a packaged file. The idea is simply to integrate the contents of index.js with its dependent modules. Then write the code to a newly created JS file.

So let’s format this code

// index.js
"use strict"
var _add = _interopRequireDefault(require("./add.js"));
var _minus = require("./minus.js");
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { "default": obj }; }
var sum = (0, _add["default"]) (1.2); var division = (0, _minus.minus)(2.1); console.log(sum); console.log(division);  Copy the code
// add.js
"use strict";
Object.defineProperty(exports, "__esModule", {  value: true});
exports["default"] = void 0;
var _default = function _default(a, b) {  returna + b; };exports["default"] = _default; Copy the code

But we can’t execute index.js because the browser doesn’t recognize require and exports.

Can’t identify why? This is because the require function and exports object are not defined. Well, we can define it ourselves.

Let’s create a function

const bundle = (file) = >{
    const depsGraph = JSON.stringify(parseModules(file))
    
}
Copy the code

Let’s save the depsGraph we got in the previous step.

Now return a fully integrated string code.

How do I get back? Change the bundle function

const bundle = (file) = >{
    const depsGraph = JSON.stringify(parseModules(file))
    return `(function (graph) {
                function require(file) {
 (function (code) {  eval(code)  })(graph[file].code)  }  require(file)  })(depsGraph)`  } Copy the code

So let’s look at the code that comes back

 (function (graph) {
        function require(file) {
            (function (code) {
                eval(code)
            })(graph[file].code)
 }  require(file)  })(depsGraph) Copy the code

In fact, is

  1. Pass the saved depsGraph into an instant-execute function.
  2. Pass the main module path to the require function for execution
  3. When the reuire function is executed, an immediate function is executed, passing in the code value
  4. Perform eval (code). This is the code that executes the main module’s code

Let’s look at the value of code again

// index.js
"use strict"
var _add = _interopRequireDefault(require("./add.js"));
var _minus = require("./minus.js");
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { "default": obj }; }
var sum = (0, _add["default") (1, 2);var division = (0, _minus.minus)(2, 1); console.log(sum); console.log(division);  Copy the code

And yes, when I execute this code, I’m going to use the require function again. The add.js path is not an absolute path for require. So write a function absRequire to convert. How do you do that? So let’s look at the code

(function (graph) {
    function require(file) {
        function absRequire(relPath) {
            return require(graph[file].deps[relPath])
        }
 (function (require,code) {  eval(code)  })(absRequire,graph[file].code)  }  require(file) })(depsGraph) Copy the code

It actually implements a layer of interception.

  1. Execute the require (‘./ SRC /index.js’) function
  2. To perform the
(function (require,code) {
    eval(code)
})(absRequire,graph[file].code)
Copy the code
  1. Execute eval, which is the code that executes index.js.
  2. The execution goes to the require function.
  3. And then this requires, which is the absRequire that we passed in
  4. The execution absRequire is executedreturn require(graph[file].deps[relPath])So this code, this code right here, executes the require

Here,return require(graph[file].deps[relPath])We’ve converted the path to an absolute path. So when you do an external require, you pass in the absolute path.

  1. After require (“./ SRC /add.js”), eval is executed, which is the code that executes the add.js file.

Isn’t that a little convoluted? It’s actually a recursion.

This brings the code together, but there is a problem: exports are not defined when executing add.js code. As shown below.

// add.js
"use strict";
Object.defineProperty(exports, "__esModule", {  value: true});
exports["default"] = void 0;
var _default = function _default(a, b) {  returna + b; };exports["default"] = _default; Copy the code

Exports are not defined yet, so we can define an exports object ourselves.

(function (graph) {
    function require(file) {
        function absRequire(relPath) {
            return require(graph[file].deps[relPath])
        }
 var exports = {}  (function (require,exports,code) {  eval(code)  })(absRequire,exports,graph[file].code)  return exports  }  require(file) })(depsGraph) Copy the code

We added an empty object, exports. When we execute add.js code, we add properties to this empty object.

// add.js
"use strict";
Object.defineProperty(exports, "__esModule", {  value: true});
exports["default"] = void 0;
var _default = function _default(a, b) {  returna + b; };exports["default"] = _default; Copy the code

For example, after executing this code

exports = {
  __esModule：{  value: true},Default:function _default(a, b) {  returna + b; }}
Copy the code

And then we return the exports.

var _add = _interopRequireDefault(require("./add.js"));
Copy the code

So, the value that returns out is received by _interopRequireDefault, and _interopRequireDefault returns the default property to _add, So _add = function _default(a, b) {return a + b; }

Now you can see why the ES6 module introduces an object reference, because exports are an object.

So far, deal with; The two key words are complete.

const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
const traverse = require('@babel/traverse').default
const babel = require('@babel/core')
const getModuleInfo = (file) = >{  const body = fs.readFileSync(file,'utf-8')  const ast = parser.parse(body,{  sourceType:'module' // specifies the ES module to parse  });  const deps = {}  traverse(ast,{  ImportDeclaration({node}){  const dirname = path.dirname(file)  const abspath = ". /" + path.join(dirname,node.source.value)  deps[node.source.value] = abspath  }  })  const {code} = babel.transformFromAst(ast,null, { presets: ["@babel/preset-env"]  })  const moduleInfo = {file,deps,code}  return moduleInfo } const parseModules = (file) = >{  const entry = getModuleInfo(file)  const temp = [entry]  const depsGraph = {}  for (let i = 0; i<temp.length; i++){ const deps = temp[i].deps  if (deps){  for (const key in deps){  if (deps.hasOwnProperty(key)){  temp.push(getModuleInfo(deps[key]))  }  }  }  }  temp.forEach(moduleInfo= >{  depsGraph[moduleInfo.file] = {  deps:moduleInfo.deps,  code:moduleInfo.code  }  })  return depsGraph } // Add code const bundle = (file) = >{  const depsGraph = JSON.stringify(parseModules(file))  return `(function (graph) {  function require(file) {  function absRequire(relPath) {  return require(graph[file].deps[relPath])  }  var exports = {}  (function (require,exports,code) {  eval(code)  })(absRequire,exports,graph[file].code)  return exports  }  require('${file}') }) (${depsGraph}) `  } const content = bundle('./src/index.js')  console.log(content); Copy the code

So let’s do that and see what happens

The execution succeeded. Next, write the returned code into the newly created file

// Write to our dist directoryfs.mkdirSync('./dist');
fs.writeFileSync('./dist/bundle.js',content)
Copy the code

This is the end of our core principle of handwritten WebPack.

Let’s take a look at the bundle.js file that was generated

Discovery is simply passing in all the dependencies we collected earlier as arguments to the immediate execution function, and then executing the code for each dependency recursively via eval.

Now let’s introduce the bundle.js file into index.html and see if it works

Success… pleasantly surprised

Thank you also congratulations you see here, I can humble beg a star!!

github：https://github.com/Sunny-lucking/howToBuildMyWebpack

This article was typeset using MDNICE