What is a bundler

There are many Bundlers out there, most notably Webpack, but also browserify, rollup, parcel, etc. While today’s Bundler has evolved a variety of features, they all share a common purpose: to introduce a modular approach to front-end development, better dependency management, and better engineering.

Modules

There are two most common types of modular systems:

ES6 Modules:

// Import the module import _ from'lodash'; // Export the moduleexport default someObject;
Copy the code

CommonJS Modules:

// introduce the module const _ = require('lodash'); // exports = someObject; // exports = someObject;Copy the code

Dependency Graph

A typical project needs an entry point file, from which Bundler enters, finds all the modules that the project depends on, forming a dependency graph, from which Bundler further packages all modules into a single file.

Dependency graph:

Bundler implementation

To implement a Bundler, there are three main steps:

  1. Parse a file and extract its dependencies

  2. Recursively extract dependencies and generate dependency diagrams

  3. Package all dependent modules into one file

This article uses a small example to show how to implement Bundler. As shown in the following figure, there are three JS files: entry file entry.js, entry.js dependency file greeting.js, and greeting.js dependency file name.js:

The contents of the three files are as follows:

Entry. Js:

import greeting from './greeting.js';

console.log(greeting);
Copy the code

The greeting. Js:

import { name } from './name.js';

export default `hello ${name}! `;Copy the code

Name. Js:

export const name = 'MudOnTire';
Copy the code

Realize bundler

First we create a new bundler. Js file, where the main logic of bundler is written.

1. Introduce JS Parser

For our implementation, we first need to be able to parse the contents of a JS file and extract its dependencies. We could read the contents of the file as strings and use the re to retrieve the import and export statements, but this approach is not very elegant and efficient. A better approach is to use JS Parser to parse the contents of the file. JS Parser can parse JS code and transform it into a high-order model of abstract syntax tree (AST). Abstract syntax tree is to disassemble JS code into a tree structure, from which more details of code execution can be obtained.

You can see the result of javascript code parsing into an abstract syntax tree at AST Explorer. For example, the contents of greeting.js are parsed using Acron Parser to look like this:

You can see that the abstract syntax tree is actually a JSON object, with each node having a type attribute and the result of import, export statement parsing, and so on. It is easier to extract the key information after the code is transformed into an abstract syntax tree.

Next, we need to introduce a JS Parser into the project. We chose Babylon (which is also Babel’s internal JS Parser and currently exists in Babel’s main repository as @babel/parser).

Install the Babylon:

npm install --save-dev @babel/parser
Copy the code

Or yarn:

yarn add @babel/parser --dev
Copy the code

In bundler. Js, insert Babylon:

Bundler. Js:

const parser = require('@babel/parser');
Copy the code

2. Generate an abstract syntax tree

With the JS Parser, it is very easy to generate an abstract syntax tree. All we need to do is to get the contents of the JS source file and pass it to the parser for parsing.

Bundler. Js:

const parser = require('@babel/parser');
const fs = require('fs'); /** * obtain the abstract syntax tree of JS source file * @param {String} filename filename */function getAST(filename) {
  const content = fs.readFileSync(filename, 'utf-8');
  const ast = parser.parse(content, {
    sourceType: 'module'
  });
  console.log(ast);
  return ast;
}

getAST('./example/greeting.js');
Copy the code

Run node bundler. Js and the result is as follows:

3. Dependency resolution

After generating the abstract syntax tree, we can look for dependencies in our code, either by writing our own query methods to find them recursively or by using @babel/traverse modules, which maintain the state of the tree and replace, delete, and add nodes.

Install @ Babel/traverse by:

npm install --save-dev @babel/traverse
Copy the code

Or yarn:

yarn add @babel/traverse --dev
Copy the code

Using @babel/traverse is an easy way to get import nodes.

Bundler. Js:

const traverse = require('@babel/traverse').default; /** * Get ImportDeclaration */functiongetImports(ast) { traverse(ast, { ImportDeclaration: ({ node }) => { console.log(node); }}); } const ast = getAST('./example/entry.js');
getImports(ast);
Copy the code

Run node bundler. Js. The command output is as follows:

From this we can obtain the dependent modules in entry.js and the paths of those modules. Modify the getImports method slightly to get all dependencies:

Bundler. Js:

functiongetImports(ast) { const imports = []; traverse(ast, { ImportDeclaration: ({ node }) => { imports.push(node.source.value); }}); console.log(imports);return imports;
}
Copy the code

Execution result:

Finally, we encapsulate the method to generate unique dependency information for each source file, including the id of the dependent module, the relative path of the module, and the dependencies of the module:

let ID = 0;

function getAsset(filename) {
  const ast = getAST(filename);
  const dependencies = getImports(ast);
  const id = ID++;
  return {
    id,
    filename,
    dependencies
  }
}

const mainAsset = getAsset('./example/entry.js');
console.log(mainAsset);
Copy the code

Execution result:

4. Generate a Dependency Graph

Then, we need to write a method to generate the dependency graph. This method should take the entry file path as a parameter and return an array containing all the dependencies. Dependency diagrams can be generated recursively or by queuing. In this paper, the queue is used. The principle is to iterate over asset objects in the queue. If the asset object’s dependencies are not empty, the asset is generated for each dependency and added to the queue, and the mapping attribute is added for each asset to record the relationship between dependencies. This process continues until the elements in the queue are fully traversed. The specific implementation is as follows:

bundler.js

/** * Generate dependency graph * @param {String} entry file path */function createGraph(entry) {
  const mainAsset = getAsset(entry);
  const queue = [mainAsset];

  for (const asset of queue) {
    const dirname = path.dirname(asset.filename);
    asset.mapping = {};
    asset.dependencies.forEach((relPath, index) => {
      const absPath = path.join(dirname, relPath);
      const child = getAsset(absPath);
      asset.mapping[relPath] = child.id;
      queue.push(child);
    });
  }

  return queue;
}
Copy the code

The generated dependencies are as follows:

5. Packaging

Finally, we need to package all the files into one file based on the dependency diagram. There are several key points in this step:

  1. The packaged file needs to run in the browser, so the ES6 syntax in the code needs to be compiled by Babel first

  2. In the browser environment, compiled code still needs to implement references between modules

  3. After merging into a single file, the different modules still need to be scoped independently

(1). Compile source code

First install Babel and introduce:

npm install --save-dev @babel/core
Copy the code

Or yarn:

yarn add @babel/core --dev
Copy the code

Bundler. Js:

const babel = require('@babel/core');
Copy the code

And then modify the getAsset methods, here we use the Babel. TransformFromAstSync () method for the compilation of the abstract syntax tree, compiled into the browser can execute JS:

functiongetAsset(filename) { const ast = getAST(filename); const dependencies = getImports(ast); const id = ID++; / / compile const. {code} = Babel transformFromAstSync (ast, null, {presets:'@babel/env']});return {
    id,
    filename,
    dependencies,
    code
  }
}
Copy the code

The dependency graph generated after source code compilation is as follows:

You can see that the compiled code also has the require(‘./greeting.js’) syntax, whereas the require() method is not supported in the browser. So we also need to implement the require() method to implement references between modules.

(2) module reference

First, the packaged code needs its own scope, so as not to pollute other JS files. Use IIFE packages here. We can first outline the structure of the packaging method by adding the bundle() method in bundler.

Bundler. Js:

/** * package * @param {Array} graph */function bundle(graph) {
  let modules = ' '; // Pass IIFE graph. ForEach (mod => {modules += '${mod.id}: [function (require, module, exports) { ${mod.code}},
      ${JSON.stringify(mod.mapping)}], `}) / /return `
    (function(a) ({{})${modules}`}); }Copy the code

Let’s first take a look at the result after executing the bundle() method (with js-beautify and CLI-highlight for easy reading) :

Now that we need to implement references between modules, we need to implement the require() method. The implementation idea is: Js (‘./greeting.js’);}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}} The full implementation of the bundle() method is as follows:

Bundler. Js:

/** * package * @param {Array} graph */function bundle(graph) {
  let modules = ' '; // Pass IIFE graph. ForEach (mod => {modules += '${mod.id}: [function (require, module, exports) { ${mod.code}},
      ${JSON.stringify(mod.mapping)}
    ],`
  })

  const bundledCode = `
    (function (modules) {

      function require(id) {
        const [fn, mapping] = modules[id];

        function localRequire(relPath) {
          return require(mapping[relPath]);
        }

        const localModule = { exports : {} };
        
        fn(localRequire, localModule, localModule.exports);

        return localModule.exports; } require(0); ({})${modules}`}); fs.writeFileSync('./main.js', bundledCode);
}
Copy the code

Finally, let’s run the contents of main.js in the browser and see what happens:

A simple version of Webpack is ready!

This article source: github.com/MudOnTire/b…