Why write this article

When I was in G and Baidu, I saw some good articles, but they always gave me an impression and I forgot them. But this time, AFTER reading them, I plan to write an article I understand from the perspective of algorithm to see what packaging tools like Webpack do. What’s the hardest part?

Train of thought

So let’s take a look at this picture

The diagram above shows the implementation flow of WebPack

Where do we wonder?

Question: How do I collect dependencies by walking through the AST? What algorithm did it go through?

Because when you have a lot of complex dependencies, how to solve the problem of these dependencies, should be the most difficult point

The instance

Look at the source file

First file

//word.js
export const word = 'hello'
Copy the code

Second file

//message.js
import { word } from './word.js';
const message = `say ${word}`
export default message;
Copy the code

The third file

//index.js
import message from './message.js'
console.log(message)
Copy the code

See index –> message –> word

conversion

See how these three files depend on each other; You have some ideas in mind:

  1. Use Babel to convert code and generate single file dependencies: @bable/parse can generate AST; @babel/traverse AST traversal to record dependencies; Finally, the code is converted with @babel/core and @babel/preset-env
  2. Dependency map generation
  3. Generate the final package code

That’s our problem, writing a function that takes the above ES6 code and turns it into ES5 and takes that file code and produces a piece of code that the browser can run

Code implementation

Implement the code along the lines above

The first step:
// Install the corresponding package first
npm install @babel/parser @babel/traverse @babel/core @babel/preset-env -D

/ / import packages
const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
const traverse = require('@babel/traverse').default
const babel = require('@babel/core')

// handle the function
function stepOne(filename){
    // Read the file
    const ast = readFile(filename)
    // Iterate over the AST abstract syntax 🌲
    const dependencies = traverseAST(ast)
    
    // The code is converted via @babel/core and @babel/preset-env
    const { code } = babel.transformFromAst(ast, null, {
        presets: ["@babel/preset-env"]})// Returns the file name and dependencies
    return {
        filename,
        dependencies,
        code
    }
}

// Maybe you've seen code neatness recently, and you've made a point of naming things properly and trying not to put them together, even if it's just a demo
function readFile(filename){
    const content =  fs.readFileSync(filename, 'utf-8')
    const ast = parser.parse(content, {
        sourceType: 'module'// Babel specifies that this parameter must be added, otherwise the ES Module cannot be identified
    })
    return ast
}

function traverseAST(ast){
    const dependencies = {}
    traverse(ast, {
        // Get the module imported through import
        ImportDeclaration({node}){
            const dirname = path.dirname(filename)
            const newFile = '/' + path.join(dirname, node.source.value)
            // Save the dependent module
            dependencies[node.source.value] = newFile
        }
    })
    
    return dependencies
}

Copy the code
Step 2: Generate the dependency graph.
function stepTwo(entry){

    // Get the AST object with the dependencies
    const entryModule = stepOne(entry)
    
    // Use the depth algorithm
    const graphArray = getGraphArray(entryModule)
    
    // The next step is to generate the graph
    const graph = getGraph(graphArray)
    
    // Return the graph
    return graph
}

function getGraphArray(entryModule) {
    const graphArray = [entryModule]
    for(let i = 0; i < graphArray.length; i++){
        const item = graphArray[i];
        const {dependencies} = item;// Get the set of modules on which the file depends (key-value pair storage)
        for(let j in dependencies){
            graphArray.push(
                one(dependencies[j])
            )// Knock on the blackboard! Key code to place an entry module and all its associated modules into an array}}}function getGraph(graphArray) {
    const graph = {}
    graphArray.forEach(item= > {
        graph[item.filename] = {
            dependencies: item.dependencies,
            code: item.code
        }
    })
    return graph
}

Log (stepTwo('./ SRC /index.js'))

Copy the code

So you can kind of get a sense of how your code handles this AST tree, just like the algorithm handles binary trees, finding a pattern, repeating itself, all right

Step 3: Generate code strings

function stepThree(entry){
    // // must first convert the Object to a string, otherwise the following template string will default to the toString method of the Object, the argument is changed to [Object Object], obviously not
    const graph = JSON.stringify(stepTwo(entry))
    
    return '(function(graph) {//require a module of code, Function require(module) {// Exports function require(module) {// Exports function require(relativePath) { return require(graph[module].dependencies[relativePath]); } var exports = {}; (function(require, exports, code) { eval(code); })(localRequire, exports, graph[module].code); return exports; // Exports variables are not destroyed after function execution} require('${entry}')
        })(${graph}) `
}

Log (stepThree('./ SRC /index.js'))
Copy the code

Conclusion:

In fact, what you’ll find is that if you find the most complex problem, and you solve it algorithmatically or otherwise, you’re pretty much done with Webpack; Of course, the real WebPack does a lot of other things, which is not the focus of this title; Read all kinds of classic tools, you will find the core, the most complex or algorithm!