Hand-write a simple Webpack

Lu Xun said: When we can use something, we should properly understand how it works.

What is Webpack

• The introduction of webpack

Write a simple Webpack

1. Look at the flow chart for Webpack

Of course, I won’t be able to implement all the features, because the capabilities are limited, so I’ll just pick a few important implementations

2. Preparation

Create two projects, one for the project juejin-webpack and one for our own packaging tool named xydPack

1) The main entry file content and packaging configuration content of the Juejin-WebPack project are:

// webpack.config.js

const path = require('path')
const root = path.join(__dirname, '/')

const config = {
    mode : 'development',
    entry : path.join(root, 'src/app.js'),
    output : {
        path : path.join(root, 'dist'),
        filename : 'bundle.js'
    }
}

module.exports = config
Copy the code

// app.js

/* 
    // moduleA.js
        let name = 'xuyede'
        module.exports = name
*/

const name = require('./js/moduleA.js')

const oH1 = document.createElement('h1')
oH1.innerHTML = 'Hello ' + name
document.body.appendChild(oH1)
Copy the code

2) In order to facilitate debugging, we need to link our own xydPack package to the local, and then introduce it into Juejin-webpack, the specific operation is as follows

// 1. Add the bin attribute to the package.json file of the xydPack project, and configure the corresponding command and execution file {"name": "xydpack"."version": "1.0.0"."main": "index.js"."license": "MIT"."bin": {
    "xydpack" : "./bin/xydpack.js"// 2. Add the corresponding path to the xydpack.js file in the xydPack project, and add the running mode of the file at the top#! /usr/bin/env node
console.log('this is xydpack') // 3. Type NPM link // 4 on the xydPack project command line. Type NPM link xydpack // 5 on the command line of the juejin-webpack project. Enter NPX xydpack on the command line of the juejin-webpack project and output this is xydpackCopy the code

3. Write xydpack. Js

From the flowchart of the first step, we can see that the first step of webPack packaging file is to obtain the contents of the packaging configuration file, then instantiate a Compiler class, and then run to open the compilation, so I can change xydpack.js to

#! /usr/bin/env node

const path = require('path')
const Compiler = require('.. /lib/compiler.js')
const config = require(path.resolve('webpack.config.js'))

const compiler = new Compiler(config)
compiler.run()

Copy the code

Then write the content of compiler.js

Ps: Write xydPack by using NPX xydPack in juejin-webPack project to debug

4. Write a compiler. Js

1. Compiler

Based on the above call, Compiler is a class and has a run method to enable compilation

class Compiler {
    constructor (config) {
        this.config = config
    }
    run () {}
}

module.exports = Compiler

Copy the code

2. buildModule

There is a buildModule method in the flowchart to implement the building module dependencies and get the path to the main entry, so we add this method as well

const path = require('path')

class Compiler {
    constructor (config) {
        this.config = config
        this.modules = {}
        this.entryPath = ' 'This.root = process.cwd()} buildModule (modulePath, isEntry) {// modulePath: modulePath (absolute path) // isEntry: is the main entry}run () {
        const { entry } = this.config
        this.buildModule(path.resolve(this.root, entry), true)
    }
}

module.exports = Compiler
Copy the code

In the buildModule method, we need to get the module path and the corresponding code block from the main entry, and change the require method in the code block to the __webpack_require__ method

const path = require('path')
const fs = require('fs')

class Compiler {
    constructor (config) { //... }
    getSource (modulePath) {
        const content = fs.readFileSync(modulePath, 'utf-8')
        returnContent} buildModule (modulePath, isEntry) {// The source code of the modulelet source= this.getsource (modulePath) // the modulePathlet moduleName = '/' + path.relative(this.root, modulePath).replace(/\\/g, '/')

        if (isEntry) this.entryPath = moduleName
    }
    run () {
        const { entry } = this.config
        this.buildModule(path.resolve(this.root, entry), true)
    }
}

module.exports = Compiler

Copy the code

3. parse

After you get the source code of a module, you need to parse, replace the source code, and retrieve the dependencies of the module, so add a parse method to do this. Parsing the code requires two steps:

1. Use the AST abstract syntax tree to parse the source code
2. Several packages are needed

@babel/parser -> generate the source code from the AST. @babel/parser -> Generate the source code from the AST. @babel/parser -> Generate the source code from the ASTCopy the code

Note: @babel/traverse and @babel/ Generator are packages for ES6 and require a default export

const path = require('path')
const fs = require('fs')
const parser = require('@babel/parser')
const t = require('@babel/types')
const traverse = require('@babel/traverse').default
const generator = require('@babel/generator').default

class Compiler {
    constructor (config) { //... }
    getSource (modulePath) { //... }
    parse (source, dirname) {// Generate the ASTlet ast = parser.parse(sourceTraverse (AST, {}) traverse(AST, {}) generates new codelet sourceCode = generator(ast).code
    }
    buildModule (modulePath, isEntry) {
        let source = this.getSource(modulePath)
        let moduleName = '/' + path.relative(this.root, modulePath).replace(/\\/g, '/')

        if (isEntry) this.entryPath = moduleName

        this.parse(source, path.dirname(moduleName))
    }
    run () {
        const { entry } = this.config
        this.buildModule(path.resolve(this.root, entry), true)
    }
}

module.exports = Compiler

Copy the code

So what’s your AST? You can go to the AST Explorer and see what your code looks like when it’s parsed into an AST.

When there is a function call statement similar to the require (s)/document. The createElement method (s)/document. The body. The appendChild (), there will be a CallExpression attribute to save this information, so the next thing to do is:

• The function call that needs to be changed in the code isrequire“, so there is a layer of judgment to be made
• The referenced module path plus the main modulepathThe directory name

const path = require('path')
const fs = require('fs')
const parser = require('@babel/parser')
const t = require('@babel/types')
const traverse = require('@babel/traverse').default
const generator = require('@babel/generator').default

class Compiler {
    constructor (config) { //... }
    getSource (modulePath) { //... }
    parse (source, dirname) {// Generate the ASTlet ast = parser.parse(source) // List of module dependenciesletTraverse (AST, {CallExpression (p) {const node = p.countif (node.callee.name === 'require') {// Replace the function name with node.callee.name ='__webpack_require__'// Replace the pathlet modulePath = node.arguments[0].value
                    if(! path.extname(modulePath)) { // require('./js/moduleA') Throw new Error(' No file found:${modulePath}To check if the correct file suffix ')} modulePath = is added'/' + path.join(dirname, modulePath).replace(/\\/g, '/') node.arguments = [t.stingLiteral (modulePath)] // Save module dependencies dependencies. Push (modulePath)}}}) // Generate new codelet sourceCode = generator(ast).code
        return { 
            sourceCode, dependencies
        }
    }
    buildModule (modulePath, isEntry) {
        let source = this.getSource(modulePath)
        let moduleName = '/' + path.relative(this.root, modulePath).replace(/\\/g, '/')

        if (isEntry) this.entryPath = moduleName

        let { sourceCode, dependencies } = this.parse(source, path.dirname(moduleName))
    }
    run () {
        const { entry } = this.config
        this.buildModule(path.resolve(this.root, entry), true)
    }
}

module.exports = Compiler
Copy the code

Recursively retrieve all module dependencies and save all paths and dependent modules

const path = require('path')
const fs = require('fs')
const parser = require('@babel/parser')
const t = require('@babel/types')
const traverse = require('@babel/traverse').default
const generator = require('@babel/generator').default

class Compiler {
    constructor (config) { //... }
    getSource (modulePath) { //... }
    parse (source, dirname) { //... }
    buildModule (modulePath, isEntry) {
        let source = this.getSource(modulePath)
        let moduleName = '/' + path.relative(this.root, modulePath).replace(/\\/g, '/')

        if (isEntry) this.entryPath = moduleName

        let { sourceCode, dependencies } = this.parse(source, path.dirname(moduleName))

        this.modules[moduleName] = JSON.stringify(sourceCode)

        dependencies.forEach(d => this.buildModule(path.join(this.root, d)), false)}run () {
        const { entry } = this.config
        this.buildModule(path.resolve(this.root, entry), true)
    }
}

module.exports = Compiler
Copy the code

4. emit

After all the module dependencies and the main entry are obtained, the next step is to insert the data into the template and write output.path in the configuration item

Since we need a template, we borrow the webPack template and use EJS to generate the template. If you don’t know EJS, click here. The template content is:

// lib/template.ejs

(function (modules) {
    var installedModules = {};
  
    function __webpack_require__(moduleId) {
      if (installedModules[moduleId]) {
        return installedModules[moduleId].exports;
      }
  
      var module = installedModules[moduleId] = {
        i: moduleId,
        l: false,
        exports: {}
      };
  
      modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
      module.l = true;
      return module.exports;
    }
  
    return __webpack_require__(__webpack_require__.s = "<%-entryPath%>"); < % ({})for (const key in modules) {%>
        "<%-key%>":
        (function (module, exports, __webpack_require__) {
            eval(<%-modules[key]%>); }}), < % % >});Copy the code

Let’s write the emit function

const path = require('path')
const fs = require('fs')
const parser = require('@babel/parser')
const t = require('@babel/types')
const traverse = require('@babel/traverse').default
const generator = require('@babel/generator').default
const ejs = require('ejs')

class Compiler {
    constructor (config) { //... }
    getSource (modulePath) { //... }
    parse (source, dirname) { //... }
    buildModule (modulePath, isEntry) { //... }
    emit () {
        const { modules, entryPath } = this
        const outputPath = path.resolve(this.root, this.config.output.path)
        const filePath = path.resolve(outputPath, this.config.output.filename)
        if(! fs.readdirSync(outputPath)) { fs.mkdirSync(outputPath) } ejs.renderFile(path.join(__dirname,'template.ejs'), { modules, entryPath })
            .then(code => {
                fs.writeFileSync(filePath, code)
            })
    }
    run () {
        const { entry } = this.config
        this.buildModule(path.resolve(this.root, entry), true)
        this.emit()
    }
}

module.exports = Compiler
Copy the code

So if I write here, if I type NPX xydPack into the juejin-webpack project it will generate a dist directory with a bundle.js file that I can run in the browser

Three plus Loader

After two, just a simple turn of the code, seems to be meaningless ~

So what we’re going to do is we’re going to add loader, if we’re not familiar with Loader, we’re going to add loader here, because it’s written by hand, so we’re going to add Loader itself

Note: Since this is quite simple, you can only play with the style loader, so you can’t play with the other style loader, so I will show you how to write the style loader

1. Style loader

Personally, I use stylus to write styles, so I use stylus-loader and style-loader for styles

First, add Loader to the configuration item, and then introduce init.styl in app.js

// webpack.config.js
const path = require('path')
const root = path.join(__dirname, '/')

const config = {
    mode : 'development',
    entry : path.join(root, 'src/app.js'),
    output : {
        path : path.join(root, 'dist'),
        filename : 'bundle.js'
    },
    module : {
        rules : [
            {
                test: /\.styl(us)? $/, use : [ path.join(root,'loaders'.'style-loader.js'),
                    path.join(root, 'loaders'.'stylus-loader.js')
                ]
            }
        ]
    }
}

module.exports = config
-----------------------------------------------------------------------------------------
// app.js

const name = require('./js/moduleA.js')
require('./style/init.styl')

const oH1 = document.createElement('h1')
oH1.innerHTML = 'Hello ' + name
document.body.appendChild(oH1)

Copy the code

Create a loaders directory in the root directory to write our loader

// stylus-loader

const stylus = require('stylus')
function loader (source) {
    let css = ' '
    stylus.render(source, (err, data) => {
        if(! err) { css = data }else {
           throw new Error(error)
        }
    })
    return css
}
module.exports = loader
-----------------------------------------------------------------------------------------
// style-loader

function loader (source) {
    let script = `
        let style = document.createElement('style')
        style.innerHTML = ${JSON.stringify(source)}
        document.body.appendChild(style)
    `
    return script
}
module.exports = loader
Copy the code

Loaders operate when reading files, so modify compiler.js and add the corresponding operation to getSource

const path = require('path')
const fs = require('fs')
const parser = require('@babel/parser')
const t = require('@babel/types')
const traverse = require('@babel/traverse').default
const generator = require('@babel/generator').default
const ejs = require('ejs')

class Compiler {
    constructor (config) { //... }
    getSource (modulePath) {
        try {
            let rules = this.config.module.rules
            let content = fs.readFileSync(modulePath, 'utf-8')

            for (let i = 0; i < rules.length; i ++) {
                let { test, use } = rules[i]
                let len = use.length - 1

                if(test.test(modulePath)) {// recursively process all loadersfunction loopLoader () {
                        let loader = require(use[len--])
                        content = loader(content)
                        if (len >= 0) {
                            loopLoader()
                        }
                    }
                    loopLoader()
                }
            }

            returnContent} catch (error) {throw new error (' get data error:${modulePath}`)
        }
    }
    parse (source, dirname) { //... }
    buildModule (modulePath, isEntry) { //... }
    emit () { //... }
    run () { //... }
}

module.exports = Compiler
Copy the code

Then run NPX XydPack, which adds a piece of code like this

"./src/style/init.styl":
(function (module, exports, __webpack_require__) {
    eval("let style = document.createElement('style'); \nstyle.innerHTML = \"* {\\n padding: 0; \\n margin: 0; \\n}\\nbody {\\n color: #f40; \\n}\\n\"; \ndocument.head.appendChild(style);");
}),

Copy the code

And then just run it. Demo

*2. Loader of the script

The script loader, the first thought is babel-Loader, we write a babel-Loader, but need to use Webpack to package, modify the configuration file to

// webpack.config.js

resolveLoader : {
    modules : ['node_modules', path.join(root, 'loaders')]
},
module : {
    rules : [
        {
            test : /\.js$/,
            use : {
                loader : 'babel-loader',
                options : {
                    presets : [
                        '@babel/preset-env']}}}]}Copy the code

Use the Babel need three packages: @ Babel/core | @ Babel/preset – env | loader – utils after installation, then write the Babel – loader

const babel = require('@babel/core')
const loaderUtils = require('loader-utils')

function loader (source) {
    let options = loaderUtils.getOptions(this)
    let cb = this.async();
    babel.transform(source, { 
        ...options,
        sourceMap : true,
        filename : this.resourcePath.split('/').pop(),}, (err, result) => {sourceCb (err, result.code, result.map)})} module.exports = loaderCopy the code

Then just use WebPack

4. To summarize

At this point, we can take a rough guess as to how WebPack works:

1. Obtaining configuration Parameters
2. Instantiate the Compiler and use the run method to enable compilation
3. Based on the entry file, create dependencies and recursively get the dependent modules for all modules
4. Use the Loader to parse the matched module
5. Get templates and bundle parsed data into different templates
6. Output file to the specified path

Note: THIS is just for fun. To learn Webpack, click here

Ps: immediately graduated and then unemployed, there is no which company lacks the page son please contact me, cut diagram also, I am very resistant to make

Email address: [email protected]