Preface >In this paper, the sample

Handwritten loader

What is the loader?

Loader converts the source code of a module. Loader allows you to preprocess files when importing or “loading” modules. Thus, loaders are similar to “tasks” in other build tools and provide a useful way to handle front-end build steps. Loader can convert files from different languages (such as TypeScript) to JavaScript or inline images to data urls. Loader even allows you to import CSS files directly from JavaScript modules!

For WebPack, all resources are modules, but since WebPack only supports ES5 JS and JSON by default, es6+, React, CSS and so on are processed by Loader.

Loader code structure

Loader is simply a JS module exported as a function.

module.exports = function(source, map) {
	return source;
}
Copy the code

Source indicates the matched file resource string, and map indicates SourceMap.

Note: Do not write the arrow function, because the internal loader properties and methods need to be called by this, such as the default to enable loader caching, this.cacheable(false) to turn caching off

Synchronous loader

Requirement: replace a string in js

Implementation:

A new replaceLoader. Js:

module.exports = function (source) {
  return `${source.replace('hello'.'world')} `;
};
Copy the code

Webpack. Config. Js:

const path = require('path');

module.exports = {
  mode: 'development'.entry: './src/index.js'.output: {
    path: path.join(__dirname, 'dist'),
    filename: 'main.js',},module: {
    rules: [{ test: /\.js$/, use: './loaders/replaceLoader.js'}],}};Copy the code

Passing parameters

The above replaceLoader is fixed to replace a string (hello), the real scenario is more passed in as an argument

Webpack. Config. Js:

const path = require('path');

module.exports = {
  mode: 'development'.entry: './src/index.js'.output: {
    path: path.join(__dirname, 'dist'),
    filename: 'main.js',},module: {
    rules: [{test: /\.js$/.// Pass the parameters through the options parameter
        use: [
          {
            loader: './loaders/replaceLoader.js'.options: {
              name: 'hello',}},],// Pass the parameter through a string
        // use: './loaders/replaceLoader.js? name=hello'},],}};Copy the code

If the query attribute is used to obtain the parameter, it will appear that the string parameter is a string, and the options parameter is an object format, which is difficult to handle. It is recommended to use the Loader-utils library for this.

The getOptions function of the library is used to receive the parameters

const { getOptions } = require('loader-utils');
module.exports = function (source) {
  const params = getOptions(this);
  return `${source.replace(params.name, 'world')} `;
};

Copy the code

Exception handling

First: throw directly from the loader

const { getOptions } = require('loader-utils');
module.exports = function (source) {
  const params = getOptions(this);
  throw new Error('Something went wrong');
};
Copy the code

Second: pass the error through this.callback

this.callback({
    // Return an Error to Webpack when the original content cannot be converted
    error: Error | Null,
    // The converted content
    content: String | Buffer,
    // Convert the content to the Source Map of the original content (optional)sourceMap? : SourceMap,// Generate the AST syntax tree from the original content (optional)abstractSyntaxTree? : AST })Copy the code

The first argument represents the error message. When null is passed, it acts like the preceding direct return string. It is recommended to return the content in this way

const { getOptions } = require('loader-utils');
module.exports = function (source) {
  const params = getOptions(this);
  this.callback(new Error("Something went wrong."), `${source.replace(params.name, 'world')} `);
};

Copy the code

Asynchronous processing

This.async () tells WebPack that the current loader is running asynchronously when there is an asynchronous requirement to handle, such as fetching a file.

const fs = require('fs');
const path = require('path');
module.exports = function (source) {
  const callback = this.async();
  fs.readFileSync(
    path.resolve(__dirname, '.. /src/async.txt'),
    'utf-8'.(error, content) = > {
      if (error) {
        callback(error, ' ');
      }
      callback(null, content); }); };Copy the code

Where callback is the same use as this.callback above.

The output file

File writing is done through this.emitfile.

const { interpolateName } = require('loader-utils');
const path = require('path');
module.exports = function (source) {
  const url = interpolateName(this.'[name].[ext]', { source });
  this.emitFile(url, source);
  this.callback(null, source);
};

Copy the code

resolveLoader

If you want to use the resolveLoader, you can use the resolveLoader to define the path to the loader file.

const path = require('path');

module.exports = {
  mode: 'development'.entry: './src/index.js'.output: {
    path: path.join(__dirname, 'dist'),
    filename: 'main.js',},resolveLoader: { modules: ['./loaders/'.'node_modules']},module: {
    rules: [{test: /\.js$/.// Pass the parameters through the options parameter
        use: [
          {
            loader: 'asyncLoader.js'}, {loader: 'emitLoader.js'}, {loader: 'replaceLoader.js'.options: {
              name: 'hello',}},],// Pass the parameter through a string
        // use: './loaders/replaceLoader.js? name=hello'},],}};Copy the code

Working Mechanism of Plugin

Before writing plugins, let’s talk about how plugins work in WebPack.

In webpack.js we have the following code:

compiler = new Compiler(options.context);
compiler.options = options;
if (options.plugins && Array.isArray(options.plugins)) {
	for (const plugin of options.plugins) {
		if (typeof plugin === "function") {
			plugin.call(compiler, compiler);
		} else{ plugin.apply(compiler); }}}Copy the code

As you can see, the options. Plugins are traversed and the apply method is called in turn. If plugin is a function, of course, the call is called.

Tapable

In the above code, you can see that a Compiler instance is created, which is passed to each plugin. So what exactly does Compiler do?

Entering compiler.js and compilation.js, you can see that both classes inherit from Tapable

class Compiler extends Tapable {}
class Compilation extends Tapable {}
Copy the code

Tapable is a library similar to Node.js’s EventEmitter, which controls the publishing and subscribing of hook functions and webPack’s plug-in system. The Tapable library exposes many Hook classes, including synchronous hooks such as SyncHook. There are also asynchronous hooks, such as AsyncSeriesHook.

New a hook to get the hook we need. This method accepts the array parameter options, which is not required. Such as:

const hook1 = new SyncHook(["arg1"."arg2"."arg3"]);
Copy the code

Hook Binding and execution of a hook

The binding and execution of synchronous and asynchronous hooks are different:

Async*	Sync*
Binding: tapAsync/tapPromise/tap	Binding: tap
Execution: callAsync/promise	Implementation: the call

const { SyncHook } = require('tapable');
const hook1 = new SyncHook(["arg1"."arg2"."arg3"]); 
// Bind events to the WebAPck event stream
hook1.tap('hook1'.(arg1, arg2, arg3) = > console.log(arg1, arg2, arg3)) / / 1, 2, 3
// Execute the bound event
hook1.call(1.2.3)
Copy the code

Simulate plug-in execution

Simulation of a Compiler. Js

const { SyncHook, AsyncSeriesHook } = require('tapable');

module.exports = class Compiler {
  constructor() {
    this.hooks = {
      add: new SyncHook(), // No parameter synchronization
      reduce: new SyncHook(['arg']), // Synchronization with parameters
      fetchNum: new AsyncSeriesHook(['arg1'.'arg2']), / / asynchronous hook
    };
  }
  // Entry executes function
  run() {
    this.add();
    this.reduce(20);
    this.fetchNum('async'.'hook');
  }
  add() {
    this.hooks.add.call();
  }
  reduce(num) {
    this.hooks.reduce.call(num);
  }
  fetchNum() {
    this.hooks.fetchNum.promise(... arguments).then(() = > {},
      (error) = > console.info(error) ); }};Copy the code

Customize a plugin that binds to the hooks defined above

class MyPlugin {
  apply(compiler) {
    compiler.hooks.add.tap('add'.() = > console.info('add'));
    compiler.hooks.reduce.tap('reduce'.(num) = > console.info(num));
    compiler.hooks.fetchNum.tapPromise('fetch tapAsync'.(num1, num2) = > {
      return new Promise((resolve, reject) = > {
        setTimeout(() = > {
          console.log(`tapPromise to ${num1} ${num2}`);
          resolve();
        }, 1000); }); }); }}module.exports = MyPlugin;

Copy the code

Simulation execution

const MyPlugin = require('./my-plugin');
const Compiler = require('./compiler');

const myPlugin = new MyPlugin();
const options = {
  plugins: [myPlugin],
};
const compiler = new Compiler();
for (const plugin of options.plugins) {
  if (typeof plugin === 'function') {
    plugin.call(compiler, compiler);
  } else {
    plugin.apply(compiler);
  }
}
compiler.run();

Copy the code

See MyPlugins for the specific code

The Compiler and Compliation

Compiler: Compiler manager. After WebPack starts, a Compiler object is created that lives until the end.

Compliation: A manager for a single compilation process, such as watch = true, that runs with only one Compiler but creates a new Compilation object each time a file change triggers a recompilation

Write a plugin

With that in mind, now write the plugin by hand

What is a plugin

Plug-ins are the backbone of WebPack. Webpack itself is built on top of the same plugin system you use in your WebPack configuration! Plug-ins are designed to solve other things that the Loader cannot do.

Plugins-like React, Vue lifecycle, emit at a certain point in time, such as emit hooks: execute before you output the asset to the output directory. Done hook: Executed at compile completion.

Plugin code structure

Plugin is a class that has an apply method that accepts the plugin definition with the compiler parameter:

class DemoPlugin {
  // Plug-in name
  apply(compiler) {
    // Define an apply method
    // Synchronizes the hook, uses the tap, the second function argument is only the compilation parameter
    compiler.hooks.compile.tap('demo plugin'.(compilation) = > {
      // Insert hooks
      console.info(compilation); // Plugins handle the logic}); }}module.exports = DemoPlugin;
Copy the code

Plug-in use:

plugins: [ new DemoPlugin() ]
Copy the code

Passing parameters

Just accept it in the constructor class

Receiving parameters:

class DemoPlugin {
  constructor(options) {
    this.options = options;
  }
  // Plug-in name
  apply(compiler) {
    // Define an apply method
    // Synchronizes the hook, uses the tap, the second function argument is only the compilation parameter
    compiler.hooks.compile.tap('demo plugin'.(compilation) = > {
      // Insert hooks
      console.info(this.options); // Plugins handle the logic}); }}module.exports = DemoPlugin;
Copy the code

Pass parameter:

plugins: [new DemoPlugin({ name: 'zhangsan'})].Copy the code

File is written to

During the Emit phase, webPack writes the compliation.assets file to disk. So you can use the Compilation. assets object to set the file to write to.

class CopyRightWebpackPlugin {
  apply(compiler) {
    Tap (); the second function argument is the compilation and cb arguments. You must call cb()
    compiler.hooks.emit.tapAsync(
      'CopyrightWebpackPlugin'.(compilation, cb) = > {
        compilation.assets['copyright.txt'] = {
          source() {
            return 'copyright by webInRoad';
          },
          size() {
            return 11; }}; cb(); }); }}module.exports = CopyRightWebpackPlugin;

Copy the code

Webpack lite

This version does not include the handling of options parameters, such as WebpackOptionsApply, which translates all configuration options parameters into webPack internal plug-ins. It also does not include processing for non-JS, only implementing es6 JS files into browser-side running code. It involves turning JS into AST, obtaining dependency graph and output file.

Webpack lite

Project initialization

npm init -y 
Copy the code

Initialize package.json and create a SRC directory with index.js and welcome.js under it. Where index.js refers to welcome.js. The directory structure is as follows:File code is as follows:

// index.js
import { welcome } from './welcome.js';
document.write(welcome('lisi'));

// welcome.js
export function welcome(name) {
  return 'hello' + name;
}
Copy the code

Create index. HTML in the root directory

<! DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta  name="viewport" content="width=device-width, < img SRC ="./ SRC /index.js"></script> </head> </body> </ HTML >Copy the code

Using a browser to access the index.html is obviously an error because the browser does not currently support the import syntax directly

Uncaught SyntaxError: Cannot use import statement outside a module
Copy the code

Start writing

Essentially, WebPack is a static module packaging tool for modern JavaScript applications. When WebPack processes an application, it builds a dependency graph internally that maps to each module required by the project and generates one or more bundles.

According to the definition of WebPack given on the official website, the simple version of WebPack that we want to implement generally has the following functions:

1. Read configuration file
2. Starting from the entry file, it recursively reads the contents of the files that the module depends on to generate the dependency graph
3. Based on the dependency graph, generate the final code that the browser can run
4. Generate the bundle file

Read configuration file

First create a simplepack.config.js configuration file similar to webpack.config.js

'use strict';

const path = require('path');

module.exports = {
    entry: path.join(__dirname, './src/index.js'),
    output: {
        path: path.join(__dirname, './dist'),
        filename: 'main.js'}};Copy the code

Then create a lib folder under the root directory to implement the simple version of WebPack.

Create index.js to refer to compiler.js and simplepack.config.js

const Compiler = require('./compiler');
const options = require('.. /simplepack.config');
new Compiler(options).run();
Copy the code

The compiler.js file is created there first.

Processing of individual files

Create new parser.js in lib (to parse files) and test.js (to test parser.js functionality)

Read the content

Define a function getAST in parser.js to load the contents of the file using the Node FS package

const fs = require('fs');

module.exports = {
  getAST: (path) = > {
    const content = fs.readFileSync(path, 'utf-8');
    returncontent; }};Copy the code

In the test. In the js

const { getAST } = require('./parser');
const path = require('path');
const content = getAST(path.join(__dirname, '.. /src/index.js'));
console.info(content);

Copy the code

Nodetest.js gets the string contents of the entry file:

Access to rely on

You can use regular expressions to obtain the contents of import and export as well as the corresponding path file names, but when there are too many import files in the file, this method will be troublesome. Here we use Babylon to parse the file and generate the AST abstract syntax tree.

parser.js:

const babylon = require('babylon');
// Generate the AST from the code
getAST: (path) = > {
  const content = fs.readFileSync(path, 'utf-8');
  return babylon.parse(content, {
    sourceType: 'module'.// The ES Module is used in the project
  });
},
Copy the code

test.js

const { getAST } = require('./parser');
const path = require('path');
const ast = getAST(path.join(__dirname, '.. /src/index.js'));
console.info(ast.program.body);
Copy the code

The contents of the file are in ast.program.body.

Run node test.js and print outAs you can see, there are two nodes in the array. Each Node has a type attribute. For example, the first Node has a type value of ImportDeclaration, which corresponds to the first import statement in index.js, and the second line is an expression. So type is ExpressionStatement.

We can get a value of type ImportDeclaration by traversing a Node whose source.value attribute is the relative path of the referenced module. But it’s a little complicated, and we use babel-traverse to get the dependency:

Added the getDependencies function in parser.js to retrieve dependencies from the AST

const fs = require('fs');
const babylon = require('babylon');
const traverse = require('babel-traverse').default;

module.exports = {
  // Generate the AST from the code
  getAST: (path) = > {
    const content = fs.readFileSync(path, 'utf-8');

    return babylon.parse(content, {
      sourceType: 'module'}); },// Analyze dependencies
  getDependencies: (ast) = > {
    const dependencies = [];
    traverse(ast, {
      ImportDeclaration: ({ node }) = >{ dependencies.push(node.source.value); }});returndependencies; }};Copy the code

test.js

const { getAST, getDependencies } = require('./parser');
const path = require('path');
const ast = getAST(path.join(__dirname, '.. /src/index.js'));
const dependencies = getDependencies(ast);
console.info(dependencies);

Copy the code

Perform the node test. Js

Gets the dependent file path relative to the entry file

Compile the content

After obtaining the dependencies, we need to make a syntax conversion on the AST to convert es6 syntax to ES5 syntax using the Babel core modules @babel/core and @babel/preset-env

Add the transform method in parser.js to generate es5 code based on the AST.

const fs = require('fs');
const babylon = require('babylon');
const traverse = require('babel-traverse').default;
const { transformFromAst } = require('babel-core');

module.exports = {
  // Generate the AST from the code
  getAST: (path) = > {
    const content = fs.readFileSync(path, 'utf-8');

    return babylon.parse(content, {
      sourceType: 'module'}); },// Analyze dependencies
  getDependencies: (ast) = > {
    const dependencies = [];
    traverse(ast, {
      ImportDeclaration: ({ node }) = >{ dependencies.push(node.source.value); }});return dependencies;
  },
  // Convert the AST to ES5 code
  transform: (ast) = > {
    const { code } = transformFromAst(ast, null, {
      presets: ['env']});returncode; }};Copy the code

test.js

const { getAST, getDependencies, transform } = require('./parser');
const path = require('path');
const ast = getAST(path.join(__dirname, '.. /src/index.js'));
const dependencies = getDependencies(ast);
const source = transform(ast);
console.info(source);

Copy the code

You can see that it’s converted to ES5 syntax, but there’s a require function in it, which the browser doesn’t come with, so you need to define one.

Get dependency graph

Now that you have achieved the acquisition of a single file dependency, start with the entry module, analyze each module and its dependency modules, and finally return an object containing all the module information stored in this.modules. Create compiler.js in the lib directory

1. The constructor takes the options argument (which contains entry and exit configuration information) and defines this.modules to store the contents of the module
2. The run function, as the entry function, gets all the dependencies from the entry file stored in this.modules
3. BuildModule calls a function wrapped in Parser and returns an object containing the file name, an array of dependencies, and the corresponding executable code

const path = require('path');
const { getAST, getDependencies, transform } = require('./parser');

module.exports = class Compiler {
  constructor(options) {
    const { entry, output } = options;
    this.entry = entry;
    this.output = output;
    this.modules = [];
  }
 // First get the entry information, the object contains the file name, compiled into ES5 code,
 // There is also an array of dependency modules;
 // Then iterate over the dependencies of the modules and add the module information to this.modules.
 // In this way, we can continue to get the module that the dependent module depends on, which is equivalent to recursively getting the module information
  run() {
    const entryModule = this.buildModule(this.entry, true);
    this.modules.push(entryModule);
    this.modules.map((_module) = > {
      _module.dependencies.map((dependency) = > {
        this.modules.push(this.buildModule(dependency));
      });
    });
  }

  // Module build
  // To distinguish between the entry file and the other path, because the other path is relative,
  // We need to convert to an absolute path, or relative to the project folder
  buildModule(filename, isEntry) {
    let ast;
    if (isEntry) {
      ast = getAST(filename);
    } else {
      let absolutePath = path.join(path.dirname(this.entry), filename);
      ast = getAST(absolutePath);
    }

    return {
      filename,
      dependencies: getDependencies(ast),
      transformCode: transform(ast), }; }};Copy the code

Perform the node index. Js

The generated code

The data is analyzed from the above module to produce the final code that the browser runs. Looking at the dependency diagram from the previous section, you can see that the final transformCode contains syntax such as exports and require, which are not native to the browser and need to be implemented in the code. Added the emitFile function in compiler.js to generate the final code and write it to the output

1. Module information will first of all, into a module name as the key, and define a function (the function receives the require and exports parameters, as the body of the function module code) as the value
2. Then define an IIFE, take the modules object as the result of the previous step, define a function called require, take the file name, and define an exports object. And exports as parameters. Return exports
3. Finally, the resulting code is written to the output

// Output file
  emitFiles() {
    const outputPath = path.join(this.output.path, this.output.filename);
    let modules = ' ';
    this.modules.map((_module) = > {
      modules += ` '${_module.filename}': function (require, exports) { ${_module.transformCode}}, `;
    });
    const bundle = `
            (function(modules) {
                function require(fileName) {
                    const fn = modules[fileName];
                    const exports = {};
                    fn(require, exports );
                    return exports;
                }

                require('The ${this.entry}'); ({})${modules}})
        `;

    fs.writeFileSync(outputPath, bundle, 'utf-8');
  }
Copy the code

Call emitFiles in the compiler.js run function

Compiler complete code

const fs = require('fs');
const path = require('path');
const { getAST, getDependencies, transform } = require('./parser');

module.exports = class Compiler {
  constructor(options) {
    const { entry, output } = options;
    this.entry = entry;
    this.output = output;
    this.modules = [];
  }

  run() {
    const entryModule = this.buildModule(this.entry, true);
    this.modules.push(entryModule);
    this.modules.map((_module) = > {
      _module.dependencies.map((dependency) = > {
        this.modules.push(this.buildModule(dependency));
      });
    });
    this.emitFiles();
  }

  // Module build
  buildModule(filename, isEntry) {
    let ast;
    if (isEntry) {
      ast = getAST(filename);
    } else {
      let absolutePath = path.join(path.dirname(this.entry), filename);
      ast = getAST(absolutePath);
    }

    return {
      filename,
      dependencies: getDependencies(ast),
      transformCode: transform(ast),
    };
  }

  // Output file
  emitFiles() {
    const outputPath = path.join(this.output.path, this.output.filename);
    let modules = ' ';
    this.modules.map((_module) = > {
      modules += ` '${_module.filename}': function (require, exports) { ${_module.transformCode}}, `;
    });
    const bundle = `
            (function(modules) {
                function require(fileName) {
                    const fn = modules[fileName];
                    const exports = {};
                    fn(require, exports);
                    return exports;
                }

                require('The ${this.entry}'); ({})${modules}})
        `;

    fs.writeFileSync(outputPath, bundle, 'utf-8'); }};Copy the code

Executing node index.js generates the final code and writes it to main.js in the dist directoryRemember to create the dist directory manually Introducing main.js into index.html should display the results normally