Write an understanding of Webpack from start to finish

The preparatory work

Create a project containing the following files and folders:

// /src/index.js
import a from './a.js';

console.log(a);

// /src/a.js
import b from './b.js';
const a = `b content: ${b}`;
export default a;

// /src/b.js
const b = 'Im B';
export default b;
Copy the code

Now this code will not work in browsers that do not support ESModule, so you need to use a wrapper to convert it.

Implement module packaging

Before starting the package, let’s make clear the objectives and process of packaging:

1. Find the project entry (i.e/src/index.js) and read its contents;
2. Analyze the contents of entry files, recursively search for their dependencies, and generate dependency graph;
3. From the generated dependency diagram, compile and generate the final output code

/ mybundle. js will be our wrapper, and all the related code will be written in it, so let’s get started!

1. Obtain the module content

To read the contents of the entry file, we create a method getModuleInfo that uses fs to read the contents of the file:

// myBundle.js
const fs = require('fs')
const getModuleInfo = file => {
    const content = fs.readFileSync(file, 'utf-8')
    console.log(content)
}
getModuleInfo('./src/index.js')
Copy the code

No doubt the contents of the index.js file are printed, but it’s a bunch of strings, so how do we know which modules it depends on? There are two ways:

• Regex: Regex matches the ‘import’ keyword to get the corresponding file path, but it is too cumbersome and unreliable, what if there is a string in the code that also has this content?
• Babel: Yes@babel/parserTo convert the code into an AST (Abstract Syntax Tree), and then analyze the AST to find dependencies. That seems to make sense.

No doubt, use the second way.

// mybundle. js const fs = require('fs') const parser = require('@babel/parser') const getModuleInfo = file => { const content = fs.readFileSync(file, 'utf-8') const ast = parser.parse(content, { sourceType: 'Module' // Parsing ESModule must be configured}) console.log(ast) console.log(ast.program.body)} getModuleInfo('./ SRC /index.js')Copy the code

The type attribute identifies the type of the node, the ImportDeclaration corresponds to the import statement, and the source.value is the relative path to the imported module. It has all the data you want, isn’t it great?

2. Generate a dependency table

With the data from the previous step, we need to generate a structured dependency table for subsequent code processing.

This is done by iterating through ast.program.body, extracting the ImportDeclaration node from it and storing it in the dependency table.

There’s no need to implement the details manually, just use @babel/traverse.

NPM I @babel/traverse ## Install @babel/traverseCopy the code

The getModuleInfo method is modified as follows:

// myBundle.js const fs = require('fs') const path = require('path') const parser = require('@babel/parser') const traverse = require('@babel/traverse').default const getModuleInfo = (file) => { const content = fs.readFileSync(file, 'utf-8'); const ast = parser.parse(content, { sourceType: 'module' }); const dependencies = {}; // Traverse (ast, {ImportDeclaration({node}) {const dirname = path.dirname(file);  const newFile = '.'+ path.sep + path.join(dirname, node.source.value); // Add dependencies[node.source.value] = newFile; }}); console.log(dependencies); }; getModuleInfo('./src/index.js')Copy the code

The following output is displayed:

We can then return a complete module information.

Here, we incidentally convert the code to ES5 syntax via Babel’s tools (‘ @babel/core, @babel/preset-env ‘).

NPM I @babel/core @babel/preset-env ## Preset @babel/core @babel/preset-env // mybundle. js const fs = require('fs'); const path = require('path'); const parser = require('@babel/parser'); const traverse = require('@babel/traverse').default; const babel = require('@babel/core'); const getModuleInfo = (file) => { const content = fs.readFileSync(file, 'utf-8'); const ast = parser.parse(content, { sourceType: 'module' }); const dependencies = {}; traverse(ast, { ImportDeclaration({ node }) { const dirname = path.dirname(file); const newFile = '.'+ path.sep + path.join(dirname, node.source.value); dependencies[node.source.value] = newFile; // the relative path is key and the absolute path is value}}); const { code } = babel.transformFromAst(ast, null, { presets: ['@babel/preset-env'] }); const moduleInfo = { file, dependencies, code }; console.log(moduleInfo); return moduleInfo; }; getModuleInfo('./src/index.js');Copy the code

The output is as follows:

Now, the module code has been converted into an object, containing the absolute path of the module, the dependency, and the code converted by Babel. However, the above only handles the index.js dependency. The dependency of A. js is not processed, so it is not a complete dependency table.

Starting at the entry point, each module and its dependencies are analyzed by calling getModuleInfo to return a dependency graph (dependency Graph).

Let’s just write a new method to handle it:

// myBundle.js
const generDepsGraph = (entry) => {
	const entryModule = getModuleInfo(entry);
	const graphArray = [ entryModule ];
	for(let i = 0; i < graphArray.length; i++) {
		const item = graphArray[i];
		const { dependencies } = item;
		if(dependencies) {
			for(let j in dependencies) {
				graphArray.push(
					getModuleInfo(dependencies[j])
				);
			}
		}
	}
	const graph = {};
	graphArray.forEach(item => {
		graph[item.file] = {
			dependencies: item.dependencies,
			code: item.code
		};
	});
	return graph;
};
Copy the code

Now that we have a complete dependency table, we can use this data to generate the final code.

3. Generate output code

Before generating the code, we first look at the above code, we can find that it contains commonJS syntax such as export and require, but our runtime environment (in this case, the browser) does not support this syntax, so we need to implement these two methods ourselves. Post the code first, then slowly write:

Create a new build method to generate the output code.

// myBundle.js
const build = (entry) => {
	const graph = JSON.stringify(generDepsGraph(entry)); 
	return `
		(function(graph){
			function require(module) {				
				var exports = {};				
				return exports;
			};
			require('${entry}')
		})(${graph});
	`;
};

const code = build('./src/index.js');
Copy the code

Description:

• The third lineJSON.stringifyIs to string the data, otherwise it will be received in the execute now function below[object object]Because the following is used in a string template, type conversions occur.
• The returned code is wrapped in IIFE (execute functions now) to prevent module – to – module scope contamination.
• requireThe function needs to be defined in the output, not in the current runtime environment, because it will be executed in the generated code.

Next, we need to get the code for the entry file and use the eval function to execute it:

// myBundle.js
const build = (entry) => {
	const graph = JSON.stringify(generDepsGraph(entry));
	return `
		(function(graph){
			function require(module) {				
				var exports = {};
				(function(require, exports, code){
					eval(code)
				})(require, exports, graph[module].code);
				return exports;
			};
			require('${entry}')
		})(${graph});
	`;
};

const code = build('./src/index.js');
console.log(code);
Copy the code

Description:

• In case the code in code conflicts with our scope (in the return string), we still use the IIFE wrapper and pass in the required parameters.
• graph[module].codeThe code for the entry can be obtained from the dependency table above.

The output is as follows:

This is the result of packing, but before you get too excited, there’s still a big hole.

The way we import modules in the generated code is based on the relative path of ‘index.js’. If the module path introduced by another module is not the same as that of index.js, the corresponding module will not be found (the path is incorrect), so we have to deal with the module path. The absolute path of the module is recorded in the Dependencies property of the previous table.

Add a localRequire function to get the module’s absolute path from Dependencies.

// myBundle.js
const build = (entry) => {
	const graph = JSON.stringify(generDepsGraph(entry));
	return `
		(function(graph){
			function require(module) {
				function localRequire(relativePath) {
					return require(graph[module].dependencies[relativePath]);
				}
				var exports = {};
				(function(require, exports, code){
					eval(code)
				})(localRequire, exports, graph[module].code);
				return exports;
			};
			require('${entry}')
		})(${graph});
	`;
};
Copy the code

Next, write the output code to a file.

// myBundle.js
const code = build('./src/index.js')
fs.mkdirSync('./dist')
fs.writeFileSync('./dist/bundle.js', code)
Copy the code

Finally, I’ll introduce it in HTML to test if it works. There is no doubt that it will work. :smile::smile:

Finally post the complete code :: cow::beers:

// myBundle.js const fs = require('fs'); const path = require('path'); const parser = require('@babel/parser'); const traverse = require('@babel/traverse').default; const babel = require('@babel/core'); const getModuleInfo = (file) => { const content = fs.readFileSync(file, 'utf-8'); const ast = parser.parse(content, { sourceType: 'module' }); const dependencies = {}; traverse(ast, { ImportDeclaration({ node }) { const dirname = path.dirname(file); const newFile = '.'+ path.sep + path.join(dirname, node.source.value); dependencies[node.source.value] = newFile; }}); const { code } = babel.transformFromAst(ast, null, { presets: ['@babel/preset-env'] }); const moduleInfo = { file, dependencies, code }; return moduleInfo; }; const generDepsGraph = (entry) => { const entryModule = getModuleInfo(entry); const graphArray = [ entryModule ]; for(let i = 0; i < graphArray.length; i++) { const item = graphArray[i]; const { dependencies } = item; if(dependencies) { for(let j in dependencies) { graphArray.push( getModuleInfo(dependencies[j]) ); } } } const graph = {}; graphArray.forEach(item => { graph[item.file] = { dependencies: item.dependencies, code: item.code }; }); return graph; }; const build = (entry) => { const graph = JSON.stringify(generDepsGraph(entry)); return ` (function(graph){ function require(module) { function localRequire(relativePath) { return require(graph[module].dependencies[relativePath]); } var exports = {}; (function(require, exports, code){ eval(code) })(localRequire, exports, graph[module].code); return exports; }; require('${entry}') })(${graph}); `; }; const code = build('./src/index.js'); fs.mkdirSync('./dist'); fs.writeFileSync('./dist/bundle.js', code);Copy the code

Until this case is completed, Webpack is like a noble beauty to be kept away from. And through this case, we tear off her mysterious coat, discover inside originally so wonderful, is not beautiful. Of course, the actual situation is not so simple, to handle various boundary cases, but also support loader and plugin, beauty or something 😍.

Knowing so much about the basics of WebPack, it is natural to encounter it in real projects and interviews, and finally we discuss the performance optimization of WebPack

• Extensions: This configuration means that WebPack will look for file suffixes based on extensions, so if our project is mainly written in TS, we can start with.tsx and.ts so that WebPack can quickly parse

  resolve: { extensions: [‘.ts’, ‘.tsx’, ‘.js’] }

• Alias: This configuration is used to map paths so that WebPack can quickly resolve the file path to find the corresponding file and reduce packaging time

module: {
  noParse: [/proj4\.js/]
}
Copy the code

• .eslintignore: This is not a WebPack configuration, but ESLint has a big impact on packaging, and it can speed up packaging times by excluding files that don’t need to be checked by ESLint

node_modules/
test/
mock/
...
Copy the code

npm install --save-dev autodll-webpack-plugin

// build/webpack.base.conf.js
plugins: [
  //...
  new AutoDllPlugin({
    inject: true, 
    filename: '[name].js',
    entry: {
      vendor: [
        'vue'
      ]
    }
  })
]
Copy the code

• For some loaders with high performance overhead, we can also use caching to process. Using cache-loader to cache the results to disk to reduce the number of rebuild files is also a great way to speed up builds.

npm install cache-loader -D // build/webpack.base.conf.js module.exports = { module: { rules: [{ test: /\.jsx?$/, use: ['cache-loader','babel-loader']Copy the code

• In JS code, when we encounter complex calculation, we will try to perform calculation in Web worker, that is, open one more thread. Similarly, when webpack is compiled, can we open an extra process for the complicated and time-consuming loader? The answer is yes. We can use Thread-Loader to enable parallel construction and allocate the resource-consuming Loader to a worker process, thus reducing the performance overhead of the main process.

npm install thread-loader -D // build/webpack.base.conf.js module.exports = { module: { rules: [ { test: /\.jsx?$/, use: ['thread-loader', 'babel-loader']Copy the code

• We know that the smaller the file is, the faster the rendering speed is, so we often use compression when configuring Webpack. However, compression also consumes time. We can use the Terser-webpack-plugin recommended by WebPack4 to enable parallel compression and reduce the compression time.