preface
Recently saw the webpack in the live class to achieve ideas, consciously benefited, so the content of the inside half carried over to write a blog to share with you, in the process of their own hand to tap the code to deepen the impression, this article is suitable for the webpack have a preliminary understanding of the crowd, not to say more, open the whole 🔧
www.bilibili.com/video/BV1ub…
Front knowledge
Webpack is used to package modules, so this is a brief introduction for those who are not familiar with the concept of modularity. Those who are already familiar with this part can skip to the next section 🔨
Js module concept
A module is usually a code organization mechanism provided by a programming language that allows programs to be broken down into separate, common code units.
- What problem has the advent of modularity solved
The emergence of modularity to solve the code segment, scope, isolation, module dependencies between management and release to the environment of production automation packaging and processing, and other problems, just think, from a modular development approach, our code organization will be very headache, using the file at the same time also have to consider its internal complex dependencies.
2. Modular specifications 🏁
Before Es6,AMD/CMD/CommonJs is the standard for MODULAR DEVELOPMENT of JS. The corresponding implementation is RequireJs/SeaJs/nodeJs. CommonJs is mainly aimed at the server side, while AMD/CMD is mainly aimed at the browser side. The CommonJS specification is most commonly used in Node modules, where exposed modules use module.exports and exports, while there should be a global method require for loading modules.
Standard ES Module at this stage
After the release of ES6 standard, Module has become a standard. The standard uses export instruction to export interface and import module, which has well replaced the previous CommonJS and AMD specifications and become a common module solution for browsers and servers.
Compatibility issues exist and es6 support is required, which is usually compiled into ES5 syntax for downward compatibility using Babel.
Thoroughly understand javascript require, import, and export
Webpack profile
In essence, Webpack is a static Module bundler for modern JavaScript applications. When WebPack works with an application, it recursively builds a Dependency graph containing every module the application needs, and then packages all of those modules into one or more bundles.
Let’s start with the core concepts in WebPack
- Entry The entry file at webPack build time from which dependencies can be established, which can be one or more entries
- Outputs specify the pure output directory of webPack’s bundle, as well as one or more
- Loader is responsible for converting file types (except JS) that are not recognized by WebPack into valid modules that WebPack can handle
- At the root of webPack’s power, plugins broadcast corresponding hook events at different stages of the build. Listening to these events allows for a high degree of customization of the packaging process, from package optimization and compression to redefining variables in the environment.
To realize the mini – webpack
Based on the above, let’s tease out what a basic packer should do for us:
- Locate the package entry based on the configuration file
- Parse the entry file to collect its dependencies
- Recursively look for dependencies and build dependency diagrams between files
- Pack all the files into one file
The basic configuration
Initialize a project:
npm init -y
Copy the codeCreate SRC directory and create module-1.js,module-2.js,module-3.js in SRC directory as follows:
// module-1.js
import res from './module-2'
console.log(`mini-webpack: ${res}`);
Copy the code// module-2.js
import module3 from './module3'
let res = `module2 import ${module3} from module-3.js`
export default res
Copy the code// module-3.js
export default 'hello world
Copy the codeYou can easily see the dependencies as follows: Module1 -> Module2 -> Module3
New file mini-webpack-config.js as our mini-webpack configuration file, the structure of reference to the official webpack:
const path = require("path");
module.exports = {
entry: "./src/module-1.js".output: {
path: path.join(__dirname, "./dist"),
filename: "bundle.js",}};Copy the codeParse the configuration to get the entry entry
Create a new file mini-webpack.js
// mini-webpack.js
const fs = require("fs");
const path = require("path");
// Read the configuration
const config = require("./mini-webpack-config.js");
/ / the main function
function main() {
let absPath = path.resolve(__dirname, config.entry);
let entry = fs.readFileSync(absPath, "utf-8");
console.log(entry);
}
main();
Copy the codeThis step is relatively simple. Use the Node file module to read the entry content for us
Analyze ast and remove dependencies
First, how do we know from the file contents which modules we refer to?
Parsing the AST tree is used here, which is an abstract representation of the syntactic structure of source code. It represents the syntactic structure of a programming language as a tree, with each node in the tree representing a structure in the source code. (In fact, is it also possible to use the regular 🙋)
As you can see, the value of the source property of the ImportDeclaration object is the dependency we are looking for.
We can compile code to ast with @babel/ Parser
npm i @babel/parser
Copy the codeconst babelParser = require("@babel/parser");
function main() {
let absPath = path.resolve(__dirname, config.entry);
let content = fs.readFileSync(absPath, "utf-8");
console.log(
babelParser.parse(content, {
// Indicates the mode in which code is parsed. By default, script needs to be specified as module module
sourceType: "module }) ); }Copy the codeThis gives the structure of the code in the AST tree:
The second step is to traverse the body of the node and find the ImportDeclaration object mentioned above to get file dependencies. Babel /traverse
/ / installation
npm i @babel/traverse
-------------------------------
const traverse = require("@babel/traverse").default;
function main() {
let entry = config.entry;
let content = fs.readFileSync(entry, "utf-8");
// Rely on storage
let dependecies = []
let ast = babelParser.parse(content, {
sourceType: "module".// Indicates the pattern in which the code should be parsed. Files with ES6 imports and exports are considered "modules", otherwise "scripts".
});
traverse(ast, {
ImportDeclaration: ({ node }) = > {
dependecies.push(node.source.value)
},
});
// --> ['./module-2.js']
console.log(dependecies);
}
Copy the codeThe function collectDependencies returns a dependent object based on the name of the object passed in, adding an incrementing ID attribute.
Function main() {let entry = config.entry; @param {*} filename * @returns */ function collectDependencies(filename) { let code = fs.readFileSync(filename, "utf-8"); let dependecies = []; Let ast = babelparser. parse(code, {sourceType: "module", // indicates the mode in which the code should be parsed. Files with ES6 imports and exports are considered "modules", otherwise "scripts". }); traverse(ast, { ImportDeclaration: ({ node }) => { // console.log(node); dependecies.push(node.source.value); }}); let id = ID++ return { id, filename, dependecies } }Copy the codeBuilding a dependency graph
Currently we have got module1’s dependencies (Module2), so the next thing to do is: Find the dependencies (module2) of the dependencies (Module3), build the dependency graph, use allAsset to represent the dependency graph, traverse allAsset, and push the dependencies found in each loop into allAsset to obtain the complete dependency relationship
- Note that since we import relative paths, the path in dependecies is relative to module-1, which is not in the same directory as our current file, mini-webpack.js, which requires a layer of path conversion.
- Added an attribute mapping to map storage paths to dependent ids, which is useful for generating bundles later
/ / the main function
function main() {
let entry = config.entry;
let mainAsset = collectDependencies(entry);
// Build the dependency graph
let graph = createDependGraph(mainAsset);
}
/** * import dependency *@param {*} mainAsset
* @returns * /
function createDependGraph(mainAsset) {
let allAsset = [mainAsset];
let i = 0;
while (i < allAsset.length) {
let asset = allAsset[i];
let dirname = path.dirname(asset.filename);
asset.mapping = {};
asset.dependecies.forEach((relativePath) = > {
let absPath = path.join(dirname, relativePath);
let childAsset = collectDependencies(absPath);
asset.mapping[relativePath] = childAsset.id;
allAsset.push(childAsset);
});
i++;
}
return allAsset;
}
Copy the codeGet the following output:
The ast es5
Ultimately, our goal is to package multiple files into a single file, so we need to get the contents of the file first. As mentioned earlier, the import in our code needs to be translated by Babel in order to be recognized by the browser. Babel translates ECMAScript 2015+ code into a backward compatible JavaScript syntax by modifying the AST.
Install two toolkits, babel-core and babel-preset-env, and modify the collectDependencies function slightly:
NPM I Babel - core Babel - preset - env -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- lineconst core = require("babel-core");
function collectDependencies(filename) {
let content = fs.readFileSync(filename, "utf-8");
let dependecies = [];
const ast = babaylon.parse(content, {
// Specify the mode in which the code is parsed. The default is' script '. Files with ES6 imports and exports are considered 'modules'.
sourceType: "module"}); traverse(ast, {ImportDeclaration: ({ node }) = >{ dependecies.push(node.source.value); }});// ast --> es5
let { code } = core.transformFromAst(ast, null, {
presets: ["env"]});let id = ID++;
return {
id,
filename,
dependecies,
code,
};
}
Copy the codeSee what is currently returned:
Deja vu, require, Module, exports… Is it similar to the CommonJS modular specification? In fact, the project essentially uses Babel to transcode ES6 to ES5 and then executes it. Import will be transcoded to require.
Packed into bundles
This step is not particularly well understood, but I work backwards from the original WebPack bundle. First consider what the bundle function needs to do:
- First we need to return a string code block
- Blocks of code need to be able to execute automatically
- You need to run the code for each dependent module
As mentioned above, import is transcoded to require, but the browser itself does not provide require, exports, module, so we need a function to provide input arguments, and execute the compiled code as the module corresponding to the function body, and pass in these three arguments externally.
The mapping between the module and the function needs to be constructed, and the module ID above can be used as an identifier.
The code is as follows:
/ / the main function
function main() {
let entry = config.entry;
let mainAsset = collectDependencies(entry);
// Build the dependency graph
let graph = createDependGraph(mainAsset);
/ / output
let res = bundle(graph);
// Output directory
if(! fs.existsSync(config.output.path)) { fs.mkdirSync(config.output.path); }// Output file
let opath = path.join(config.output.path, config.output.filename);
if (fs.existsSync(opath)) {
fs.unlinkSync(opath);
}
let ws = fs.createWriteStream(opath, {
encoding: "utf-8"}); ws.write(res); }function bundle(graph) {
let modules = "";
graph.forEach((module) = > {
// Construct the mapping between modules and this function by iterating over the dependency graph
modules += `
The ${module.id}: [
function(require,module,exports){
The ${module.code}
},
The ${JSON.stringify(module.mapping)}
],
`;
});
let res = '(function(modules){function require(id) {// fn: executes module code to exports // mapping: { Dependency id} mapping let [fn,mapping] = modules[id] Function localRequire(relativePath) {return require(mapping[relativePath])} let module = {exports: {}} / / within the fn modul. Exports to assign the fn (localRequire, the module, the module exports) / / this is the export of the module return module. Exports} / / Id -0 corresponding entry return require(0)})({${modules}})
`;
return res;
}
Copy the codeIn the end, bundle.js was successfully generated in the dist directory, and bundles. Js was introduced into THE HTML to perform the test, and the expected results were output.
Packaging optimization
Uglip-js is introduced to compress package files, and build command is added to facilitate package
npm i uglify-js -g
-------------------------------
// package.json
"scripts": {
"build": "node mini-webpack.js && uglifyjs ./dist/bundle.js -m -o ./dist/bundle.js"
},
Copy the codeYou can see the code compressed into a single line, reduced in size, and hidden variable names, enhancing source security. Webpack also takes into account scenarios such as caching and loading on demand for modules, which is something we can learn from optimizations
Afterword.
This paper introduces the principle of Webpack for module packaging, from 0 to 1 to complete a minimal mini-Webpack, although there are many imperfections in the function, but the focus is on learning how to analyze and solve the problem, in the process of Babel is how to translate our code has a more intuitive understanding. If there is any doubt or wrong place in the code, you are welcome to correct it and make progress together. Please like three times QAQ🔥🔥
Links:
- Let’s talk about AST in Javascript
- What is your import compiled by Webpack?
- Astexplorer online