SourceMap
Webpack version 5.28.0
SourceMap is also important in today’s projects because the code after packaging is obfuscated, compressed, and not well positioned. If you want to see the exact location of your code, Source Maps solve this problem by providing a mapping between the original code and the transformed code.
The general sections of this article are as follows:
- Historical origin
- use
sourceMap
sourceMap
Part of the- Webpack source code
sourceMap
(Source perspective) sourceMap
The role of
Historical origin
In a 2009 Google article, When introducing Cloure Compiler (a JS compression optimization tool comparable to Uglip-JS), Google also introduced a debugging tool by the way: Closure Inspector plugin for Firefox to facilitate debugging of compiled code. This is the original sourcemap generation!
You can use the compiler with Closure Inspector , a Firebug extension that makes debugging the obfuscated code almost as easy as debugging the human-readable source.
In 2010, in the second generation, Closure Compiler Source Map 2.0, Sourcemap settled on a unified JSON format and its residual specification, which is almost in its current form. The biggest difference is the mapping algorithm, which is the key to Sourcemap. In the second generation mapping was decided to use base 64 encoding, but the algorithm was different and the resulting.map was much larger than it is now. In 2011, the third generation, Source Map Revision 3.0, was released, which is the current version of Sourcemap. The document’s name suggests that Sourcemap has evolved from Clousre Compiler into a stand-alone tool supported by browsers. Compared with the second generation, the biggest change in this version is the compression replacement of mapping algorithm, which uses VLQ encoding to generate mapping before Base64, greatly reducing the size of. Map files.
The interesting thing about Sourcemap’s history is that it was developed as an accessibility tool. Eventually the object it was helping faded away, and it became the technical body, written into the browser.
Sourcemap V1 initially generated a Sourcemap file about 10 times the size of the converted file. Sourcemap V2 reduced this by 50%, and V3 reduced it by 50% from V2. So the current 133K file corresponds to a Sourcemap file size of around 300K.
Use sourceMap in the browser
How to use sourceMap in your browser? Soruce Map is enabled by default in Chrome. If it is turned off, it can be turned on manually, as shown below:
Mac OS Google Chrome Version 89.0.4389.90 (Official Build) (x86_64)
Create a project
You can create a VUE project using vuetemplates-cli scaffolding, or you can create a project using my own scaffolding vuetemplates-cli:
Install scaffolding globally
$ npm install -g vuetemplates-cli
Initialize the project
$ vuetemplates init template-spa my-project-name
# change directory
$ cd my-project-name
# installation NPM
$ npm install
Copy the code
After installation, modify./build/webpack.prod.js as follows:
{
devtool: 'source-map'
}
Copy the code
Create the souremap_test.js file in the SRC folder and write:
export default {
name: 'I AM CHRIS'
}
Copy the code
Import (‘./ souremap_test.js ‘) from mian.js.
After the modification, run the following command:
The command is packaged to start a port 7087
npm run start
Copy the code
This allows you to see the sourceMap mapped source code on the browser side. If the code runs in error, it can be precisely located to the source code of the error code, rather than the packaged code.
Easy debug code
If you find this template too cumbersome, you can debug it directly with the code in the Debug WebPack configuration.
Debug/SRC /index.js:
'I AM CHRIS'
Copy the code
Just start debugging in vscode.
SourceMap component
Using the simple debugging method above, start vscode debugging or run webpack directly from node./debug/start.js, which will be displayed in debug/dist/main.js.map.
// ./debug/dist/main.js.map
{
"version": 3."sources": [ "webpack://debug/./src/index.js"]."names": []."mappings": ";;;;; AAAA,a"."file": "main.js"."sourcesContent": [ "'I AM CHRIS'"]."sourceRoot": ""
}
Copy the code
SourceMap format
{
"version": 3.// The source map version.
"sources": [].// File before conversion, the item is an array, indicating that there may be multiple file merges.
"names": [].// All variable and attribute names before conversion.
"mappings": "".// A string to record location information.
"file": "".// The converted file name.
"sourcesContent": [""] // Convert the contents of the previous file. This parameter is used when sources is not configured.
}
Copy the code
Focus on the field;;;;; in mappings AAAA,a. To represent a line of position information; To represent a column of location information;
SourceMap implements the mapping
When analyzing the sourceMap mapping, we used a simple example that would be complicated if we added the Babel transformation.
I AM CHRIS -- > Handle transformations -- > CHRIS I AMCopy the code
Those mappings are saved in sourceMap through a series of transformations from I AM CHRIS.
Note that the following is only a theoretical analysis, the real source-map and Webpack do not follow the following part of the analysis.
The simplest and most crude way
Each character position in the output file corresponds to the original position in the input file name and is mapped one by one. The above mapping should result in the following table:
character | Output location | Position in the input | Name of the file entered |
---|---|---|---|
C | Row 1, column 1 | Row 1, column 6 | sourceMap.js |
H | Row 1, column 2 | Row 1, column 7 | sourceMap.js |
F | Row 1, column 3 | Row 1, column 8 | sourceMap.js |
I | Row 1, column 4 | Row 1, column 9 | sourcemap.js |
S | Row 1, column 5 | Line 1, column 10 | sourcemap.js |
I | Row 1, column 7 | Row 1, column 1 | sourcemap.js |
A | Row 1, column 9 | Row 1, column 3 | sourcemap.js |
M | Row 1, column 10 | Line 1, column 4 | sourcemap.js |
Note: Since the input information may come from multiple files, the input file information is also recorded here.
The above table into the mapping table, looks like this (use the symbol “|” character segmentation)
mappings: “| 1 | sourcemap. Js | 1 | 6, 1 | 2 input file 1. TXT 7, 1 | | 1 | 3 | sourcemap. Js | 1 | 8, 1 4 | | sourcemap. Js | 1 | 9, 1 | | 5 sourcemap. Js | 1 | 10, 1 7 | | sourcemap. Js | 1 | 1, 1 | | 9 sourcemap. Js | 1 | 3, 1 | | 10 sourcemap. Js | 1 | 4 “(144) length:
This method does restore the processed content to the pre-processed content, but as the content increases and the transformation rules become more complex, the number of records in the code table grows rapidly. Currently only 10 characters, the mapping table is 144 characters long. How can we further optimize this mapping table?
Note: the mappings: “line output file location input filename | | output file column position line Numbers | | input file input file column number,…”
Optimization method 1: Do not output line numbers in the file
After all the compression and obfuscation, the code is basically not many lines (especially JS, which is usually only 1 or 2 lines). In this case, you can remove the number of lines at the output position from the previous section by using “;” Number to identify the new line. The mapping information then becomes the following
mappings: “1 | sourcemap. Js | 1 | | 6, 2 sourcemap. Js 7, 3 | | 1 | sourcemap. Js 8, 4 | | 1 | sourcemap. Js | 1 | 9, 5 | sourcemap. Js 10, 7 | | 1 | sourcemap. Js | | 1, 9 | so Urcemap. Js | 1 | 3, 10 | sourcemap. Js | 1 | 4. If there is a second line “(Length: 129)
Note: the mappings: “output file input filename column position | | input file line Numbers | input file column number,…”
Note: Mozilla /source-map cannot omit the line number when addMapping otherwise an error will be reported.
Optimization method 2: Extract the input file name
Because there can be multiple input files and the information describing the input file is long, you can store the information of the input file in an array and record only its index value in the array. After this step, the mapping information is as follows:
{
sources: ['sourcemap.js'].mappings: "| | 0 1 6, 2 | | 0 | 1 | 7, 3 1 8, 4 | | | | 0 0 | 1 | 9, 5 | 0 10, 7 | | 1 | 0 | 1 | 1, 9 | 0 | 1 | 3, 10 | | 1 | 0 4;" // (length: 65)
}
Copy the code
The mappings number has decreased from 129 to 65 characters after the transformation. 0 means the value of sources[0].
Note: the mappings: “output file input filename column position | index number line | | input file input file column number,…”
Optimization method 3: the extraction of symbolized characters
After the optimization of the last step, the number of mappings has been greatly reduced, so it can be seen that extracting information is a very useful simplification method. Of course. For example, the CHRIS character in the output file, once we find the position of its first character C in the source file (row 1, column 6), we do not need to find the location of the rest of the HRIS, because CHRIS can be viewed as a whole. Think of the variable names, the function names, in the source code as a whole. You can now extract the characters as a whole and store them in an array, and then just record their index values in the Mapping, just like file names. This avoids the hassle of having to remember every single character and greatly reduces the mappings length.
Add an array containing all the symbolizable characters:
names: ['I', 'AM', 'CHRIS']
So CHRIS was mapping from
| | 0 1 6, 2 | | 0 | 1 | 7, 3 1 8, 4 | | | | 0 0 | 1 | 9, 5 | 0 | 1 | 10
Turned out to be
| | 0 1 6 | | 2
The final mapping information becomes:
{
sources: ['sourcemap.js'].names: ['I'.'AM'.'CHRIS'].mappings: "| | 0 1 6 2, 7 | | | 0 | 1 | 1 | 0, 9 | 0 | 1 | 3 | 1" // (length: 29)
}
Copy the code
Remark:
"I AM CHRIS." "
In the"I"
It doesn’t really make sense to pull it out and put it in an array, because it only has one character. But just for the sake of demonstration, I’ll just pull it out and put it in an array.- The mappings: “output file input filename column position | index number line | | input file input file column number | character index,…”
Note: source-map used in babel-loader does not extract strings, it extracts variables. So ‘I AM CHRIS’ will not split.
Optimization technique 4: Record relative position
Before recording location information (mainly columns), the record is absolute location information, imagine, when the file content is large, these numbers may become very large, how to solve this problem? You can solve this problem by recording only relative positions (except for the first character). Let’s see how this works, using the previous mappings code as an example:
The mappings: "1 (absolute position output columns) | | 1 | 0 6 (absolute position of input columns) | 2, 7 (absolute position output columns) | | 1 | 0 1 (input columns of absolute position) | 0, 9 (absolute position output columns) | 0 | 1 | 3 | 1" (input columns absolute position)
Convert to recording only relative positions
The mappings: "1 (absolute position output columns) | | 1 | 0 6 (absolute position of input columns) | 2, 6 (the relative position of output columns) | | 1 | 0-3 (the relative position of the input columns) | 0, 2 (the relative position of output columns) | | 1 | 0-2 (absolute position of input columns) | 1"
The benefits of this approach may not be obvious from the example above, but as files grow, using relative positions can save a lot of character length, especially for characters that record output file column information.
After recording the relative positions above, we have negative values in our numbers, so we should not be surprised to see negative values after parsing the Source Map file
Another thing I thought about is that for the output position, because it is increasing, the relative position does reduce the number, but for the input position, the effect is not necessarily the same. For map above the final group, the original value is 10 | 0 | 1 | | 0 2, after changing relative to 6 | | 1 | 0-9 | 1. Even if the minus sign is removed from the fourth digit value, because its position in the source file is actually uncertain, the relative value can become very large, the original one digit record, may become two or even three digits. This should be rare, however, because the increase in length is much smaller than the savings from using relative notation for the output position, so overall space is saved
Optimization method 5: VLQ coding
After the above steps, now the most should optimize number should be used to partition Numbers “|”. How should this optimization be implemented? Before answer to see such a question – if you want to order a record of the 4 Numbers, the simplest is to use “|”, segmentation.
1 | 2 | 3 | 4
If each number has only one digit, it can be written as
1234
But a lot of times there’s more than one digit in each number, for example
12 | 3 | | 7 456
In this case, symbols must be used to separate the numbers, as in our example above. Is there a good way? You can handle this situation well by using VLQ encoding. Let’s look at the definition of VLQ:
VLQ definition
A variable-length quantity (VLQ) is a universal code that uses an arbitrary number of binary octets (eight-bit bytes) to represent an arbitrarily large integer.
VLQ is the encoding of any binary byte group to represent an arbitrary number.
There are many different forms of VLQ encoding, and the following ones are explained in this article:
- A group contains six binary bits.
- The first C in each group indicates whether it will be followed by another VLQ byte group. A value of 0 indicates that it is the last VLQ byte group, and a value of 1 indicates that it is followed by another VLQ byte group.
- In the first group, the last 1 bit is used to indicate a sign, with a value of 0 representing a positive number and 1 representing a negative number. The last digit in all the other groups is a number.
- All the other groups are numbers.
This encoding is also called Base64 VLQ encoding because each group corresponds to a Base64 encoding.
A small example illustrates VLQ
Now we use the VLQ encoding rules to encode 12 | 3 | 456 | 7, converts the number to a binary number first.
12-- >1100
3-- >11
456-- >111001000
7-- >111
Copy the code
- I’m going to code 12
12 requires 1 bit for symbol, 1 bit for continuation, and the remaining 4 bits for numbers
B5(C) | B4 | B3 | B2 | B1 | B0 |
---|---|---|---|---|---|
0 | 1 | 1 | 0 | 0 | 0 |
I’m going to code 3
B5(C) | B4 | B3 | B2 | B1 | B0 |
---|---|---|---|---|---|
0 | 0 | 0 | 1 | 1 | 0 |
Code 456
As can be seen from the transformation relationship, the binary number 456 corresponds to has exceeded 6 bits. It is definitely not possible to represent it with 1 group. Here, two groups of bytes are needed to represent it. Remove the last four numbers (1000) and place them in the first byte group, and place the rest in the following byte group.
B5(C) | B4 | B3 | B2 | B1 | B0 | B5(C) | B4 | B3 | B2 | B1 | B0 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 |
I’m going to code 7
B5(C) | B4 | B3 | B2 | B1 | B0 |
---|---|---|---|---|---|
0 | 0 | 1 | 1 | 1 | 0 |
Finally, the following VLQ codes are obtained:
011000 000110 110000 011100 001110
After converting through Base64:
Finally, the following results were obtained:
YGwcO
To verify
Create a file vlq.js and write the following:
// Import VLQ package
const vlq = require('vlq'); The incomingconst string = vlq.encode([12.3.456.7]);
console.log('string: ', string); // YGwcO
Copy the code
Verification completes the theoretical analysis with the same code generated by the example.
The example before the transformation
Through the transformation process the VLQ above the previous example, first to encode | 0 | 1 | | 2. Converted to VLQ:
1-- >1Binary -- >000010(VLQ)
0-- >0Binary -- >000000(VLQ)
1-- >1Binary -- >000010(VLQ)
6-- >110Binary -- >001100(VLQ)
2-- >10Binary -- >000100(VLQ)
Copy the code
After merging, the code is:
000010 000000 000010 001100 000100
Converted into Base64:
BABME
Others are encoded in this way, and the resulting mapping file is as follows:
{
sources: ['sourcemap.js'].names: ['I'.'AM'.'CHRIS'].mappings: "BABME,OABBA,SABGB" // (length: 17)
}
Copy the code
In the webpack sorceMap
The devtool configuration in Webpack controls the generation of sourcemap.map files. Devtool can be broadly divided into the following categories:
The pattern is: [inline-|hidden-|eval-][nosources-][cheap-[module-]]source-map.
- Eval: Used by each module
eval()
Execution, and both//@ sourceURL
. This option builds very quickly. The main disadvantage is that since it maps to the transformed code, not to the original code (there is no mapping fromloader
To derivesource map
), so the number of rows cannot be displayed correctly. - Source-map: Generates a single
source map
File, that is,.map
File. (note withsource map
This general concept distinction) - Being:
source map
No column mapping, ignoredloader source map
. - The module:
loader source map
Simplify to one mapping per line, such as JSX to JS, Babel’s Source map,Added error and warning tracking for third-party libraries. - The inline:
source map
throughData URLs
The way to addbundle
In the. - Hidden: No
bundle
Add a reference comment. - Nosources:
source map
Does not containsourcesContent
(Source code content).
**-hidden-** or **-hidden-** is a very important configuration for online environments to gather stack information without exposing your own source map.
The following uses devtool: source-map as a configuration item to see how soruceMap is generated from the perspective of Webpack source code.
If YOU don’t know where to break points, my breakpoint looks like this:
The main process is as follows:
- runLoaders
- babel-loader
- babel-loader/transfrom
- @babel-core/transfrom
- @babel-core/_transformation.run
- @babel-core/_generate.default
- @babel-core/source-map
How is sourceMap generated in Webpack
We here debugging code or by debugging webpack code in the documentation of the code to do the example. Devtool: ‘source-map’ is configured in webpack.config.js in this instance. To clarify the main points:
- In the webpack
source-map
Is in the runningrunLoader
Theta is generated at theta, which is at thetabable-loader
Generated in thesource-map
. bable-loader
through(mozilla source - map) [https://github.com/mozilla/source-map]
The generated.
Debugging the source code in WebPack can be complicated and tedious, and most of the time it’s pointless because few people want to know about it because it’s enough.
If you want to understand the webPack compilation process, you can read this article. Loader position resolution and loaderContext resolution are also complex, so we will not expand it here. If there is a chance, we will write a loader position resolution later.
Generate your own source-map
Sourcemap.js = sourcemap.js = sourcemap.js = sourcemap.js = sourcemap.js = sourcemap.js
// Import the Mozilla /source-map package
const sourceMap = require('source-map')
// Instantiate SourceMapGenerator
var map = new sourceMap.SourceMapGenerator({
file: 'sourceMap.js.map'
})
// There is a key operation called addMapping to add the mapped column and the original column of the code; In this case, we will directly borrow the _rawMappings generated by babel-loader
/ / the Babel - loader in webpack/node_modules / @ Babel/generator/lib/buffer. Create _rawMappings js file
const RawMapping = [
{
name: undefined.generated: {
line: 1.column: 0
},
source: "sourceMap.js".original: {
line: 1.column: 0}},]// Add the source code to the netmap instance. The first parameter should be the same as the source code
map.setSourceContent("sourceMap.js"."'I AM CHRIS'")
RawMapping.forEach(mapping= > map.addMapping(mapping))
// Prints the sourceMap object
console.log(map.toString())
// {"version":3,"sources":["sourceMap.js"],"names":[],"mappings":"AAAA","file":"sourceMap.js.map","sourcesContent":["'I AM CHRIS'"]}
Copy the code
NPX sourcemap.js (sourceMap); Restore the two “sourcemaps”.
runLoaders
This is done directly from runLoaders, at which point the corresponding loader is called to parse the source file as follows:
Webpack source. / lib/NormalModule. Js
const { getContext, runLoaders } = require("loader-runner");
/ / webpack source code
// ./lib/NormalModule.js
doBuild(options, compilation, resolver, fs, callback) {
// call this.createloadercontext to createLoaderContext
const loaderContext = this.createLoaderContext(
resolver,
options,
compilation,
fs
);
const processResult = (err, result) = > {
/ / to omit
callback(err)
}
// Execute the corresponding hook
try {
hooks.beforeLoaders.call(this.loaders, this, loaderContext);
} catch (err) {
processResult(err);
return;
}
/ / run runLoaders
runLoaders(
{
/ / address entry file '/ Users/admin/Desktop/velen/student/webpack/debug/SRC/index, js'
resource: this.resource,
// babel-loader
loaders: this.loaders,
// loaderContext contains compiler, compilation, file address, and so on
context: loaderContext,
processResource: (loaderContext, resource, callback) = > {
const scheme = getScheme(resource);
if (scheme) {
hooks.readResourceForScheme
.for(scheme)
.callAsync(resource, this.(err, result) = > {
if (err) return callback(err);
if (typeofresult ! = ="string" && !result) {
return callback(new UnhandledSchemeError(scheme, resource));
}
return callback(null, result);
});
} else{ loaderContext.addDependency(resource); fs.readFile(resource, callback); }}},(err, result) = > {
if(! result) {return processResult(
err || new Error("No result from loader-runner processing"),
null
);
}
// omit the code
// Execute the incoming callback functionprocessResult(err, result.result); }); }Copy the code
Call runLoaders and pass in the source code to process, loaders, and context, which will be used in subsequent calls to loader. RunLoaders is another NPM package loader-Runner that I can use to debug while developing my own loader.
RunLoaders will go to node_modules/babel-loader/lib/index.js, execute _loader to configure loaderOptions, and then call transform(source, options) to convert the code. The code is as follows:
babel-laoder => _loader()
node_moduels/babel-loader/lib/index.js
// node_moduels/babel-loader/lib/index.js
// Note that the transform is under babel-loader
const transform = require("./transform");
function _loader() {
_loader = _asyncToGenerator(function* (soruce, inputSourceMap, overrids) {
/ / loaderOptions processing
// omit the code
// Determine the sourceMap parameter passed in
const programmaticOptions = Object.assign({}, loaderOptions, {
// '/Users/admin/Desktop/velen/student/webpack/debug/src/index.js'
filename,
// undefined
inputSourceMap: inputSourceMap || undefined.// Set the default sourcemap behavior based on Webpack's mapping flag,
// but allow users to override if they want.
sourceMaps: loaderOptions.sourceMaps === undefined ? this.sourceMap : loaderOptions.sourceMaps,
// Ensure that Webpack will get a full absolute path in the sourcemap
// so that it can properly map the module back to its internal cached
// modules.
sourceFileName: filename
}); // Remove loader related options
// a series of parameter processing
if (config) {
// Parameter configuration
// Check whether there is a cache
if (cacheDirectory) {
/ / to omit
} else {
// Execute the transform method to pass in the source string and configuration object
result = yieldtransform(source, options); }}// Wait for the above transform asynchronous method to complete execution
if (result) {
if (overrides && overrides.result) {
result = yield overrides.result.call(this, result, {
source,
map: inputSourceMap,
customOptions,
config,
options
});
}
const {
code,
map,
metadata
} = result;
metadataSubscribers.forEach(subscriber= > {
subscribe(subscriber, metadata, this);
});
return[code, map]; }})return _loader.apply(this.arguments);
}
Copy the code
Babel-loader has many asynchronous processes, so it is difficult to sort out the details of the execution process. Here is the main process:
- call
_loader
Method,_loader
There is one pass inside the method_asyncToGenerator
Wrapping method - right
loaderOptions
Perform a series of configurations if inwebpack.config.js
Among them thebabel-loader
If it is configured, it will be merged loaderOptions
After processing is complete, determine if there is a cache, if not a callresult = yield transform(source, options);
.transform
Is an asynchronous, wait for this asynchrony to complete the next operation
Also note that babel-Loader uses a lot of generators to ensure that code is executed asynchronously. If you are interested, check out my other article on front-end generators
_loader() => transform(source, options)
node_moduels/babel-loader/lib/transform.js
// **node_moduels/babel-loader/lib/transform.js**
// Introduce the core package of Babel
const babel = require("@babel/core");
Transform babel.transform to promise via promisify
const transform = promisify(babel.transform);
module.exports = /*#__PURE__*/function () {
var _ref = _asyncToGenerator(function* (source, options) {
let result;
try {
// Call the transform method on Babel to convert the source code to the AST abstract syntax tree
result = yield transform(source, options);
} catch (err) {
throw err.message && err.codeFrame ? new LoaderError(err) : err;
}
if(! result)return null;
// Destruct the result
const {
ast,
code,
map,
metadata,
sourceType
} = result;
if(map && (! map.sourcesContent || ! map.sourcesContent.length)) { map.sourcesContent = [source]; }// Returns the deconstructed value
return {
ast,
code,
map,
metadata,
sourceType
};
});
return function (_x, _x2) {
return _ref.apply(this.arguments); }; } ();Copy the code
The transform method was hijacked via Object.defineProperty in @babel/core/lib/index.js. In the execution promisify (Babel. The transform); _transform.transform; In the transfrom file is an IIFE(self-executing function), and the transform() internally executes as follows:
- The introduction of
@babel/core
Bag, and putpromisify(babel.transform);
convertpromise
Function of type module.exports
Export aIIFE
(self-executing function), which defines another function inside_ref
Is a_asyncToGenerator
Step method_ref
Call @babel/transform directly insidetransform(source, options)
@babel/core/lib/transform => transform(source, options)
The transform method of @babel/core handles the configuration parameters first, and then calls _transformRunner. Run to start the actual transformation. The code is as follows:
// node_module/@babel/core/lib/transform.js
// Import config
var _config = _interopRequireDefault(require("./config"));
var _transformation = require("./transformation");
const gensync = require("gensync");
const transformRunner = gensync(function* transform(code, opts) {
// Process opts with _config.default
const config = yield* (0, _config.default)(opts);
// Return null if config is null
if (config === null) return null;
// execute _transformation. Run and pass in config, code
return yield* (0, _transformation.run)(config, code);
});
// Define the transform method to pass in code, opts, callback
const transform = function transform(code, opts, callback) {
if (typeof opts === "function") {
callback = opts;
opts = undefined;
}
if (callback === undefined) return transformRunner.sync(code, opts);
/ / call transformRunner
transformRunner.errback(code, opts, callback);
};
Copy the code
@babel/core/lib/transformation/index.js
Transformation /index.js generates the AST with @babel/parser, optimizes the AST with @babel/traverse, and finally calls _generate to generate code. The simplified code is as follows:
// @babel/core/lib/transformation/index.js
// Export the run method by default
exports.run = run;
// Load traverse to generate AST for traversal maintenance + optimization
function _traverse() {
const data = _interopRequireDefault(require("@babel/traverse"));
_traverse = function () {
return data;
};
return data;
}
// Convert code to AST with @babel/parser in _normalizefile. default
var _normalizeFile = _interopRequireDefault(require("./normalize-file"));
// Maintain updated AST for code generation with @babel/traverse
var _generate = _interopRequireDefault(require("./file/generate"));
// config is undefined for the config ast created in transform
function* run(config, code, ast) {
// _normalizefile. default Internally calls @babel/parser to generate an AST for code and return the generated AST and configuration object
// File contains the following:
/ / {
// ast: {}:node,
// code: String,
// hub: { file: this, getCode: function, getScope: function }:Object,
// inputMap: null,
// path: {}:NodePath,
// scope: {}:Scope
// }
const file = yield* (0, _normalizeFile.default)(config.passes, (0, _normalizeOpts.default)(config), code, ast);
const opts = file.opts;
try {
// Execute transformFile to pass in the file object and configuration items
yield* transformFile(file, config.passes);
} catch (e) {
throw e;
}
try {
if(opts.code ! = =false) {
({
outputCode,
outputMap // Call the _generate method for code generation
} = (0, _generate.default)(config.passes, file)); }}catch (e) {
throw e;
}
// Return the object
return {
metadata: file.metadata,
options: opts,
ast: opts.ast === true ? file.ast : null.code: outputCode === undefined ? null : outputCode,
map: outputMap === undefined ? null : outputMap,
sourceType: file.ast.program.sourceType
};
}
// Create the transformFile method, passing in the AST object converted from parser
function* transformFile(file, pluginPasses) {
const passPairs = [];
const passes = [];
const visitors = [];
for (const plugin of pluginPairs.concat([(0, _blockHoistPlugin.default)()])) {
const pass = new _pluginPass.default(file, plugin.key, plugin.options);
passPairs.push([plugin, pass]);
passes.push(pass);
visitors.push(plugin.visitor);
}
/ / create a visitor
const visitor = _traverse().default.visitors.merge(visitors, passes, file.opts.wrapPluginVisitorMethod);
// Call _traverse handler AST incoming
(0, _traverse().default)(file.ast, visitor, file.scope);
}
Copy the code
I won’t go through @babel/ Parser and @babel/traverse here. If you’re interested, read Babel
Perform the following steps:
- in
run
Method is executed_normalizeFile.default
Before they call(0, _normalizeOpts.default)(config)
To process configuration items, returnoptions
. (0, _normalizeFile.default)(config.passes, options, code, ast)
Start putting the incomingcode
through@babel/parser
generateAST
- The returned
file
Object after executiontransformFile(file, config.passes)
. - in
transformFile
First of all, it generatesVisitor (Visitor)
Object, and then execute(0, _traverse().default)(file.ast, visitor, file.scope);
The incomingAST
,visitor
,scope
. Will iterate over the updatesAST
The upper node. - The following is executed
(0, _generate.default)(config.passes, file))
Used to generate post-translationalEs2015 code
andsourceMap
@babel/core/lib/transformation/file/generate.js
_generate.default is the generateCode method:
// @babel/core/lib/transformation/file/generate.js
exports.default = generateCode;
function _generator() {
/ / the introduction of the generator
const data = _interopRequireDefault(require("@babel/generator"));
_generator = function () {
return data;
};
return data;
}
// Encapsulate the generateCode method
function generateCode(pluginPasses, file) {
// code is now 'I AM CHRIS'
const { opts, ast, code, inputMap } = file;
const results = [];
if (results.length === 0) {
// Call the method defined above to instantiate the SourceMap object and generate RwaMapping
// Return result in _print
result = (0, _generator().default)(ast, opts.generatorOpts, code);
}
// Destruct the result object returned by the _generator method;
// Fetching the map value triggers the internally bound hijacking method
let { code: outputCode, map: outputMap } = result;
return { outputCode, outputMap };
}
Copy the code
A source-map instance is generated internally by _generator, and row and column information is added to _rawMapping, which returns a Result object. Result. map is a monitored property, and get() in source-map is performed by destructing the map. AddMapping generates the sourceMap object we want by passing _rawMapping to map.addMapping.
@babel/generate/lib/index.js
“@ Babel/generator”, pointing to the file is @ the Babel/generate/lib/index, js, look at the below:
// @babel/generate/lib/index.js
exports.default = generate;
// Import the source-map class
var _sourceMap = _interopRequireDefault(require("./source-map"));
// Import printer class
var _printer = _interopRequireDefault(require("./printer"));
// Generator inherits the _printer class
class Generator extends _printer.default {
// Initialize the constructor
constructor(ast, opts = {}, code) {
// Handle default arguments
const format = normalizeOptions(code, opts);
// instantiate _sourceMap
const map = opts.sourceMaps ? new _sourceMap.default(opts, code) : null;
// execute the constructor of _printer class
// Saves the current map object in the _printer of the instance
super(format, map);
this.ast = void 0;
this.ast = ast;
}
generate() {
// Generate RawMapping is called on printer
return super.generate(this.ast); }}// Process parameters
function normalizeOptions(code, opts) {
/ /... Omit code
}
// Create the generate method
function generate(ast, opts, code) {
// Instantiate the Generator class
const gen = new Generator(ast, opts, code);
// Call generate method on gen instance to start generating code
return gen.generate();
}
Copy the code
Execution steps:
- Call defined
generate
Method, instantiateGenerator
class - instantiation
Generator
Class will passnormalizeOptions
Parameter processing - call
_printer.generate(this.ast);
To carry outRawMapping
I’m not going to expand it here _printer.generate
Will call againbuffer.get
, return a hijackresult
objectbuffer
generateRawMapping
Object,buffer
Will be calledsourceMap.mark
And fromThe map (sourceMap instances). _rawMappings
Add the parsed row and column code- Returns after a series of operations
result = {code, map, rawMappings}
Object,result
The object’smap
The attribute is throughObject.defineProperty
Hijack
After returning to the result object here and will perform to the @ Babel/core/lib/transformation/file/generate js file let {code: outputCode, map: outputMap} = the result; Methods.
RawMapping contains multiple mapping, and each mapping contains the following fields:
{
// Names will be merged into the generated.map
name: ' '.generated: {
line,// Generate the line position of the code
column// Generate the column position of the code
},
source, // File location
origin: {
line,// Line position of source code
column// Column position of source code}}Copy the code
First look at the @ Babel/generate/lib/source – map. In the js code:
// @babel/generate/lib/source-map.js
var _sourceMap = _interopRequireDefault(require("source-map"));
class SourceMap {
constructor(opts, code) {
// A list of attribute assignments
this._cachedMap = void 0;
this._code = void 0;
this._opts = void 0;
this._rawMappings = void 0;
this._lastGenLine = void 0;
this._lastSourceLine = void 0;
this._lastSourceColumn = void 0;
this._cachedMap = null;
this._code = code;
this._opts = opts;
this._rawMappings = [];
}
/ / to hijack the get ()
get() {
if (!this._cachedMap) {
// Set sourceRoot in source-map
const map = this._cachedMap = new _sourceMap.default.SourceMapGenerator({
sourceRoot: this._opts.sourceRoot
});
const code = this._code;
// Set SourceContent to string
if (typeof code === "string") {
map.setSourceContent(this._opts.sourceFileName.replace(/\\/g."/"), code);
else if (typeof code === "object") {
// If SourceContent is set for object loop
}
// Loop through the _rawMappings array created in buffer and add it to the map instance using addMapping
this._rawMappings.forEach(mapping= >map.addMapping(mapping), map); }}// Return the _rawMappings data copy of the instance
getRawMappings() {
return this._rawMappings.slice();
}
// Will be called in buffer to add the escaped row information to the _rawMappings array
mark(generatedLine, generatedColumn, line, column, identifierName, filename, force) {
this._rawMappings.push({
name: identifierName || undefined.generated: {
line: generatedLine,
column: generatedColumn
},
source: line == null ? undefined : (filename || this._opts.sourceFileName).replace(/\\/g."/"),
original: line == null ? undefined : {
line: line,
column: column } }); }}Copy the code
Source-map in Babel is implemented based on Mozilla/source-Map. Babel is mainly responsible for generating line and column numbers of code.
@babel/core/lib/transformation/file/generate.js
Let {code: outputCode, map: outputMap} = result; , the map object in result obtained by deconstruction will go to buffer.js class to execute map.get(), and then execute the get() method in source-map.js. After the map.addmapping method is executed, the creation of the map is completed. Using _cachedMap.JSON returns the JSON object for the current sourceMap.
Webpack output sourceMap
The sourceMap file is generated by babel-loader. This is because //# sourceMappingURL=main.js.map corresponds to the sourceMap path to the generated chunk.js.
This code breakpoint is as follows:
Complete output through two plug-ins SourceMapDevToolPlugin and EvalSourceMapDevToolPlugin according to different devtool configuration items, different plug-ins to load. Directly on the code:
// ./lib/webpack.js
const WebpackOptionsApply = require("./WebpackOptionsApply");
const createCompiler = rawOptions= > {
const options = getNormalizedWebpackOptions(rawOptions);
applyWebpackOptionsBaseDefaults(options);
const compiler = new Compiler(options.context);
compiler.options = options;
new NodeEnvironmentPlugin({
infrastructureLogging: options.infrastructureLogging
}).apply(compiler);
if (Array.isArray(options.plugins)) {
for (const plugin of options.plugins) {
if (typeof plugin === "function") {
plugin.call(compiler, compiler);
} else {
plugin.apply(compiler);
}
}
}
applyWebpackOptionsDefaults(options);
compiler.hooks.environment.call();
compiler.hooks.afterEnvironment.call();
// Load the webpack.config.js loader and so on
new WebpackOptionsApply().process(options, compiler);
compiler.hooks.initialize.call();
return compiler;
};
Copy the code
Since this is covered in another article, see the Webapck compilation process for more information. Without further details, look directly at the webPackage OptionsApply plug-in that loads sourceMap.
// ./lib/WebpackOptionsApply.js
class WebpackOptionsApply extends OptionsApply {
constructor() {
super(a); }process(options, compiler) {
// Check whether devtool is configured in webpack.config.js
if (options.devtool) {
// Determine if the devtool field contains the source-map string
if (options.devtool.includes("source-map")) {
const hidden = options.devtool.includes("hidden");
const inline = options.devtool.includes("inline");
// Whether the eval string is included
const evalWrapped = options.devtool.includes("eval");
const cheap = options.devtool.includes("cheap");
const moduleMaps = options.devtool.includes("module");
const noSources = options.devtool.includes("nosources");
// Load different fields according to the evalWrapped field
const Plugin = evalWrapped
? require("./EvalSourceMapDevToolPlugin")
: require("./SourceMapDevToolPlugin");
// Initialize the loaded plug-in; The Compiler object is passed in
new Plugin({
filename: inline ? null : options.output.sourceMapFilename,
moduleFilenameTemplate: options.output.devtoolModuleFilenameTemplate,
fallbackModuleFilenameTemplate:
options.output.devtoolFallbackModuleFilenameTemplate,
append: hidden ? false : undefined.module: moduleMaps ? true : cheap ? false : true.columns: cheap ? false : true.noSources: noSources,
namespace: options.output.devtoolNamespace
}).apply(compiler);
} else if (options.devtool.includes("eval")) {
const EvalDevToolModulePlugin = require("./EvalDevToolModulePlugin");
new EvalDevToolModulePlugin({
moduleFilenameTemplate: options.output.devtoolModuleFilenameTemplate,
namespace: options.output.devtoolNamespace }).apply(compiler); }}}Copy the code
Process (options, compiler) to load different plugins in WebpackOptionsApply(). Process (options, compiler) The ‘source-map’ configuration will then generate different code based on eval, inline, and source-map comparisons.
SourceMapDevToolPlugin implementation
Process of instantiating the SourceMapDevToolPlugin in WebpackOptionsApply., see below SourceMapDevToolPlugin what specific to sit.
// ./lib/SourceMapDevToolPlugin.js
class SourceMapDevToolPlugin {
constructor(options = {}) {
validate(schema, options, {
name: "SourceMap DevTool Plugin".baseDataPath: "options"
});
// Process the options passed in
this.sourceMappingURLComment =
options.append === false
? false
: options.append || "\n//# source" + "MappingURL=[url]";
/ * *@type {string | Function} * /
this.moduleFilenameTemplate =
options.moduleFilenameTemplate || "webpack://[namespace]/[resourcePath]";
/ * *@type {string | Function} * /
this.fallbackModuleFilenameTemplate =
options.fallbackModuleFilenameTemplate ||
"webpack://[namespace]/[resourcePath]? [hash]";
/ * *@type {string} * /
this.namespace = options.namespace || "";
/ * *@type {SourceMapDevToolPluginOptions} * /
this.options = options;
}
apply (compiler) {
// Handle necessary configuration fields such as sourceMapFilename and sourceMappingURLComment
const outputFs = compiler.outputFileSystem;
const sourceMapFilename = this.sourceMapFilename;
const sourceMappingURLComment = this.sourceMappingURLComment;
const moduleFilenameTemplate = this.moduleFilenameTemplate;
const namespace = this.namespace;
const fallbackModuleFilenameTemplate = this.fallbackModuleFilenameTemplate;
const requestShortener = compiler.requestShortener;
const options = this.options;
options.test = options.test || /\.(m? js|css)($|\?) /i;
compiler.hooks.compilation.tap("SourceMapDevToolPlugin".compilation= > {
/ / instantiate SourceMapDevToolModuleOptionsPlugin will bind buildModule, runtimeModule hook
new SourceMapDevToolModuleOptionsPlugin(options).apply(compilation);
compilation.hooks.processAssets.tapAsync(
{
name: "SourceMapDevToolPlugin".stage: Compilation.PROCESS_ASSETS_STAGE_DEV_TOOLING,
additionalAssets: true
},
(assets, callback) = > {
asyncLib.each(
files,
(file, callback) = > {
// Whether to generate the source-map path to the file
}, err= > {
if (err) {
return callback(err);
}
// Go through a series of processes
const chunkGraph = compilation.chunkGraph;
const cache = compilation.getCache("SourceMapDevToolPlugin");
// Process the files field
const tasks = [];
asyncLib.each(
files,
(file, callback) = > {
/ / processing cache
// It creates a task for each object file, as appropriate, and adds the sourceMappingURL to the end of the file that created the task.
const task = getTaskForFile(
file,
asset.source,
asset.info,
{
module: options.module,
columns: options.columns
},
compilation,
cacheItem
);
if (task) {
Are not present in the modules / / cycle moduleToSourceNameMapping to moduleToSourceNameMapping added
for (let idx = 0; idx < modules.length; idx++) {
const module = modules[idx];
if(! moduleToSourceNameMapping.get(module)) {
moduleToSourceNameMapping.set(
module,
ModuleFilenameHelpers.createFilename(
module,
{
moduleFilenameTemplate: moduleFilenameTemplate,
namespace: namespace }, { requestShortener, chunkGraph } ) ); }}// Task added to taskstasks.push(task); }},err= > {
if (err) {
return callback(err);
}
// Splice the map path and name
asyncLib.each(
tasks,
(task, callback) = > {
# sourceMappingURL=[url]' //# sourceMappingURL=[url]'
let currentSourceMappingURLComment = sourceMappingURLComment;
let asset = new RawSource(source);
// Handle sourceMap configurations, such as hashes, and so on
if(currentSourceMappingURLComment ! = =false) {
/ / the currentSourceMappingURLComment added to the compilation of the asset
// Add source map url to compilation asset, if currentSourceMappingURLComment is set
// instantiate ConcatSource to write source-map path to source code
asset = new ConcatSource(
asset,
compilation.getPath(
currentSourceMappingURLComment,
Object.assign({ url: sourceMapUrl }, pathParams)
)
);
}
// Update the compilation asset
compilation.updateAsset(file, asset, assetInfo);
// Output file
compilation.emitAsset(
sourceMapFile,
sourceMapAsset,
sourceMapAssetInfo
);
}, err= > {
reportProgress(1.0); callback(err); })})})})})})})}}}Copy the code
Using devtool: ‘source-map’ as an example, the SourceMapDevToolPlugin executes as follows:
- instantiation
SourceMapDevToolModuleOptionsPlugin
Plugins will be incompiler.hooks.compilation
To bind the callback function - And binding
compilation.hooks.processAssets
The hook’s asynchronous callback function;compilation.hooks.processAssets
The execution time is at
Compilation. CreateChunkAssets execution is completed, namely chunkAsses generated after.
- See if it generated
source-map
, if generated createtask
And add totasks
In, the subsequent looptasks
datasource-map
Path creation for - through
ConcatSource
theRawSource
andcurrentSourceMappingURLComment
Merge and pass againcompilation.updateAsset
Update correspondingassets
Object.
conclusion
At this point runLoaders() returns the source code compiled by babel-Loader and the sourceMap object, Behind by binding SourceMapDevToolPlugin compilation. Hooks. ProcessAssets hook callback function, the corresponding source – the path of the map to add into the corresponding the chunk source, the follow-up is the corresponding calls to emit process.
Other sourceMap related
In webpack. Config. Also can be directly in the plugin configuration in js webpack. SourceMapDevToolPlugin to specify sourceMap – url generation rules.
Note: devtool and webpack SourceMapDevToolPlugin don’t used at the same time
Optimization. minimizer can also configure whether sourceMap: false is generated.
In previous versions of Webpack 4.x, the uglifyjs-webpack-plugin was used to compress the code, but it is not referenced in webPack 5.x.
Different configuration of devtool in Webpack
Webpack includes eval, source-Map, cheap, Module, Inline, Hidden, and Nosources. Compare the generated sourceMap memory.
Devtool: “source – map” configuration
Generate a separate *.map file to store the mapping path, rows and columns, and source code.
mian.js
// Size is 213btyes
/ * * * * * * / (() = > { // webpackBootstrap
var __webpack_exports__ = {};
/ *! * * * * * * * * * * * * * * * * * * * * * *! * \! *** ./src/index.js ***! \ * * * * * * * * * * * * * * * * * * * * * * /
"I AM CHRIS";
/ * * * * * * / })()
;
//# sourceMappingURL=main.js.map
Copy the code
main.js.map
// Size 163btyes
{"version":3."sources": ["webpack://debug/./src/index.js"]."names": []."mappings":";;;;; AAAA,a"."file":"main.js"."sourcesContent": ["\"I AM CHRIS\""]."sourceRoot":""}
Copy the code
Devtool: “eval” configuration
mian.js
// The size is 1KB
/ * * * * * * / (() = > { // webpackBootstrap
/ * * * * * * / var __webpack_modules__ = ({
/ * * * / "./src/index.js":
/ *! * * * * * * * * * * * * * * * * * * * * * *! * \! *** ./src/index.js ***! \ * * * * * * * * * * * * * * * * * * * * * * /
/ * * * / (() = > {
eval("\"I AM CHRIS\"; \n\n//# sourceURL=webpack://debug/./src/index.js?");
/ * * * / })
/ * * * * * * / });
/ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * /
/ * * * * * * /
/ * * * * * * / // startup
/ * * * * * * / // Load entry module and return exports
/ * * * * * * / // This entry module can't be inlined because the eval devtool is used.
/ * * * * * * / var __webpack_exports__ = {};
/ * * * * * * / __webpack_modules__["./src/index.js"] ();/ * * * * * * /
/ * * * * * * / })()
;
Copy the code
Devtool: “cheap-source-map” or “cheap-module-source-map” configuration
cheap-source-map
: Generates independent.map
File, but no column mapping (column mapping
)source map
To ignoreloader source map
.cheap-module-source-map
: No column mapping (column mapping
)source map
That will beloader source map
Simplifies to one mapping per row (mapping
).
mian.js
// Size is 213btyes
/ * * * * * * / (() = > { // webpackBootstrap
var __webpack_exports__ = {};
/ *! * * * * * * * * * * * * * * * * * * * * * *! * \! *** ./src/index.js ***! \ * * * * * * * * * * * * * * * * * * * * * * /
"I AM CHRIS";
/ * * * * * * / })()
;
//# sourceMappingURL=main.js.map
Copy the code
main.js.map
// Size is 154btyes
{"version":3."file":"main.js"."sources": ["webpack://debug/./src/index.js"]."sourcesContent": ["\"I AM CHRIS\";"]."mappings":";;;;; AAAA;; A"."sourceRoot":""}
Copy the code
Devtool: “the inline – source – map” configuration
The complete Source Map object that is added to the bundle after the Source map is converted to the DataUrl.
mian.js
// The size is 465btyes
/ * * * * * * / (() = > { // webpackBootstrap
var __webpack_exports__ = {};
/ *! * * * * * * * * * * * * * * * * * * * * * *! * \! *** ./src/index.js ***! \ * * * * * * * * * * * * * * * * * * * * * * /
"I AM CHRIS";
/ * * * * * * / })()
;
//# sourceMappingURL=data:application/json; charset=utf-8; base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbIndlYnBhY2s6Ly9kZWJ1Zy8uL3NyYy9pbmRleC5qcyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7O ztBQUFBLGEiLCJmaWxlIjoibWFpbi5qcyIsInNvdXJjZXNDb250ZW50IjpbIlwiSSBBTSBDSFJJU1wiIl0sInNvdXJjZVJvb3QiOiIifQ==
Copy the code
Devtool: “hidden – source – map” configuration
This differs from source-map in that there is no reference to the source map path in mian.js.
mian.js
// The size is 180btyes
/ * * * * * * / (() = > { // webpackBootstrap
var __webpack_exports__ = {};
/ *! * * * * * * * * * * * * * * * * * * * * * *! * \! *** ./src/index.js ***! \ * * * * * * * * * * * * * * * * * * * * * * /
"I AM CHRIS";
/ * * * * * * / })()
;
Copy the code
Devtool: “nosources – source – map” configuration
The sourcesContent field is not added when the source map is created.
main.js.map
// size is 127btyes
{"version":3."sources": ["webpack://debug/./src/index.js"]."names": []."mappings":";;;;; AAAA,a"."file":"main.js"."sourceRoot":""}
Copy the code
other
You can set devtool to “cheap-module-source-map” during debugging, but you cannot upload the generated *. Map file to the server when publishing to an online environment, otherwise someone will decomcompile your code.
Using Fundebug
If fundebug is used, refer to the fundebug sourceMap documentation
Use the sentry
If you use Sentry, refer to the Sentry sourceMap documentation
Reverse parse source-map
If you just want to decompile a simple *.map into source, you can use reverse-sourcemap, but the library has long been out of service.
# installation reverse - sourcemap
npm install -g reverse-sourcemap
Run the decompiler command
reverse-sourcemap -v ./debug/dist/mian.js.map -o sourcecode
Copy the code
It basically generates complete source code, but does not include third-party packages.
Implementing source location
// Get file content
const sourceMap = require('source-map');
const readFile = function (filePath) {
return new Promise(function (resolve, reject) {
fs.readFile(filePath, {encoding:'utf-8'}, function(error, data) {
if (error) {
console.log(error)
return reject(error);
}
resolve(JSON.parse(data));
});
});
};
// Find the source location
async function searchSource(filePath, line, column) {
const rawSourceMap = await readFile(filePath)
const consumer = await new sourceMap.SourceMapConsumer(rawSourceMap);
const res = consumer.originalPositionFor({
'line' : line,
'column' : column
});
consumer.destroy()
return res
}
Copy the code
Source-map is used to locate source code. The code here is just a reference to someone else’s code. A custom error listener library will be wrapped in a later version.