Webpack series 1: Common loader source code analysis, and hands-on implementation of a MD2HTml-loader
Webpack Series 2: Discover how the WebPack plug-in works
Webpack series 3: Webpack main process source code reading and implementation of a Webpack
This article will give you a brief introduction to the Webpack loader and implement a loader that uses MD to convert into an abstract syntax tree and then into an HTML string. By the way, a simple understanding of several style-loader, VUe-loader, babel-loader source code and workflow.
Introduction of loader
Webpack allows us to process files using loader, a Node module exported as function. The matched files can be converted once, and loader can chain transfer. Loader file handler is a CommonJs-style function that takes a String/Buffer input and returns a String/Buffer return value.
Loader configuration in two forms
Plan 1:
// webpack.config.js
module.exports = {
...
module: {
rules: [{
test: /.vue$/.loader: 'vue-loader'
}, {
test: /.scss$/.// Pass the result to csS-loader, and then enter style-loader.
use: [
'style-loader'.// Create style nodes from the JS string
'css-loader'.// Translate CSS to CommonJS
{
loader: 'sass-loader'.options: {
data: '$color: red; '// Compile Sass to CSS}}]}]}... }Copy the codeMethod 2 (called from right to left)
// module
import Styles from 'style-loader! css-loader? modules! ./styles.css';
Copy the codeWhen chaining multiple Loaders, keep in mind that they will be executed in reverse order. Execute from right to left or bottom up, depending on the array format. The result of the previous loader is passed to the next loader, and the last loader returns the processed result to compiler as a String or Buffer.
Loader configuration can be compiled using loader-utils and verified using schema-utils
import { getOptions } from 'loader-utils';
import { validateOptions } from 'schema-utils';
const schema = {
// ...
}
export default function(content) {
/ / get the options
const options = getOptions(this);
// Check whether loader options are valid
validateOptions(schema, options, 'Demo Loader');
// Write the conversion loader logic here
// ...
return content;
};
Copy the code- Content: indicates the source file string or buffer
- Map: represents the Sourcemap object
- Meta: metadata, auxiliary objects
Synchronous loader
To synchronize the loader, we can return the output via return and this.callback
module.exports = function(content, map, meta) {
// Some synchronization operations
outputContent=someSyncOperation(content)
return outputContent;
}
Copy the codeIf only one result is returned, you can use return to return the result. However, if you need to return something else, such as the sourceMap or AST syntax tree, you can use the API provided by WebPack, this.callback
module.exports = function(content, map, meta) {
this.callback(
err: Error | null.content: string | Buffer, sourceMap? : SourceMap, meta? : any );return;
}
Copy the codeThe first argument must be Error or NULL and the second argument must be a string or Buffer. Optional: The third parameter must be a source map that can be parsed by the module. Optional: The fourth option, which is ignored by Webpack, can be anything. You can use abstract syntax tree-ast (e.g. ESTree) as the fourth parameter (meta). If you want to share a common AST across multiple Loaders, This will help speed up compilation time. .
Asynchronous loader
Asynchronous loader, which uses this.async to get callback functions.
// Make the Loader cache
module.exports = function(source) {
var callback = this.async();
// Do things asynchronously
doSomeAsyncOperation(content, function(err, result) {
if(err) return callback(err);
callback(null, result);
});
};
Copy the codePlease refer to the API on the official website for details
Develop a simple MD-loader
const marked = require("marked");
const loaderUtils = require("loader-utils");
module.exports = function (content) {
this.cacheable && this.cacheable();
const options = loaderUtils.getOptions(this);
try {
marked.setOptions(options);
return marked(content)
} catch (err) {
this.emitError(err);
return null}};Copy the codeThe above example converts content from a Markdown file to an HTML string using an off-the-shelf plug-in, but what would you do without it? In this case, we can consider an alternative solution that uses the AST syntax tree to help us more easily manipulate transformations.
Use AST for source code conversion
Markdown-ast is an abstract syntax tree node that converts the content of a Markdown file into an array. Manipulating the AST syntax tree is much simpler and more convenient than manipulating strings:
const md = require('markdown-ast');// The string is processed into an intuitive AST syntax tree through the regex method
module.exports = function(content) {
this.cacheable && this.cacheable();
const options = loaderUtils.getOptions(this);
try {
console.log(md(content))
const parser = new MdParser(content);
return parser.data
} catch (err) {
console.log(err)
return null}};Copy the codeMd is transformed into abstract language tree by regular cutting
const md = require('markdown-ast');//md converts buffer to abstract syntax tree by regular matching
const hljs = require('highlight.js');// Code highlighting plug-in
// Use AST for source code conversion
class MdParser {
constructor(content) {
this.data = md(content);
console.log(this.data)
this.parse()
}
parse() {
this.data = this.traverse(this.data);
}
traverse(ast) {
console.log("Md to Abstract syntax tree operation",ast)
let body = ' ';
ast.map(item= > {
switch (item.type) {
case "bold":
case "break":
case "codeBlock":
const highlightedCode = hljs.highlight(item.syntax, item.code).value
body += highlightedCode
break;
case "codeSpan":
case "image":
case "italic":
case "link":
case "list":
item.type = (item.bullet === The '-')?'ul' : 'ol'
if(item.type ! = =The '-') {
item.startatt = (` start=${item.indent.length}`)}else {
item.startatt = ' '
}
body += '<' + item.type + item.startatt + '>\n' + this.traverse(item.block) + '< /' + item.type + '>\n'
break;
case "quote":
let quoteString = this.traverse(item.block)
body += '<blockquote>\n' + quoteString + '</blockquote>\n';
break;
case "strike":
case "text":
case "title":
body += `<h${item.rank}>${item.text}</h${item.rank}> `
break;
default:
throw Error("error".`No corresponding treatment when item.type equal${item.type}`); }})return body
}
}
Copy the codeSee here for the complete code
Ast abstract syntax numbers into HTML strings
Loader some development skills
- Try to ensure that one loader does one thing, and then you can combine different scenarios with different loaders
- State should not be preserved in the Loader at development time. Loader must be a pure function without any side effects. Loader supports asynchro, so it is possible to have I/O operations in loader.
- Modularity: Ensure that the Loader is modular. The generated modules of loader must comply with the same design principles as common modules.
- Use caches wisely. Proper caches reduce the cost of duplicate compilation. The cache is enabled by default during loader execution. In this way, webPack directly bypasses the rebuild process when deciding whether to recompile the Loader instance during compilation, saving the cost caused by unnecessary reconstruction. But you can turn caching off if and only if your loader has other unstable external dependencies (such as I/O interface dependencies) :
this.cacheable&&this.cacheable(false);
Copy the codeloader-runnerWebpack is a very useful tool for developing and debugging loaders. It allows you to run a loader independently of webPacknpm install loader-runner --save-dev
/ / create the run - loader. Js
const fs = require("fs");
const path = require("path");
const { runLoaders } = require("loader-runner");
runLoaders(
{
resource: "./readme.md".loaders: [path.resolve(__dirname, "./loaders/md-loader")].readResource: fs.readFile.bind(fs),
},
(err, result) =>
(err ? console.error(err) : console.log(result))
);
Copy the codePerform the node (the loader
Learn more about Loaders
Style-loader source code analysis
Inserts a style into the DOM by inserting a style tag into the HEAD and writing the style into the innerHTML of the tag.
Let’s get rid of the option code so it’s a little clearer
style-loader.js
Module.exports. pitch = function (request) {const {stringifyRequest}=loaderUtils var result = [//1. // Call addStyle to insert CSS content into the DOM (locals is true, default to export CSS). 'var Content =require(' + stringifyRequest(this, '!! '+ request) + ')', 'require(' + request (this, '! '+ path. Join (__dirname, "addstyle. Js")) +') (content), ' 'if(content.locals) module. Exports = content.locals'] return result.join('; ')}Copy the codeIt should be noted that normally we use the default method, and pitch method is used here. Pitch method has an official explanation here on the pitching loader. The simple explanation is that the default loader is executed from right to left and the pitching loader is executed from left to right.
{
test: /\.css$/.use: [{loader: "style-loader" },
{ loader: "css-loader"}}]Copy the codeThe reason for executing style-loader in the first place is that we want to output the CSS -loader as code that can be used in CSS styles instead of strings.
addstyle.js
module.exports = function (content) {
let style = document.createElement("style")
style.innerHTML = content
document.head.appendChild(style)
}
Copy the codeBabel-loader source code brief analysis
First take a look at the configuration processing of skip loader and take a look at the babel-loader output
transpile(source, options)
transpile
const babel = require("babel-core")
module.exports = function (source) {
const babelOptions = {
presets: ['env']}return babel.transform(source, babelOptions).code
}
Copy the codeVue-loader source code analysis
Vue Single file Component (SFC)
<template> <div class="text"> {{a}} </div> </template> <script> export default { data () { return { a: "vue demo" }; }}; </script> <style lang="scss" scope> .text { color: red; } </style>Copy the codeWebpack configuration
const VueloaderPlugin = require('vue-loader/lib/plugin')
module.exports = {
...
module: {
rules: [{...test: /\.vue$/.loader: 'vue-loader'
}
]
}
plugins: [
new VueloaderPlugin()
]
...
}
Copy the codeVueLoaderPlugin is used to copy and apply other rules defined in webpack.config to the corresponding language blocks in the.vue file.
plugin-webpack4.js
const vueLoaderUse = vueUse[vueLoaderUseIndex]
vueLoaderUse.ident = 'vue-loader-options'
vueLoaderUse.options = vueLoaderUse.options || {}
// cloneRule changes the resource and resourceQuery configurations of the original rule,
// File paths with special query will be applied to the corresponding rule
const clonedRules = rules
.filter(r= >r ! == vueRule) .map(cloneRule)// global pitcher (responsible for injecting template compiler loader & CSS
// post loader)
const pitcher = {
loader: require.resolve('./loaders/pitcher'),
resourceQuery: query= > {
const parsed = qs.parse(query.slice(1))
returnparsed.vue ! =null
},
options: {
cacheDirectory: vueLoaderUse.options.cacheDirectory,
cacheIdentifier: vueLoaderUse.options.cacheIdentifier
}
}
// Update the Rules configuration for Webpack so that the clonedRules configuration can be applied to individual labels in vUE single files
compiler.options.module.rules = [
pitcher,
...clonedRules,
...rules
]
Copy the codeGet the rules TAB of webpack.config.js and copy the rules to carry? vue&lang=xx… The same loader configures a common loader for the Vue file: Pitcher sets [pitchLoder,…clonedRules,…rules] as the new rules for webapck.
Look again at the output of the vue-loader result
// Vue-loader uses' @vue/component-compiler-utils' to parse the SFC source code into THE SFC descriptor, and extract the different types of SFC according to the type of module path (the type field on the query parameter) Block.
const { parse } = require('@vue/component-compiler-utils')
// Parse the contents of a Single *. Vue File into a Descriptor object, also known as a single-file Components (SFC) object
// Descriptor contains attributes and content of tags such as template, script, and style to facilitate corresponding processing for each label
const descriptor = parse({
source,
compiler: options.compiler || loadTemplateCompiler(loaderContext),
filename,
sourceRoot,
needMap: sourceMap
})
// Generate a unique hash ID for the single-file component
const id = hash(
isProduction
? (shortFilePath + '\n' + source)
: shortFilePath
)
// If a style tag contains the scoped attribute, CSS scoped processing is required
const hasScoped = descriptor.styles.some(s= > s.scoped)
Copy the codeThe next step is to add the newly generated JS Module to the compilation of webpack, that is, the AST parsing of this JS Module and the collection of related dependencies.
SelectBlock (template/script/style); selectBlock (selectBlock); Get the content of the corresponding type on the Descriptor and pass it to the next loader for processing
import { render, staticRenderFns } from "./App.vue? vue&type=template&id=7ba5bd90&"
import script from "./App.vue? vue&type=script&lang=js&"
export * from "./App.vue? vue&type=script&lang=js&"
import style0 from "./App.vue? vue&type=style&index=0&lang=scss&scope=true&"
Copy the codeSummarize the workflow of VUe-Loader
- registered
VueLoaderPluginIn the plug-in, the rules item in the current project webpack configuration is copied. When the resource path contains Query.lang, the same rules are matched by resourceQuery and a common loader is inserted when the corresponding loader is executed. In the pitch phase, a custom loader is inserted according to query.type - Called when *.vue is loaded
vue-loaderThe.vue file is parsed into onedescriptorObject containingTemplate, script, styles“Attribute corresponds to each tag. For each tag, it will be spliced according to the tag attributesrc? vue&querySRC is the single page component path, query is the parameter of some properties, including lang, type and scoped. If the lang attribute is included, the rules with the same suffix will be matched and the corresponding loaders will be applied to execute the corresponding custom loader according to type.templateWill performtemplateLoader,styleWill performstylePostLoader - in
templateLoaderMedium, will passvue-template-compilerConvert the template to the render function, passing in thescopeIdAppend to each tag and pass it as a vNode configuration propertycreateElemenetMethod, when the render function calls and renders the page, willscopeIdProperties are rendered to the page as raw properties - in
stylePostLoaderPostCSS parses the content of the style tag
The demo source code involved
Click on the Github repository
reference
- Webpack loader API
- Write webpack YamL-loader
- Yan Chuan – Webpack source code analysis series
- Analyze the implementation of CSS Scoped from vue-loader source code