Component library webpack build speed optimization
background
My main work in the company is the development of component library (UI component library based on VUE, similar to Element-UI). I have been feeling that the development and construction of the project is too slow for more than two months. It takes about 40s to open the development environment for each development, which is unbearable. A variety of optimization methods have been tried, but they are not ideal. Finally today, I found the problem, the construction speed increased by more than 50%, now only needs about 17s, the whole mood is good. Now note the various optimizations used, since this is a development environment, only build speed is considered.
Optimization of various configuration items
Add some minor optimizations to some loaders such as include exclude. This does not improve performance.
The introduction of happypack
Happypack uses multithreading to speed up projects. Yeah, I think that’s a good one. Check out the documentation on Github and test the waters.
Modify webpack some loader configuration to use happypack
// config.dev.js
{
// ...
module: {
rules: [{
test: /\.vue$/.loader: 'vue-loader'.options: {
css: 'style-loader! css-loader! sass-loader'.// There is almost no CSS code in vue files, so we just give js to happypack
js: 'happypack/loader? id=babel'}}, {test: /\.js$/.use: 'happypack/loader? id=babel'.exclude: /node_modules/.The // components directory is for components, the examples directory is mainly for Markdown documents, and the test directory is for unit tests
include: [utils.resolve('./components'), utils.resolve('./examples'), utils.resolve('./test')]}, {test: /\.scss$/.use: 'happypack/loader? id=scss'}},plugins: [
new HappyPack({
id: 'babel'.threads: 4.loaders: ['babel-loader']}),new HappyPack({
id: 'scss'.threads: 4.loaders: [
'style-loader'.'css-loader',
{
loader: 'postcss-loader'.options: {
config: {
path: utils.resolve('./postcss.config.js')}}},'sass-loader'"})"// ...
}
Copy the codeHappypack is used to process all kinds of files in the component library. In addition to js SCSS VUE, md (vue-markdown-loader) is also used to process. The configuration is similar, so it will not be listed.
Ok, configuration complete, run and test the waters. The result is an error… oh no! Vue-markdown-loader is not supported. All right, change the PROCESSING of the MD file back and run. Well, it worked this time, but the time was about 4s-5s, emMMmm, not as much as expected.
Add –progress to the command you run, and you can see that the main time is to process the MD file. Obviously, the progress bar slows down when it comes to the MD file.
Ok, the main target of optimization should be the processing of MD files.
Find some processing in build/util.js
function render(tokens, idx) {
// Tokens are the result of Markdown -it Parse
var m = tokens[idx].info.trim().match(/^demo\s*(.*)$/);
if (tokens[idx].nesting === 1) {
let index = idx + 1;
var html = ' ';
var style = ' ';
var script = ' ';
while (tokens[index].nesting === 0) {
const content = tokens[index].content;
const tag = tokens[index].info;
if (tag === 'html') {
html = convert(striptags.strip(content, ['script'.'style'])).replace(
/ (< [^ >] *) = "" (? =.*>)/g.'$1'
);
script = striptags.fetch(content, 'script');
style = striptags.fetch(content, 'style');
} else if (tag === 'js' && !script) {
script = striptags.fetch(content, 'script');
} else if(['css'.'style'.'scss'].indexOf(tag) ! = =- 1 &&
!style
) {
style = striptags.fetch(content, 'style');
}
index++;
}
var description = m && m.length > 1 ? m[1] : ' ';
var jsfiddle = { html: html, script: script, style: style };
var descriptionHTML = description ? md.render(description) : ' ';
jsfiddle = md.utils.escapeHtml(JSON.stringify(jsfiddle));
return `
<demo-block class="demo-box" :jsfiddle="${jsfiddle}">
<div class="source" slot="source">${html}</div>
${descriptionHTML}
<div class="hljs highlight" slot="highlight">
`;
}
return '</div></demo-block>\n';
}
Copy the codeBasically, put the tip into the specified container. Extract tokens some HTML js CSS code that forms an object called JSFiddle and passes it to a VUE component that provides jSbin online debugging. Using the Render method of Markdown-it, render other documents written in markdown syntax into HTML code in the specified div, and slot the HTML code (actually the sample code in the document) into the vUE component mentioned above.
There’s really no way to optimize it.
The introduction of a DLL
Another attempt is to use Webpack’s DllPlugin and DllReferencePlugin to introduce DLLS, so that some basically unchanged code is packaged into static resources, so that WebPack will deal with less things
Package the configuration of DLLS
// config.dll.js
module.exports = merge(base, {
// ...
entry: {
vendor: ['vue'.'vue-router'.'vue-i18n'.'clipboard']},output: {
path: path.resolve(__dirname, './dll'),
filename: '[name].js'.library: '[name]_[hash]'
},
plugins: [
new webpack.DllPlugin({
name: '[name]_[hash]'.path: path.resolve(__dirname, './dll/vendor.manifest.json')})]// ...
})
Copy the codeThe package configuration above generates DLL directories in the build directory containing vendor.dll.js and vendor.manifest.json
Then add the DllReferencePlugin to config.dev.js and you’re done
DllReferencePlugin configuration
{
plugins: [
new webpack.DllReferencePlugin({
manifest: require('./dll/vendor.manifest.json')]}})Copy the codeIn this way, when webpack handles vue vue-Router vue-i18n clipboard in the project, it doesn’t go to node_modules and uses Vendor.js directly
When I run NPM run dev again, I find that the time is only 1s less. After all, Big Head isn’t here.
Single component development mode
Then it suddenly occurred to me that every time I developed a component, it was not a single one. In this case, I only dealt with the MD file of the specified component, so the speed didn’t get up.
Well, that might be a way. Test the waters
Find examples/route.js where to import the MD file
function loadDocs(path) {
return r= > require.ensure([],
() => r(require(`./docs${path}.md`))); }Copy the codeThis is vue-router dynamic loading, as long as I write path as a fixed path (in this case, it is ‘/’ + component name).
The run command looks something like this
// package.json
{
"scripts": {
"dev:component": "cross-env RUN_ENV=component node build/dev-server.js",
}
}
Copy the codeSince there is no way to pass parameters to process.argv using webpack-dev-server on the command line, webpack-hot-middleware is used
// build/dev-server.js
const webpack = require('webpack');
const webpackConfig = require('./config.dev');
const express = require('express');
webpackConfig.plugins = webpackConfig.plugins || [];
// Webpack-dev-server does not need to be configured with HMR
webpackConfig.plugins.push(new webpack.HotModuleReplacementPlugin());
webpackConfig.entry.push('webpack-hot-middleware/client? path=/__webpack_hmr&timeout=20000');
const compiler = webpack(webpackConfig);
const hotMiddleware = require('webpack-hot-middleware')(compiler, {
log: false
});
const devMiddleware = require('webpack-dev-middleware')(compiler, {
publicPath: webpackConfig.output.publicPath,
quiet: true.logLevel: 'silent'
});
const app = express();
app.use(hotMiddleware);
app.use(devMiddleware);
app.use('/build', express.static('./build'));
app.listen(webpackConfig.devServer.port || 8089.'127.0.0.1', () = > {console.log('Starting server on http://localhost:8089');
});
Copy the codeThen use WebPack’s DefinePlugin to dynamically write a component name. The general idea is this. The partial implementation is as follows:
const component = process.argv[2];
// Check if it is a single component development mode, if it is, you must specify the components to run
if (process.env.RUN_ENV === 'component' && !component) {
throw new Error('component is required, like: npm run dev:component slider');
}
// Then write via DefinePlugin
Path is a variable, not a string, so it can't be "'path'".
// config.dev.js
{
// ...
plugins: [
new webpack.DefinePlugin({
'process.env': {
NODE_ENV: "'development'".RUN_ENV: process.env.RUN_ENV === 'component' ? "'component'" : "" '".component: process.env.RUN_ENV === 'component' ? JSON.stringify('/' + component) : 'path'}})]// ...
}
// Then change the source of 'route.js'
function loadDocs(path) {
return r= > require.ensure([],
() => r(require(`./docs${process.env.path}.md`))); }Copy the codeEverything’s ready. Let’s run
> DONE Compiled successfully in 10792ms
Copy the codeNot bad, not bad, only 10 seconds, open the browser to see, no problem. Well good.
The dev command is available on the terminal, but the browser has an error. (Black question mark face)
It seems that Webpack can not be handled normally, and finally changed to the following so that it can work normally
function loadDocs(path) {
return r= > require.ensure([],
() => {
if (process.env.RUN_ENV === 'component') {
r(require(`./docs${process.env.component}.md`));
} else {
r(require(`./docs${path}.md`)); }}); }Copy the codeIn addition, there is a lot of component source code that doesn’t need to be handled except for the MD file, so go ahead and modify the code
The entry to the application changes the global entry to the UI library to on-demand
// The original code
import Vue from 'vue'
import gsui from 'components'
// ...
Vue.use(gsui)
Copy the code// Modified
import Vue from 'vue'
// ...
if (process.env.RUN_ENV === 'component') {
// Some page shared components
// only require cannot import because it is static
Vue.use(require(`components/submenu`).default);
Vue.use(require(`components/menu`).default);
Vue.use(require(`components/layout`).default);
Vue.use(require(`components/menu-item`).default);
Vue.use(require(`components/header`).default);
Vue.use(require(`components/icon`).default);
Vue.use(require(`components/tooltip`).default);
Vue.use(require(`components/modal`).default);
Vue.use(require(`components/message`).default);
Vue.use(require(`components${process.env.component}`).default);
} else {
// Not a single component development model for all
Vue.use(require('components').default);
}
Copy the codeComparison of optimized single component development mode and global development mode
But it quickly became impractical because there were many components that depended on other components, and sometimes you had to look at the documentation of other components
We’ll have to find another way
Unexpectedly, the vUE – Loader version caused the performance consumption
I didn’t know where I found a UI library at-UI the day before yesterday, so I subconsciously checked their build configuration and found it was very similar to ours (in fact, webpack configuration is also similar), and also used vue-Markdown-loader. Out of curiosity, I clone it and build it locally. The result is unexpected, their construction only needs 16s, 16s, how can 16S be so much different? After checking their documents, they still have both Chinese and English copies (our component library has no English documents for the moment). Although there are not as many components as ours, there are definitely dozens more documents. And the time is not also in the MD file parsing it (again question face). Taking a closer look at their configuration and handling of MD files, it is true that there is much less code for handling MD files, but this is due to the different writing methods supported and the time difference is not that great.
Can’t figure out why, let’s try building our project with their configuration. Copy the build directory completely, modify some configuration such as Entry Alias, install some dependencies that don’t exist here, the rest basically don’t need to move, anyway, run to see.
After some corrections, I ran, but the time was still the same (37s). It was strange. Try another one, run their project with our configuration.
Copy the SRC and docs directories of their project, also modify our configuration entry alias and add some loaders, they need to process yML files, run and see. The result is more puzzled, the time is 40s (again question face). Finally, in our project, using their configuration to run their project, I wanted to check if one thing was caused by a different version of a dependency, and it was…
Next is to find out is which interdependencies, need to pay attention to some here package. Rely on the version in json Such as ^ 1.0.0, begin with ^, when installation will always be in accordance with the large version of the latest version of the Namely ^ 1.0.0 ^ 1.1.0 are pack 1 x under the latest version. ^1.0.0 and ^2.0.0 are different. Finally, several different versions of webpack(2.x and 3.x) vue-Markdown-loader (1.x and 2.x) were mainly tried. However, these two versions were still very slow after being replaced. Finally, under the reminding of my colleagues, It may be vue-loader because vue-markdown-loader is dependent on vue-loader, and both 1.x and 2.x are using vue-loader 12.x version, while we use 13.x. Finally, everything pays off. Starting with V13.1.0 of vue-Loader, builds are slower.
The reason for the slow down
The following result was discovered by one of the company’s great people
It turned out that vuE-Loader above V13.1.0 used Prettier to format code instead of JS-Beautify, which caused performance problems.
Final configuration
// config.base.js
module.exports = {
module: {
rules: [{test: /\.css$/.use: [
'style-loader'.'css-loader',
{
loader: 'postcss-loader'.options: {
config: {
path: utils.resolve('./postcss.config.js'}}}]}, {test: /\.md$/.loader: 'vue-markdown-loader'.options: {
use: [
utils.mdAnchor,
utils.demoContainer,
utils.tipContainer
],
preprocess: utils.mdPreprocess
}
},
{
test: /\.scss$/.use: 'happypack/loader? id=scss'
},
{
test: /\.jsx? $/.exclude: exclude: [/node_modules/, /^dll$/],
use: 'happypack/loader? id=babel'.include: [utils.resolve('./components'), utils.resolve('./examples'), utils.resolve('./test')]}, {test: /\.json$/.loader: 'json-loader'
},
{
test: /\.(jpg|png|gif|eot|svg|ttf|woff|woff2)(\? . *)? (#. *)? $/.loader: 'url-loader? name=[name].[hash].[ext]'
},
{
test: /\.vue$/.// use: 'happypack/loader? id=vue'
loader: 'vue-loader'.options: {
loaders: {
css: 'style-loader! css-loader! sass-loader'.js: 'happypack/loader? id=babel'}}}]},resolve: {
extensions: ['.js'.'.vue'.'.json'.'.scss'.'.css'].alias: {
'gs-ui': utils.resolve('/'),
components: utils.resolve('./components'),
examples: utils.resolve('./examples')}},plugins: [
new HappyPack({
id: 'babel'.threads: 4.loaders: ['babel-loader']}),new HappyPack({
id: 'scss'.threads: 4.loaders: [
'style-loader'.'css-loader',
{
loader: 'postcss-loader'.options: {
config: {
path: utils.resolve('./postcss.config.js')}}},'sass-loader']]}});Copy the code// config.dev.js
module.exports = merge(config, {
entry: entry,
output: {
path: '/'.publicPath: ' '.filename: '[name].js'
},
plugins: [
new webpack.DefinePlugin({
'process.env': {
NODE_ENV: "'development'".RUN_ENV: process.env.RUN_ENV === 'component' ? "'component'" : "" '".component: process.env.RUN_ENV === 'component' ? JSON.stringify('/' + component) : 'path'}}),new HtmlWebpackPlugin({
template: utils.resolve('examples/index.html'),
filename: 'index.html'.inject: true
}),
new FriendlyErrorsPlugin(),
new OpenBrowserPlugin({
url: 'http://localhost:' + PORT
}),
new webpack.DllReferencePlugin({
manifest: require('./dll/vendor.manifest.json')})],devServer: {
disableHostCheck: true.host: '0.0.0.0'.port: PORT,
quiet: true.hot: true.historyApiFallback: true
},
devtool: 'cheap-eval-source-map'
});
Copy the codeFinally, the optimized construction speed is 16-17s, and the result is relatively ideal.
conclusion
It turns out to be a build performance problem due to the dependency version, not an optimization of the WebPack build. Kind of a stomp hole
Other optimization points that can be used in the project should mainly be the Happypack DLL, which can effectively improve the speed of construction, and others need to try more