preface

Since vUE was enabled in the technology stack of the new project, the construction tool of the project naturally switched from the original internal tool to Webpack. After feeling the power of HMR and various loaders, the project gradually grew larger and depended on more and more modules. The construction efficiency of Webpack has become a short board restricting the team’s development efficiency. So, let’s take a look at how we can optimize webPack’s efficiency in multiple pages (after all, the title of this article is not completely referring to north, if there is a better way, feel free to leave me a comment).

Project background

Our project is a multi-page project based on VUE. The Webpack configuration file is rewritten based on VUE-CLI, so there are multiple entries in Webpack. The general structure of the project is as follows

| - SRC | - pages | | xxx1 - a business page 1 - App. Vue - the business main entrance vue component | - xxx1. HTML - (template file and directory name, business) | - xxx1. Js - (and directory name, Business main entrance js file) | xxx2 - a business page 2 | - App. The main entrance vue vue - the business component | - xxx2. HTML - (template file and directory name, business) | - xxx2. Js - (main entrance js file and directory name, business)Copy the code

Now, let’s take a look at how we optimized the WebPack build based on this multi-page structure (based on WebPack 3)

Common code extraction

As anyone who has used vue-CLI knows, the ** CommonsChunkPlugin is used as the code Splite tool by default when generating template projects. Essentially, minChunk is configured to extract common parts of code for easy caching across multiple pages (e.g. Both pages A and B have vendor.js, so if you visit page A, the next time you visit page B, vendor.js in page B can be directly loaded into the memory), thus achieving the purpose of performance improvement. While the Plugin** also has disadvantages, namely it is dynamically compiled and code splite. Since minChunk policies vary, the contents of the extracted common vendor.js code may vary from version to version after each release. Tripartite libraries such as VUE (Vuex Vue-Router) are basically stable and do not need to change according to business changes. Therefore, based on this we can extract these third-party libraries pre-built in advance, rather than having them build again with the release

Methods a

The easiest way is to directly merge and compress these JS and mount them on the global node, but if we do this, we can only use the various functions provided by them in the business code through the properties under the window, breaking the encapsulation of modularity, so this solution is not good.

Method 2

Considering the limitations of CommonChunkPlugin, WebPack officially provides another plugin, DllPlugin, which needs to be used in conjunction with the DLLReferencePlugin.

Those who are familiar with Windows should know what DLL stands for. In Windows, there are a number of.dll files called dynamic link libraries. Dynamically linked libraries provide a way to modularize applications so that their functions can be reused more easily.

Therefore, our purpose is to use DLL plug-in to extract the common part of the module without modification and package it separately. We first build webpack.dll.config.js, the content of this file is very simple.

const path = require('path'); const webpack = require('webpack'); const UglifyJsPlugin = require('uglifyjs-webpack-plugin'); const config = require('.. /config'); module.exports = { entry: { vendor: ['vue/dist/vue.esm.js', 'vuex', 'axios', 'vue-router', 'babel-polyfill', 'lodash'] path.join(__dirname, '.. /static/js'), // the output position of the packaged file filename: '[name].dll. Js ', /** * output.library * will be defined as window.${output.library} * in this example, Will be defined as' window.vendor_library '*/ library: '[name]_library'}, plugins: [new webpack.dllplugin ({// this plugin is used to package js /** * path * define the location generated by the manifest file * [name] parts of the entry name */ path: Path. join(__dirname, '.', '[name]-manifest.json'), /** * name * DLL bundle output to that global variable */ '[name]_library', context: path.join(__dirname, '..')}), new UglifyJsPlugin({// Use this plugin to obfuscate wrapped js uglifyOptions: { compress: { warnings: false } }, sourceMap: config.build.productionSourceMap, parallel: true }) ] };Copy the code

Perform webpack – config build/webpack. DLL. Config. Js, webpack automatically generates two files, including vendor. DLL. Js namely third-party modules after the merger packaging. The other vendor-mainifest. Json stores the mapping between each module and the required common module.

After wrapping the third-party module, we need to use DLLReferencePlugin to integrate it with our business code. We modify webpack.base. Config (VUe-CLI generation configuration) and add plugin as follows:

Plugins: [new webpack DllReferencePlugin ({context: __dirname, / / and is consistent with the context of DllPlugin manifest: require('./vendor-manifest.json') }), ...... ]Copy the code

At the same time, we need to manually insert vendor.dll.js into a template file such as index.html to take effect

<script src="/vendor.dll.js"></script>
Copy the code

This completes the process of extracting common third-party libraries using the DLL plug-in. Normally, we don’t add or subtract third-party libraries, but when that happens, we need to manually repackage them to replace them. So is there a more automatic way to do this?

Methods three

The AutoDllPlugin appears in my view. This plugin automatically does both the DllReferencePlugin and DllPlugin and is added in webpack.base.config

plugins: [
    new AutoDllPlugin({
            inject: true, // will inject the DLL bundles to html
            context: path.join(__dirname, '..'),
            filename: '[name]_[hash].dll.js',
            path: 'res/js',
            plugins: mode === 'online' ? [
                new UglifyJsPlugin({
                    uglifyOptions: {
                        compress: {
                            warnings: false
                        }
                    },
                    sourceMap: config.build.productionSourceMap,
                    parallel: true
                })
            ] : [],
            entry: {
                vendor: ['vue/dist/vue.esm.js', 'vuex', 'axios', 'vue-router', 'babel-polyfill', 'lodash']
            }
     })
]
Copy the code

No additional webpack.dll.config.js configuration is required and no manual copy of the finished package into the corresponding template file is required.

summary

In most cases, we recommend the methods, but the methods compared to method 2, increases the time start to build a new vendor, js, development phase can build a new vendor for the first time to add some extra time (measured down impact is not big), but also avoid the update of third-party libraries and forget that increase or decrease the impact of packaging on the business

Multithreaded build

Webpack and most of the other JS tools are single-threaded project processing, but the strength of webPack is that the process design is so extensible that it can be artificially added to the multi-process processing. The process of compiling files is as follows:

1. Start compiling (Compiler#run) 2. Start compiling entry files (Compilation#addEntry) 2.1 start compiling files (Compilation#buildModule => NormalModule#build) 2.2 Execute Loader to get file result (NormalModule#runLoaders) 2.3 parse dependencies based on result (NormalModule#parser.parse) 2.4 process dependent file list (Compilation# processModuleDependencies) 2.5 began to compile each dependent file (asynchronous, start from here recursive operations: compile file - > depend on - > compiler dependent on file - > parse deep dependence...).Copy the code

The key here is that recursive operation 2.5 starts compiling each dependency file. This step is designed asynchronously, and the compilation of each dependency file does not affect each other. But it’s asynchronous, but it’s still running in one thread. But this design allows for multiple processes.

The main time-consuming operation in compiling a file is the conversion operation of the Loader to the source file, and the asynchronous design of the Loader ensures that the conversion operation is not restricted to the same thread. Modify Loader to support multi-process concurrency:

LoaderWrapper receives file input as a new Loader entry LoaderWrapper creates a child process (child_process#fork). By calling the original Loader, the input file is converted, and the final result is passed to the parent process, which passes the received result to Webpack as the Loader resultCopy the code

The babel-loader is used as an example to explain how to configure HappyPack

Typically, we use the babel-loader shown below

webpack.base.config.js ... module: { rules: [ ... { test: /\.js$/, include: [resolve('src'), resolve('lib'),resolve('test'), resolve('node_modules/webpack-dev-server/client')], Use: [{loader: 'babel-loader'},], exclude: /node_modules/ / Exclude can also be used to improve build performance}}Copy the code

Convert to HappyPack and rewrite the configuration to

const HappyPack = require('happypack'); const happyThreadPool = HappyPack.ThreadPool({size: os.cpus().length}); Module. exports = {module: {rules: [{test: /\.js$/, include: [resolve('src'), resolve('lib'), resolve('test'), resolve('node_modules/webpack-dev-server/client')], use: [ { loader: }, exclude: /node_modules/}]}, plugins: [new HappyPack({// HappyPack id: 'happybabel', loaders: ['babel-loader? CacheDirectory =true'], threadPool: {// HappyPack id: 'happybabel', loaders: ['babel-loader? CacheDirectory =true'], threadPool: happyThreadPool, }) ] }Copy the code

HappyPack can not only process babel-loader, other vue-loader, CSS-loader, etc. can use it to accelerate optimization, just need to add instances and rewrite the loader. Overall optimization with HappyPack improved build speed by 70% on our projects.

Multi-page HTML-webpack-plugin optimization

The htML-webpack-plugin, as the first large plug-in in Webpack, should be used more or less. This plug-in will build corresponding HTML, EJS and even FTL files according to your template code through different template engines. In the standard SPA, the performance of this plug-in will not bottleneck. However, if you are using multiple pages, the build speed of this plug-in is absolutely hell level. For example, I simply changed a copy of a Vue file, and it took 16s in the stage, which greatly slowed down the development efficiency and did not feel the advantages of HMR


image.png

We find the EMIT event hook in the HTML-webpack-plugin and inject the event code


image.png

We found that he would execute all the code logic in emit once for each entry file




image.png

.

Therefore, we need to consider how to execute the emit following flow only at the entry we have modified to.

After browsing through a lot of issure, we found that there are already existing wheels to help us complete the judgment and caching functions.

Modify the configuration code code

const HtmlWebpackPlugin = require('html-webpack-plugin-for-multihtml'); [new HtmlWebpackPlugin({template: filePath, filename: '${filename}.html', chunks: ['manifest', 'vendor', filename], inject: true, multihtmlCache: true // Add this configuration})]Copy the code

This plugin ensures that emit fires the entire process only once in the original HTML-webpack-plugin by setting variables in the webpack Done hook function. To speed things up. After the upgrade, the HMR speed was changed from second to millisecond.


image.png

Reference documentation

HappyPack – Webpack accelerator