Improved compilation speed by 180% with webpack4

Since the local project was upgraded to Webpack 4 a few months ago, webpack was manually downgraded to version 3.12.0 in order to obtain test data, and the rest of the configuration was left largely unchanged.

During the test, Mac only ran common IM, mailbox, terminal, browser, etc. In order to avoid the impact of plug-ins on data as much as possible, I closed some optimized plug-ins and reserved only common Loader and JS compression plug-ins.

First, the webpack-dev-server plug-in needs to be updated to the latest, and webpack-CLI is also required because webpack-CLI takes on the webpack4 command line related functions.

The mode attribute must be specified, otherwise convention takes precedence over configuration, and the production environment will be compiled by default.

WARNING in configuration The ‘mode’ option has not been set, Webpack will fallback to ‘production’ for this value. Set ‘mode’ option to ‘development’ or ‘production’ to enable Defaults for each environment. You can also set it to ‘none’ to disable any default behavior. Learn more: Webpack.js.org/concepts/mo…

There are two ways to add a mode configuration.

• Specify — mode in package.json script:

  "scripts": {
      "dev": "webpack-dev-server --mode development --inline --progress --config build/webpack.dev.config.js"."build": "webpack --mode production --progress --config build/webpack.prod.config.js"
  }
  Copy the code

• Add the mode attribute to the configuration file

  module.exports = {
    mode: 'production' / / or development
  };
  Copy the code

After upgrading to webpack4, some of the default plug-ins have been replaced by the optimization configuration as follows:

• CommonsChunkPlugin abandoned by optimization. SplitChunks and optimization. RuntimeChunk substitution, the former split code, which extract the runtime code. The original CommonsChunkPlugin produced modules with duplicate code and could not optimize asynchronous modules. The configuration of Minitchunks was also complicated. SplitChunks solved this problem. . In addition, the optimization runtimeChunk is set to true (or {name: “manifest”}), will have to pick up the entrance to the module of the runtime.
• NoEmitOnErrorsPlugin abandoned by optimization noEmitOnErrors alternative, production environment opened by default.
• NamedModulesPlugin abandoned by optimization namedModules alternative, production environment opened by default.
• ModuleConcatenationPlugin abandoned by optimization concatenateModules alternative, production environment opened by default.
• Optimize.UglifyJsPlugin obsolete, optimization. Minimize replacement, production environment default on.

Not only that, Optimization provides the following default configuration:

optimization: {
    minimize: env === 'production' ? true : false.// Development environment does not compress
    splitChunks: {
        chunks: "async".// Three values are available: initial(initial module), async(on-demand module), and all(all module)
        minSize: 30000.// Modules over 30K are automatically removed to public modules
        minChunks: 1.// If the module is referenced >= once, it is split
        maxAsyncRequests: 5.// The number of concurrent requests for asynchronously loading chunk <=5
        maxInitialRequests: 3.// The number of chunks loaded concurrently in an entry <=3
        name: true.// The default module name is hash. Multiple modules with the same name will be merged into one
        automaticNameDelimiter: '~'.// Name the delimiter
        cacheGroups: { // The cache group inherits and overwrites the splitChunks configuration
            default: { // Module cache rule, set to false, default cache group will be disabled
                minChunks: 2.// The modules are referenced >=2 times, and are then split into vendors public modules
                priority: - 20./ / priority
                reuseExistingChunk: true.// Use existing modules by default
            },
            vendors: {
                test: /[\\/]node_modules[\\/]/.// splits modules in node_modules by default
                priority: - 10}}}}Copy the code

SplitChunks are important to unpack and optimize. If your project contains third-party components such as Element-UI (large components), it is recommended to unpack them separately, as shown below.

splitChunks: {
    // ...
    cacheGroups: {    
        elementUI: {
            name: "chunk-elementUI".// Unpack elementUI separately
            priority: 15.// The weight must be greater than other cache groups
            test: /[\/]node_modules[\/]element-ui[\/]/}}}Copy the code

For more usage, see the comments above or the official documentation SplitChunksPlugin.

At the beginning of this article, I mentioned that optimization is just beginning. Yes, as projects get more complex and Webpack gets slower, there must be a way to squeeze performance even further.

After running and testing multiple projects over a long period of time, the following lessons have worked well.

1. Reduce the compilation scope and reduce unnecessary compilation work. That is, modules, mainFields, noParse, includes, exclude, and alias are all used.

   const resolve = dir => path.join(__dirname, '.. ', dir);
   
   // ...
   resolve: {
       modules: [ // Specify the following directories to find third-party modules to avoid Webpack's recursive search to the parent directory
           resolve('src'),
           resolve('node_modules'),
           resolve(config.common.layoutPath)
       ],
       mainFields: ['main'].// Only use the main field as the entry file description field to reduce the search steps
       alias: {
           vue$: "vue/dist/vue.common"."@": resolve("src") // Cache SRC directories as @ symbols to avoid repeated addressing}},module: {
       noParse: /jquery|lodash/.// Ignore files that are not modularized, so jquery or LoDash will not be parsed by loaders below
       // noParse: function(content) {
       // return /jquery|lodash/.test(content)
       // },
       rules: [
           {
               test: /\.js$/,
               include: [ // Only the following directories are parsed, reducing the processing scope of the Loader
                   resolve("src"),
                   resolve(config.common.layoutPath)
               ],
               exclude: file => /test/.test(file), // Exclude the test directory file
               loader: "happypack/loader? id=happy-babel" // More on that later]}},Copy the code

2. If you want to further improve compilation speed, you need to know where the bottleneck is? Through the test, it is found that there are two slow stages: ① Babel loaders parsing stage; ② JS compression stage. Loader parsing will be discussed later, and JS compression is the last stage of publishing and compiling. Usually webPack needs to be stuck for a while. This is because js compression requires the code to be parsed into the AST syntax tree, then the AST needs to be analyzed and processed according to complex rules, and finally the AST is returned to JS. This process is computationally intensive and therefore time-consuming. As shown below, compilation looks stuck.

   

   In fact, with the Webpack-parallel-Ugliffe -plugin, the process can be sped up twice as fast. We all know that Node is single-threaded, but node can fork a task. Based on this, webpack-parallel-uglip-plugin can split the task into multiple sub-processes for concurrent execution, and then send the result to the main process, thus implementing concurrent compilation. In this way, js compression speed is greatly improved. The configuration is as follows.

   const ParallelUglifyPlugin = require('webpack-parallel-uglify-plugin');
   
   // ...
   optimization: {
       minimizer: [
           new ParallelUglifyPlugin({ // Multi-process compression
               cacheDir: '.cache/',
               uglifyJS: {
                   output: {
                       comments: false,
                       beautify: false
                   },
                   compress: {
                       warnings: false,
                       drop_console: true,
                       collapse_vars: true,
                       reduce_vars: true}}}),]}Copy the code

   Of course, I tested five sets of data, and here are the screenshots:

   

   Data analysis is as follows (unit: ms) :

With the webpack-parallel-Ugli-fi plugin, the build speed of WebPack3 can be increased by 46%. Even after upgrading to webpack4, the build speed increased by a further 35%.

1. Now let’s see how the loader parsing speed can be improved. HappyPack, like the webpack-parallel-uglip-plugin, can compile concurrently, which can greatly improve the speed of loader parsing.

   const HappyPack = require('happypack');
   const happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length });
   const createHappyPlugin = (id, loaders) => new HappyPack({
       id: id,
       loaders: loaders,
       threadPool: happyThreadPool,
       verbose: process.env.HAPPY_VERBOSE === '1' // make happy more verbose with HAPPY_VERBOSE=1
   });
   Copy the code

   Loader: “happypack/loader? Id =happy-babel”, you need to create a happy-babel plug-in instance in your plugins.

   plugins: [
       createHappyPlugin('happy-babel', [{
           loader: 'babel-loader',
           options: {
               babelrc: true,
               cacheDirectory: true // Enable caching}}]]Copy the code

   HappyPack starts 3 processes (default number of cpus -1).

   

   In addition, vue-loader and CSS-loader support happyPack acceleration, as shown below.

   plugins: [
       createHappyPlugin('happy-css', ['css-loader', 'vue-style-loader']),
       new HappyPack({
           loaders: [{
               path: 'vue-loader',
               query: {
                   loaders: {
                       scss: 'vue-style-loader!css-loader!postcss-loader!sass-loader?indentedSyntax'
                   }
               }
           }]
       })
   ]
   Copy the code

   Based on WebPack4, equipped with Webpack-parallel-Ugli-fi – Plugin and happyPack plug-in, test screenshots are as follows:

   

   Data analysis is as follows (unit: ms) :

It can be seen that the happyPack plugin can still improve the compilation speed by 17% on the basis of the webpack-parallel ugli-fi -plugin. In fact, because sass and other loaders do not support happyPack, HappyPack still has room for improvement. For more information, please refer to happypack.

1. As we all know, when webpack is packaged, some framework code is basically unchanged, such as Babel-Polyfill, vue, vuex, AXIos, Element-UI, Fastclick, etc. These modules also have a large size. Each compilation must be loaded once, which is time-consuming and laborious. These modules can be pre-packaged using the DLLPlugin and DLLReferencePlugin plug-ins.

   To complete the DLL process, we need to prepare a new WebPack configuration, webpack.dll.config.js.

   const webpack = require("webpack");
   const path = require('path');
   const CleanWebpackPlugin = require("clean-webpack-plugin");
   const dllPath = path.resolve(__dirname, ".. /src/assets/dll"); // Directory for storing DLL files
   
   module.exports = {
       entry: {
           // Put vUE related modules into a separate dynamic link library
           vue: ["babel-polyfill"."fastclick"."vue"."vue-router"."vuex"."axios"."element-ui"]
       },
       output: {
           filename: "[name]-[hash].dll.js"./ / generated vue. DLL. Js
           path: dllPath,
           library: "_dll_[name]"
       },
       plugins: [
           new CleanWebpackPlugin(["*.js"] and {// Clear the previous DLL file
               root: dllPath,
           }),
           new webpack.DllPlugin({
               name: "_dll_[name]".// manifest.json describes what the dynamic link library contains
               path: path.join(__dirname, ". /"."[name].dll.manifest.json")})]};Copy the code

   Next, you need to add the DLL command to package.json.

   "scripts": {
       "dll": "webpack --mode production --config build/webpack.dll.config.js"
   }
   Copy the code

   • 1
   • 2
   • 3

   After running NPM run DLLS, generates the. / SRC/assets/DLL/vue. DLL – [hash]. Public js and js. / build/vue. DLL. The manifest. Json resource description file, thus the DLL preparation is complete, Then reference it in wepack.

   externals: {
       'vue': 'Vue'.'vue-router': 'VueRouter'.'vuex': 'vuex'.'elemenct-ui': 'ELEMENT'.'axios': 'axios'.'fastclick': 'FastClick'
   },
   plugins: [
       ...(config.common.needDll ? [
           new webpack.DllReferencePlugin({
               manifest: require("./vue.dll.manifest.json"})] : []]Copy the code

   DLL public js does not change easily, and if an update does occur in the future, the new DLL file name will need to be added with the new hash, so that the browser does not cache the old file and cause execution errors. Because of the uncertainty of the hash, there is no way to specify a fixed link script in the HTML entry file, but the add-asset-html-webpack-plugin automatically imports DLL files.

   const autoAddDllRes = () => {
       const AddAssetHtmlPlugin = require('add-asset-html-webpack-plugin');
       return new AddAssetHtmlPlugin([{ // Inject DLL js into HTML
           publicPath: config.common.publicPath + "dll/".// The path injected into the HTML
           outputPath: "dll".// Final output directory
           filepath: resolve("src/assets/dll/*.js"),
           includeSourcemap: false,
           typeOfAsset: "js" // options js, CSS; default js
       }]);
   };
   
   // ...
   plugins: [
       ...(config.common.needDll ? [autoAddDllRes()] : [])
   ]
   Copy the code

   With the DLL plug-in, the compilation speed of Webpack4 is further improved, as shown in the screenshot below:

   

   Data analysis is as follows (unit: ms) :

It can be seen that the compilation speed of Webpack4 can still be improved by 22% with DLL installed.

To sum up, we summarize the above multiple times of data and get the following table:

ParallelUglifyPlugin, happyPack, and DLL plugins can improve the compile speed by 181% and reduce the overall compile time by nearly two-thirds. And as the project progresses, this compilation improvement becomes more and more noticeable.

In fact, in order to obtain the above test data, I turned off the cache of Babel and ParallelUglifyPlugin. After enabling the cache, the average compile time of the second time was 12.8s, which is 362% faster than webPack3 due to the previous cache. Even if you’ve upgraded to webpack4, you’ll still see a 218% increase in compile speed with all three plugins listed above!

The file size is as shown below:

In the process of upgrading webpack4, stomping pits is a must, the key is stomping pits, what do you get?

In addition, in addition to some optimization methods introduced in the article, more optimization strategies are being verified step by step…