background

As 2016 draws to a close, we can really describe the development of front-end technology in the past year as a whole: whether it is the evolution of technology stack, the push of technology framework, or various patterns, anti-pattern best practices are constantly emerging. An online article, “What’s it Like to Learn JavaScript in 2016?” Is a good summary of the current situation. Of course, to ridicule ridicule, the wheel of technology or always forward, from which we are not difficult to find that the former kind of direct script in JS. By SRC embedded into the page, and then press F5 to refresh the page to view the result of the development mode has been drifting away, basically to choose a suitable compilation and resource management tools has become the standard of all the front-end engineering, and in many build tools, webpack with its abundant function and flexible configuration puts glorious greatly in the 2016 years, React, Vue, AngularJS2 and many other well-known projects use webPack as an official build tool, which is very popular in the industry. With the increasing complexity and code size of engineering development, various performance problems exposed by WebPack become more and more obvious, which greatly affects the development experience.

Problems of induction

After the actual test of several Web projects, we summarized the performance problems gradually exposed in the construction of WebAPCK mainly in the following aspects:

  • Slow full code builds, and even small changes take a long time to see the results of updates and compilations (significant improvements with the introduction of HMR hot updates);

  • With the increase of the complexity of project business, the volume of engineering module will also increase sharply, and the built module is usually calculated in M unit.

  • Multiple projects share basic resources and have repeated packaging, and the base code reuse rate is not high.

  • Node’s single-process implementation performs poorly in cpu-consuming computational Loaders.

To address these issues, let’s take a look at how we can take advantage of some of webPack’s existing mechanisms and third-party extensions.

Where is slow

As engineers, we are always encouraged to think rationally and speak with data and facts. Statements like “I feel slow”, “too slow” and “too big” inevitably seem too general and abstract, so we might as well start to analyze from the following aspects:

  • Starting from the project structure, whether the code organization is reasonable, whether the use of dependency is reasonable;

  • Start with the optimization tools provided by WebPack itself to see which apis are not optimized;

  • From the shortcomings of Webpack itself, do targeted extension optimization, further improve efficiency;

Here we recommend using a Wepback visual resource analysis tool: WebPack-Visualizer. When WebPack is built, it will automatically help you calculate the dependency and distribution of each module in your project, which is convenient for more accurate resource dependency and reference analysis.

From the above we can find most of the engineering project, depend on the volume of a library is always a big head, usually can occupy the whole of the project volume 7-9, and in each development will reread and compile corresponding rely on resources, it is actually a lot of overhead resources waste, and has little impact, the result of the compilation, after all, in the actual business development, We rarely take the initiative to modify the source code in the third-party library. The improvement scheme is as follows:

Solution 1. Reasonably configure CommonsChunkPlugin

The resource entry of Webpack is usually compiled and extracted by entry unit. When multiple entries coexist, CommonsChunkPlugin will come into play and extract the common part of all dependent chunks. However, many people may think that extracting the common part means extracting a code snippet. In fact, it is not. It is extracted by module.

If there are entry1, entry2, and entry3 entries in our page, and these entries may refer to common modules such as utils, loadash, fetch, etc., then we can consider common machine extraction for this part. Generally, there are four methods of extraction:

1. Pass in the string parameter, which is extracted automatically by chunkPlugin



new webpack.optimize.CommonsChunkPlugin('common.js')Copy the code

By default, this will extract the common code for all entry nodes and generate a common.js

2. Selectively extract common code



new webpack.optimize.CommonsChunkPlugin('common.js'['entry1'.'entry2']);Copy the code

Just extract the entry1 node and the shared parts of the module in Entry2 to generate a common.js

3. Extract the common parts of all modules under Entry (the number of references can be specified) into a common chunk



new webpack.optimize.CommonsChunkPlugin({
    name: 'vendors',
    minChunks: function (module, count) {
       return (
          module.resource &&
          /\.js$/.test(module.resource) &&
          module.resource.indexOf(
            path.join(__dirname, '.. /node_modules'= = =))0)}});Copy the code

Extract all modules from node_modules to vendors, and can also specify a minimum number of references in minChunks;

4. Extract some lib from ENry to vendors



entry = {
    vendors: ['fetch'.'loadash']};new webpack.optimize.CommonsChunkPlugin({
    name: "vendors",
    minChunks: Infinity
});Copy the code

Add an entry named Vendors and set vendors to the desired repository, and CommonsChunk automatically extracts the specified vendors repository.

Scheme 2. Extract common libraries through the externals configuration

In real projects where you don’t need to debug the source code for various libraries in real time, you can use the External option instead.

To put it simply, external means declaring our dependent resource as an external dependency, which is then imported through an extrank-script. This is also our early introduction of a carbon copy of page developing resources, after only through configuration, can inform webapck meet with such a variable name can not parse and compile to the module’s internal documents, and to switch to read from the external variables, this can greatly improve compilation speed, at the same time also can better use the CDN to implement caching.

The external configuration is relatively simple. You only need to perform the following steps:

1. Add the lib address to the page as follows:



<head>
<script src="//cdn.bootcss.com/jquery.min.js"></script>
<script src="//cdn.bootcss.com/underscore.min.js"></script>
<script src="/static/common/react.min.js"></script>
<script src="/static/common/react-dom.js"></script>
<script src="/static/common/react-router.js"></script>
<script src="/static/common/immutable.js"></script>
</head>Copy the code

Add external configuration to webapck.config.js:



module.export = {
    externals: {
        'react-router': {
            amd: 'react-router'.root: 'ReactRouter'.commonjs: 'react-router'.commonjs2: 'react-router'
        },
        react: {
            amd: 'react'.root: 'React'.commonjs: 'react'.commonjs2: 'react'
        },
        'react-dom': {
            amd: 'react-dom'.root: 'ReactDOM'.commonjs: 'react-dom'.commonjs2: 'react-dom'}}}Copy the code

One detail to mention here is: Before the configuration of such files, the construction of these resource packages needs to adopt AMD/CommonJS/CMd-related modularization for compatibility encapsulation, that is, the packaged libraries have been packaged in UMD mode. Node_modules /react-router: umd/ reactrouter.js, which require and import * from ‘XXXX’ in webpack, can be read correctly. In this type of js header can also be seen as follows:




if (typeof exports === 'object' && typeof module= = ='object') {
    module.exports = factory(require("react"));
} else if (typeof define === 'function' && define.amd) {
    define(["react"], factory);
} else if (typeof exports === 'object') {
    exports["ReactRouter"] = factory(require("react"));
} else {
    root["ReactRouter"] = factory(root["React"]);
}Copy the code

3. It is important to include the following sentence in the output option:



output: {
  libraryTarget: 'umd'
}Copy the code

Since js modules extracted by external will not be recorded in the chunk information of Webapck, libraryTarget can inform the constructed business module. When it reads the key in externals, it needs to obtain the resource name by UMD. Otherwise, the Module will not be found.

After configuration, we can see that the corresponding resource information is already available in the browser’s Source Map.

Corresponding resources can also be directly linked to the page load, effectively reducing the size of the resource bundle.

Scheme 3. Use DllPlugin and DllReferencePlugin to precompile resource modules

Our project dependencies often reference a large number of NPM packages that are not modified during normal development, but need to be parsed repeatedly during each build. How do we avoid this loss? That’s what these two plugins are for.

To put it simply, the DllPlugin is used to pre-compile modules, and the DllReferencePlugin is used to reference these pre-compiled modules. It should be noted that DllPlugin must be executed once before DllReferencePlugin is executed. The concept of DLL should also refer to the design concept of DLL files in Windows program development.

Compared with externals, dllPlugin has the following advantages:

  • DLL precompiled modules can be reused as static resource link libraries, especially suitable for resource sharing between multiple projects, such as a site PC and mobile version;

  • DLL resources can effectively solve the problem of resource cycle dependence, part of the dependency library, such as react-addons-CSS-transition-group, which was extracted from the React core library.



    module.exports = require('react/lib/ReactCSSTransitionGroup');Copy the code

    React /lib will not be indexed properly if resources imported via externals are only identified by react.

  • Because the externals configuration item needs to be customized one by one for each dependent library, each time adding a component needs to be manually modified, which is slightly tedious. DllPlugin can be read completely through the configuration, reducing the maintenance cost.

1. Configure the resource table corresponding to dllPlugin and compile the file

To use externals, add a configuration file called webpack.dll.config.js:



const webpack = require('webpack');
const path = require('path');
const isDebug = process.env.NODE_ENV === 'development';
const outputPath = isDebug ? path.join(__dirname, '.. /common/debug') : path.join(__dirname, '.. /common/dist');
const fileName = '[name].js';

// Resource dependency package, pre-compiled
const lib = [
  'react'.'react-dom'.'react-router'.'history'.'react-addons-pure-render-mixin'.'react-addons-css-transition-group'.'redux'.'react-redux'.'react-router-redux'.'redux-actions'.'redux-thunk'.'immutable'.'whatwg-fetch'.'byted-people-react-select'.'byted-people-reqwest'
];

const plugin = [
  new webpack.DllPlugin({
    /** * path * Defines the location where the manifest file is generated * [name] parts are replaced by the name of the entry */
    path: path.join(outputPath, 'manifest.json'),
    /** * name * DLL bundle output to that global variable * same as output.library. * /
    name: '[name]',
    context: __dirname
  }),
  new webpack.optimize.OccurenceOrderPlugin()
];

if(! isDebug) { plugin.push(new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify('production')}),new webpack.optimize.UglifyJsPlugin({
      mangle: {
        except: ['$'.'exports'.'require']
      },
      compress: { warnings: false },
      output: { comments: false}}}))module.exports = {
  devtool: '#source-map',
  entry: {
    lib: lib
  },
  output: {
    path: outputPath,
    filename: fileName,
    /** * output.library * will be defined as window.${output.library} * in this case, 'window.vendor_library' */
    library: '[name]',
    libraryTarget: 'umd',
    umdNamedDefine: true
  },
  plugins: plugin
};Copy the code

Then execute the command:



 $ NODE_ENV=development webpack --config webpack.dll.lib.js --progress
 $ NODE_ENV=production webpack --config webpack.dll.lib.js --progress
Copy the code

You can compile the lib static libraries in debug and production respectively. You can also see the following resources generated automatically in the built files:



│ ├─ ├─ ├─ ├─ ├─ manifest.txt ├─ manifest.txt ├─ manifest.txt ├─ manifest.txt ├─ manifest.txtCopy the code

Document Description:

  • Lib.js can be imported directly into the page as a compiled static resource file via a SRC link. In the same way as externals, production and development environments can use proxy forwarding tools such as Charles to replace routes.

  • Manifest. json stores the pre-compilation information in webpack, which is equal to getting the chunk information in the dependency library in advance, so there is no need to repeat compilation in the actual development process.

2. Static resource introduction of dllPlugin

There is a one-to-one correspondence between lib.js and manifest.json, so we may follow this principle in the call process. If we are currently in the development stage, we can introduce lib.js and manifest.json in the common/debug folder. When switching to production environment, resources under Common/DIST need to be introduced for corresponding operations. Considering the cost of manual switchover and maintenance, we recommend add-asset-html-webpack-plugin to inject dependent resources, and the following results can be obtained:



<head>
<script src="/static/common/lib.js"></script>
</head>Copy the code

Add the following code to the webpack.config.js file:



const isDebug = (process.env.NODE_ENV === 'development');
const libPath = isDebug ? '.. /dll/lib/manifest.json' : 
'.. /dll/dist/lib/manifest.json';

// Add mainfest.json to the webpack build

module.export = {
  plugins: [
       new webpack.DllReferencePlugin({
       context: __dirname,
       manifest: require(libPath),
      })
  ]
}Copy the code

After the configuration, we can find that the corresponding resource package has completed the extraction of pure service modules

If you need to use a common lib resources among multiple projects, only need to introduce the corresponding lib. Js and manifest. Js, the configuration of the plugin also supports multiple webpack. DllReferencePlugin into use at the same time, as follows:



module.export = {
  plugins: [
     new webpack.DllReferencePlugin({
        context: __dirname,
        manifest: require(libPath),
      }),
      new webpack.DllReferencePlugin({
        context: __dirname,
        manifest: require(ChartsPath),
      })
  ]
}Copy the code

Use Happypack to speed up your code building

The above introduction is all about the optimization and improvement of chunk calculation and compilation content in Webpack, and the improvement of the actual volume of resources is also obvious. Besides, can we make some attempts to optimize the compilation process and speed of resources?

As we all know, Webpack is designed to read resources in the form of loader in order to facilitate the loading of various resources and types. However, limited by the programming model of Node, all loaders are called concurrently in the form of async. However, it still runs in a single node process and in the same event loop, which directly leads to when we need to read multiple Loader file resources at the same time. For example, babel-loader needs to transform various JSX, ES6 resource files. Happypack is designed to solve this problem where node’s single-process model doesn’t have the advantage of simultaneous computationand cpu-intensive processes.

Start the thread pool of Happypack

Happypack extends the original WebPack execution process from a single process to a multi-process mode. The original process remains unchanged, so that the compilation process can be optimized without modifying the original configuration. The specific configuration is as follows:



var HappyPack = require('happypack');
var happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length });

module: {
  loaders: [
      {
        test: /\.js[x]? $/.// loader: 'babel-loader? presets[]=es2015&presets[]=react'
        loader: 'happypack/loader? id=happybabel'
      }
    ]
  },
  plugins: [
      new HappyPack({
        id: 'happybabel',
        loaders: ['babel-loader'],
        threadPool: happyThreadPool,
        cache: true,
        verbose: true})]Copy the code

We can see that the loader provided by Happypack is configured to directly point to the loader provided by happypack. For the loader that actually matches files, it is configured to pass the description in the plugin property. Here, the loader provided by Happypack matches the plugin. Id = happyBabel. After the configuration is complete, the Laoder works in the following mode:

Happypack uses the multi-process mode to speed up the compilation process, and also enables the cache calculation, which can make full use of the cache to read the build file. The speed of the build is also very obvious. After testing, the final build speed is as follows:

Before optimization:

After the optimization:

More on Happyoack can be found at:

  • happypack

  • Happypack principle parsing

5. Enhance uglifyPlugin

UglifyJS has become the first choice of JS compression tools by virtue of its node-based development, high compression ratio, easy to use and many other advantages. However, we observed in the construction of Webpack that when the progress of Webpack build reached about 80%, there would be a long period of stagnation. It was found that uglfiyJS took too long to compress the Bunlde part of our output due to this process. For this part, we recommend using Webpack-Ugli-fi parallel to improve the compression speed.

It can be seen from the plug-in source code that the implementation principle of Webpack-Ugli-fi – Parallel is to adopt the way of multi-core parallel compression to improve our compression speed.



plugin.nextWorker().send({
    input: input,
    inputSourceMap: inputSourceMap,
    file: file,
    options: options
});

plugin._queue_len++;
                
if(! plugin._queue_len) { callback(); }if (this.workers.length < this.maxWorkers) {
    var worker = fork(__dirname + '/lib/worker');
    worker.on('message'.this.onWorkerMessage.bind(this));
    worker.on('error'.this.onWorkerError.bind(this));
    this.workers.push(worker);
}

this._next_worker++;
return this.workers[this._next_worker % this.maxWorkers];Copy the code

Using the configuration is also very simple, we just need to configure the uglifyPlugin that comes with our original Webpack:



new webpack.optimize.UglifyJsPlugin({
   exclude:/\.min\.js$/
   mangle:true,
   compress: { warnings: false },
   output: { comments: false}})Copy the code

Change to the following code:



const os = require('os');
    const UglifyJsParallelPlugin = require('webpack-uglify-parallel');
    
    new UglifyJsParallelPlugin({
      workers: os.cpus().length,
      mangle: true,
      compressor: {
        warnings: false,
        drop_console: true,
        drop_debugger: true}})Copy the code

Applicable scenario

In the actual development process, it can flexibly choose the optimization means suitable for its own business scenarios.

Optimization means The development environment The production environment
CommonsChunk Square root Square root
externals   Square root
DllPlugin Square root Square root
Happypack Square root  
uglify-parallel   Square root

Engineering Demo

summary

Performance optimization is no trivial matter, and there is no end to the pursuit of speed. In today’s increasingly large and complex front-end engineering, continuous improvement of the performance of construction tools for actual projects is extremely beneficial to the improvement of project development efficiency and in-depth understanding of tools.