The road to WebPack application compilation optimization

Author: Miao Dian

The most widely used and widely used application packaging tool is WebPack. What else can we do beyond the optimization capabilities that WebPack already provides (for example, Tree Shaking, Code Splitting, etc.) This article mainly introduces some exploration of The Didi WebApp team on this road.

preface

Now more and more projects are using ES2015+ development, and with Webpack + Babel as the engineering foundation, and through NPM to load third-party dependency libraries. At the same time, in order to achieve the purpose of code reuse, we will take some self-developed component libraries or JSSDK into independent warehouse maintenance, and load through NPM.

Most people are used to this approach and find it very convenient and practical. But behind the convenience, there are two problems:

• Code redundancy

  Generally speaking, these NPM packages are also developed based on ES2015+, and each package needs to be Babel compiled and distributed before it can be used by the main application. This compilation process often adds a lot of “compiled code”; Each package has some of the same compiled code, which creates a lot of code redundancy that cannot be removed by techniques such as Tree Shaking.

• Unnecessary dependencies

  Considering the component library scenario, we usually introduce all the components at once for convenience; However, in reality, only some components may be used in an application, and if all of them are introduced, the code will be redundant.

Redundant code leads to longer load times and execution times for static resource packs, which has a direct impact on performance and experience. Now that we recognize that there are such problems, let’s look at how to solve these two problems.

The core

Our core solutions to these two problems are: post-compile and on-demand import.

The effect

Let’s first take a look at the data of Didi ticket project (ticket users) before and after optimization (non-GZIP, the size of the whole project after compression) :

• Normal packaging: 455 KB
• Post compilation: 423 KB
• Post compiled & Imported on demand: 388 KB
• Post-compiled & Imported on Demand & babel-preset-env: 377 KB

The result was a reduction of about 80 KB, which is a pretty impressive optimization.

The above data is for the component libraries and some internal generic JSSDKS using the post-compilation and on-demand introduction strategy. It is important to note that the on-demand introduction is project dependent and the data here is for reference only.

Let’s take a look at these two points in detail.

After the compilation

To explain:

Post-compile: this means that the NPM packages that the application depends on are not compiled before release, but are compiled as the application is compiled and packaged.

The core of post-compilation is to delay the compilation of dependency packages and compile them uniformly. Let’s first look at its WebPack configuration.

configuration

For a specific project, we don’t need to do much to post compile. We just need to include the dependencies in the webpack configuration file (Webpack 2+) :

// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      // ...
      {
        test: /\.js$/.loader: 'babel-loader'.// Note the include here
        // In addition to SRC there are two additional packages under node_modules
        include: [
            resolve('src'),
            resolve('node_modules/A'),
            resolve('node_modules/B')]},// ...]},// ...
}Copy the code

We just need to include the post-compiled modules A and B through the Include configuration of WebPack.

But there are some problems, for example, the following picture:

The application shown above relies on packages A and B that require post-compilation, which in turn relies on packages C and D that require post-compilation, and B relies on package E that does not require post-compilation. To focus on dependency package A: A needs to be post-compiled, and A’s dependencies C and D also need to be post-compiled. This scenario is called nested post-compilation. If you use the same webPack configuration, you must also include C and D. It only knows that it needs to post compile packages A and B. It doesn’t know that A also has packages C and D that need to post compile, so the application should not include packages C and D. Instead, it should let A show which modules it needs to post compile.

To solve the above problems, we developed a webpack plug-in, webpack-post-compile-plugin, which is used to automatically collect post-compiled dependency packages and their nested dependencies. Take a look at the core code of the plug-in:

var util = require('./util')

function PostCompilePlugin (options) {
  // ...
}

PostCompilePlugin.prototype.apply = function (compiler) {
  var that = this
  compiler.plugin(['before-run'.'watch-run'].function (compiler, callback) {
    // ...
    var dependencies = that._collectCompileDependencies(compiler)
    if (dependencies.length) {
      var rules = compiler.options.module.rules
      rules && rules.forEach(function (rule) {
        if (rule.include) {
          if (!Array.isArray(rule.include)) {
            rule.include = [rule.include]
          }
          rule.include = rule.include.concat(dependencies)
        }
      })
    }
    callback()
  })
}Copy the code

The idea is to collect dependencies in the webpack Compiler before-run and watch-run event hooks and attach them to the webpack module.rule include. The rule to collect is to look for the compileDependencies declared in the package.json file of your application or dependencies.

So for the above application, use the webpack configuration of the webpack-post-compile-plugin plugin:

var PostCompilePlugin = require('webpack-post-compile-plugin')
// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      // ...
      {
        test: /\.js$/.loader: 'babel-loader'.include: [
            resolve('src')]},// ...]},// ...
  plugins: [
    new PostCompilePlugin()
  ]
}Copy the code

Add the compileDependencies field to your current project’s package.json to specify post-compilation dependencies:

// app package.json
{
  // ...
  "compileDependencies": ["A"."B"]
  // ...
}Copy the code

A also has post-compile dependencies, so you need to specify compileDependencies in package A’s package.json:

// A package.json
{
  // ...
  "compileDependencies": ["C"."D"]
  // ...
}Copy the code

advantages

• Common dependencies can be shared, but only compiled once. It is recommended to manage dependencies through peerDependencies.
• The Babel transformation API (such as Babel-plugin-transform-Runtime or Babel-Polyfill) has only one copy of the code.
• You don’t need to configure the compile package for every dependency, and you can even distribute it directly at the source level.

PS: With regard to the choice between Babel-plugin-transform-Runtime and Babel-Polyfill, for the application we recommend the use of Babel-polyfill. Because some third-party package dependencies can determine whether or not a feature is globally supported, it is not a polyfill. For example, vuex checks if promises are supported and reports an error if they are not. Or babel-plugin-transform-Runtime will not handle code like “foobar”.includes(“foo”).

Of course, a post-compiled technical solution is certainly not perfect, and it does have some drawbacks.

disadvantages

• The Babel configuration of the primary application must be compatible with the Babel configuration of the dependent packages.
• You can’t use aliases for dependencies, and you can’t conveniently use the DefinePlugin (which can be compiled once, but without Babel).
• Applications take longer to compile.

There are some drawbacks, but considering the costs/benefits, post-compilation is still a good choice for now.

According to the need to introduce

Post-compilation mainly solves the problem of code redundancy, while on-demand introduction mainly solves the problem of unnecessary dependencies.

On demand import scenarios mainly include component libraries and tool class dependency packages. Because component libraries and dependency packages are often “large and comprehensive”, we may only use part of their capabilities when developing applications. If we introduce all of them, it will waste a lot of resources.

To solve this problem, we need to introduce on demand. At present, mainstream component libraries or toolkits also provide the ability to import on demand, but basically provide the import of compiled modules.

What we recommend is an on-demand introduction of source code with post-compiled packaging.

But actually we may encounter some backward compatibility problem, not a bamboo pole killed, such as the project has been created before, there is no human or time to do the corresponding upgrade, then some of the component library we internally or kit now needs to do a bit of sacrifice: provide two entrances, a compiled entrance, a source entry.

The battle for the entrance

Webpack 2+ or Rollup has already dealt with this problem. The compiled entry uses the package.json main field, and the source entry uses the Module field. See rollup pkg.module wiki. In this way, we can achieve two entry sharing, both to ensure backward compatibility, but also to ensure that the use of WebPack 2+ or rollup entry is directly pointing to the source code, in this basis can be very direct use after compilation.

Vue component library compilation

One of the most typical scenarios for post-compilation and on-demand introduction is our component libraries. Here we share our practical experience with component libraries (based on Vue).

Import on demand, in the absence of post-compilation, in fact, we have implemented the compilation of each module automatically at the time of compilation and release, so that the user can directly import the entry file of the corresponding directory. The principle is simple: walk through the module directory under the source directory to get to the entry, dynamically modifying the entry of the webpack configuration of the component library. This process does not exist in the post-compilation scenario and can be directly introduced into the corresponding module entry of the source code, because the post-compilation does not require the dependency package to compile itself, just the application to compile.

For a component, if it is pre-compiled, we usually compile the import/export JS file, as well as the style/CSS file, so that if we implement the import on demand, it might look like this:

import Dialog from 'cube-ui/lib/dialog'
import 'cube-ui/lib/dialog/style.css'Copy the code

Even in the post-compilation scenario, where there is no need to deal with styling issues, the path is not elegant enough when introduced on demand:

import Dialog from 'cube-ui/src/modules/dialog'Copy the code

Either way, it’s not elegant enough, but babel-plugin-transform-Imports is available to help with elegant, on-demand imports. But for our compiled scene, we still need to introduce styles, so we made it uniform by adding the enhanced Babel-plugin-transform-modules plugin on top of the babel-plugin-transform-imports. The style configuration item has been added.

So whether we use post-compile or not, if we want to import on demand, we just need to:

import { Dialog } form 'cube-ui'Copy the code

If you are using post compilation and importing the source code directly, just add the following configuration to the.babelrc file:

"plugins": [["transform-modules", {
     "cube-ui": {
       "transform": "cube-ui/src/modules/${member}"."preventFullImport": true."kebabCase": true}}]]Copy the code

If you are using WebPack 1 or if you are using a compiled component library, just add a style item:

"plugins": [["transform-modules", {
     "cube-ui": {
       "transform": "cube-ui/lib/${member}"."preventFullImport": true."kebabCase": true."style": true}}]]Copy the code

This gives us an elegant on-demand introduction through a plug-in, whether post-compilation is used or not, and allows developers to simply modify the Babel configuration without having to drastically change the import path in the source code.

conclusion

With that said, we explored a little bit of WebPack-based compilation optimization. Here are some “best practices” for compiling and packaging applications using WebPack:

Post compile + import on demand

With babel-preset-env, Babel-plugin-transform-modules, the development experience and benefits are better.

Welcome to the Didi FE blog: github.com/DDFE/DDFE-b…