Author: Kun Chen

preface

Esm-based build tools such as Snowpack/Vite have emerged to eliminate the need to build a complete bundle for engineering builds of projects. Many of us feel that the time is coming when we no longer need to pack tools. With the capabilities of the browser ESM, some code can be run without a build. For Webpack, this wave of ESM enthusiasm in the community has pushed the speed of WebPack compilation to the forefront. The v5 version of Webpack also makes a lot of efforts for the performance of compilation, in addition to providing physical cache optimization, but also provides Module Federation solution, which brings a lot of imagination to our upper application practice.

Webpack tends to be a one-size-fits-all build tool, but now we have more options tailored to the business.

ESM: ES Modules is the official JavaScript standardized module system. Compared to module specifications such as CommonJS and AMD, the latest browsers support module functionality natively without additional packaging

Why you need to pack

Many times in JavaScript programming, we are modifying variables, in a complex project development process, how to manage the function and variable scope, it is particularly important. The modularity of JavaScript gives us a better way to organize and maintain functions and variables. We are familiar with JavaScript modules in addition to the above ESM, there are CJS, AMD, CMD, UMD and other specifications. Under the background of NPM ecological development, CJS module is unavoidable in the development process. However, since browsers cannot execute cJS-BASED packaged modules directly, packaging tools such as WebPack have emerged.

For early Web applications, packaging modules can handle BOTH JS modularization and multiple modules can be packaged to combine network requests. Using such build tools to package projects is a good option. Today, almost all major browser versions support ESM, and the performance issues associated with concurrent network requests are not as significant as they used to be with the popularity of HTTP/2. Based on the current experience, native ESM seems to be far faster to build during development than packaging tools like WebPack.

An exploration of ESM building tools

Using the ESM

<script src="index.js" type="module"></script>
Copy the code

Type =”module” tells the browser that the script is using ESM mode. The browser will build a dependency graph and complete the process of module search, parsing, instantiation and execution with the help of the browser’s native ESM capabilities.

Why do fast

Why is snowpack/Vite, an ESM-based build tool, so much faster than WebPack when it was built

Two core features:

  1. First, their build complexity is very low, changing any component is a single file compilation, and the time complexity is always O(1).
  2. With the power of ESM, modularity is handed over to the browser, there is no problem with reloading resources, and if JSX or typescript syntax is not involved, it can even be run without compilation

The build process

If only the source code to the browser execution, is not to meet the demands of most projects, source code is usually we are directly import third-party modules, in addition to the import style, resources, including source code development process using the latest ES syntax, JSX, TS syntax, these can not run directly in the browser.

The build tool addresses these issues according to type and requirement. Here is a simplified overview of the process:

Import statement processing

For ESM modules, the first thing to deal with is the import statement, and the following situations are common during project development:

  • Import third dependencies, such as import React from ‘React ‘;

  • Import img from ‘./img.png’;

  • import css import './index.css'

The import dependence

Snowpack converts all dependencies to /web_modules/*.js

import React from 'react'; // convert import React from '/web_modules/react.js';Copy the code

If all dependencies in NPM can be exported to the standard ESM module, this step can be as simple as intercepting all web_modules requests and returning the ESM module under its node_modules. However, the reality is that many of the dependencies in the NPM ecosystem are not yet supported, so most current practices are pre-packaged at the initial startup of the project to complete the conversion of CJS/UMD into ESM operations.

Configuration logic also supports active filtering of generated ESM modules, reducing the cost of pre-packaging

Import images

Snowpack rewrites the import statement for this type of resource when importing image resources in the hope of eventually returning the URI of the static resource:

import img from './img.png'; // import img from './ img.pg.proxy.js ';Copy the code

In the img.pg.proxy. js file, the corresponding file address can be exported by default:

// img.png.proxy.js

export default '/dist/assets/img.png';
Copy the code

When generating *.proxy.js, the tool can copy the resources to the specified output path.

import css

The rewriting rules for style imports are similar to those for images, except that the content in *.css.proxy.js is generated. The idea is also to import the CSS into JS modules, for CSS, processing through the link way to introduce, but also through the style tag injection, the CSS module proxy rules have become quite clear:

Const code = ${json.stringify (code)}; const styleEl = document.createElement("style"); const codeEl = document.createTextNode(code); styleEl.type = 'text/css'; styleEl.appendChild(codeEl); document.head.appendChild(styleEl);Copy the code

If you want to start CSS Modules, the *.module. CSS file in Snowpack will enable the CSS Module function. Compared to the above method of inserting labels, add the style class name corresponding to the class name of the export, namely:

. // let json = ${JSON.stringify(moduleJson)}; let json = { "test": "App--Test--3kX9Z4E"} export default json;Copy the code

In addition to rewriting import types mentioned above, there are json, less, sass and other files processing logic is essentially similar, all through the import proxy to the newly generated file, and adding a specific script to complete the transformation of the final ESM module, to support the browser to handle loading.

Module Federation application

ESM is fast because it doesn’t need to analyze node_module dependencies in source code, merge, split, and package them in addition to compiling, as WebPack does.

Both the authorities and the community have gone to great lengths to design various solutions to optimize performance, Cache-loader, Thread-loader, dllPlugin, Babel cacheDirectory, hard-source-webpack-plugin, etc. And there are certain use costs. Until the emergence of WebPack 5, its long-term caching capability can only compile related files in the dependency tree according to the dependency relationship when file changes are detected, and greatly improve the build speed through the optimized build cache and resolver cache. Another feature, Module Federation (MF), brings a new way to collaborate with WebPack-based development, making it easier to reuse modules between different build tasks.

MF’s design motivation is to enable different teams to collaborate on one or more applications. This kind of assistance model is similar to the micro front end development model that was in full swing last year.

MF solution can split an application and export different modules, and the underlying three-party libraries that the modules depend on can also be shared. With this capability we can build a common dependency for the application ahead of time, reducing compilation and packaging volume.

The core usage

First, let’s understand the two core concepts in MF:

  • Host: Provides the Remotes option to apply the Expose module found in other Remote apps

  • Remote: Provides an object reception option and modules for other applications

With MF’s capability, each application can reference and be used by other applications, achieving bidirectional sharing

Usage:

// webpack const { ModuleFederationPlugin } = require("webpack").container; . plugins: [ new ModuleFederationPlugin({ name: 'remoteRuntime', // must have a unique ID, as the output module name, when using remotes with ${name}/${expose} : ['remote'], // Optional, representing the objects that are sold as hosts: {// Optional, representing the attributes that are sold as remote objects './ComponentA': './ SRC /components/A',}, shared: ['react', 'react-dom'],Copy the code

In addition to the introduction of the remoteEntry (module dependency) script generated by Host, the code level needs to be modified accordingly when the application of an object reception module is used:

Import ${name}/${expose} import ComponentA from 'remoteRuntime/ComponentA';Copy the code

Perform logical

To make use of the webPack 5 Module federation capabilities, in addition to the engineering configuration, you need to make changes to the source code:

// index.js import('./bootstrap'); // bootstrap.js import React from 'react'; import ReactDOM from 'react-dom'; import ComponentA from 'ComponentA'; .Copy the code

Before the rendering logic is executed, a layer of bootstrap import logic needs to be added. In essence, the main logic can be executed after the asynchronous loading is completed. The core process is as follows:

The code core execution flow is changed as follows:

  1. We load index.js first, which is usually the main bundle of the application
  2. It’s going to be loaded in the main bundle,__webpack_require__ (". / boostrap js ")
  3. The Boostrap module will contain various chunk-dependent information
  • Based on the overridables logic, the system determines the shared information that the chunk depends on and loads the information
  • Load external dependencies using the remotes logic, information already provided in remoteEntry

  1. After all application dependencies are loaded, the final application logic is executed

Depend on the pull away mode

With that in mind, let’s go back and see if it’s possible to have a WebPack project that doesn’t pack dependencies like the ESM model, with all the three-way dependencies prepped and directly dependent on these precompiled modules when the main application starts.

With MF’s ability, all third-party dependencies can be introduced as remote dependencies. In the practice of ICE, the use of MF scheme and physical cache of WebPack 5 has indeed brought great speed to the construction of the project development. The core implementation logic of the scheme is as follows:

  1. First of all, the project runtime dependencies are exported through the reception capability, and the construction of remoteEntry and related remote modules is completed
  2. The project turns on MF capabilities and relies on remote built in
  3. Convert runtime dependencies in a project to remote module loading mode with Babel capability, i.e. Import XX fromIn the form of {expose}

After the above processing is complete, the bundle after the project starts will no longer bundle together the remote runtime dependencies that have been packaged.

Before optimization:

After solution optimization is enabled:

The above schemes have been practiced in different types of businesses. Please contact 👏 to discuss the application scenarios

Comparison of similar schemes

dll

The tripartite dependencies of an application are compiled into DLLS. Every time the application dependencies change, the application needs to be rebuilt and cannot be loaded on demand. The DLL cannot be shared by multiple applications, which lacks dynamic characteristics.

externals

In the externals scenario, dependencies are built into files, and the application needs to declare which external modules are referenced when it is packaged. Externals cannot be loaded on demand, and the dependency relationship and loading order of scripts need to be maintained.

summary

Last year, snowpack/Vite’s engineering ecology and build customization capabilities were still far behind WebPack, but with the domestic focus on ESM ecology, more and more build tools began to try to develop in the way of ESM.

The ESM development mode largely solves the problem of startup speed in dev development, and also provides a way to precompile when many modules are not exported from ESM. Tools like Snowpack also provide webPack-based plug-ins in production build mode, allowing developers to apply the final product without too much burden. It is an incremental approach, but certainly not a permanent solution.

Webpack’s slowness aside from the impact of having to analyze dependencies and package them into a bundle, it can be very time-consuming to use capabilities like Babel compilation and Sass-Loader. This is why esBuild is the default compilation tool for the most popular ESM modules in the community. It compiles fast enough. Webpack also optimizes the physical cache to improve build performance, especially with secondary builds and hot updates providing a significant build speed boost.

The capabilities webPack provides are more like an enterprise-level solution, providing sufficient hooks and capabilities for developers to customize any source/build node. ESM takes advantage of the browser’s module loading capability, does not parse module dependencies, and takes fast internal implementation logic as the primary consideration. If it can be given to the browser to run directly, it will not compile. For some application development frameworks of ICE or the community, most of the solutions will combine engineering and runtime capabilities to simplify the development cost in order to reduce the developer’s cognition and development cost, and the logic processing depends to a certain extent on the webpack ecosystem. Then how to combine these capabilities with ESM development mode to help developers get the ultimate improvement in engineering construction experience and source development experience will be another outlet to focus on the next.