How long does it take to start a large old project?
In my work, I encountered several large old projects, which started very slowly, usually 2-3 minutes.
Here’s a more dramatic one:
It’s really torture to start a project like this.
Two, prepare knowledge
(a) esbuild
The reason I mention esbuild is because Vite uses esBuild builds, and WebPack can also be speed-up using esbuild-Loader.
ESbuild is a WebPack-like build tool. It builds dozens of times faster than WebPack.
Why so fast?
- Js is single-threaded serial, esbuild is a new process, and then multi-threaded parallel, give full play to the advantages of multi-core
- Go is pure machine code and is definitely faster than JIT
- Optimizes the build process by not using AST (also brings some disadvantages, some babel-plugins that handle code through AST don’t have a good way to transition into ESBuild, If your project uses babel-plugin-import, or some custom babel-plugin, it will not work.)
3. Webpack performance analysis
1.webpack-bundle-analyzer
Analyze package sizes using the Bundle-Analyzer analysis tool:
Start with admin-CRM-web, which has more than 80 routes:
The startup time is one and a half minutes, occupying 6.77G of memory, and the total amount of all chunks is 100MB
Even if you manually select a route, it still takes about 10 seconds.
2, speed – measure – webpack – the plugin
Use speed-measure-webpack-plugin to analyze and measure the time spent by each plug-in and loader:
Third, optimization plan
1. Browser-driven route cutting compilation
If the previous project was to be launched, our colleagues in the company encapsulated the plug-in by themselves to reduce the packaging route during construction and let us make manual selection, which was not convenient enough:
1. Too many routes are inconvenient to choose
2, select wrong, need to close and restart again select
Think about what Vite does. When vite starts up, it doesn’t do any compilation. When you visit the page, you request resources from the browser’s ESM, and Vite then compiles the resources you need on demand and returns them to the browser.
In order to improve the user experience and prompt the development efficiency, I wonder if I can not select route when I start the page, and then tell Webpack that I need to pack the page when my browser visits the page, and then webpack the current page?
My idea is to package empty routes during Webpack development, then inject scripts into the page, listen for route changes, send requests to Webpack, listen for requests through webpack-dev-server’s before hook, get parameters for route changes, and then write routes dynamically.
Here is my implementation:
Points that can be improved:
1. The page is blank when you package. You can add loading or package progress output
2. You need a.routes.ts file that is the same as your routing file but can be rewritten with Babel to match in importDecleration and filter in memory
UglifyJsPlugin start PARALLEL
Multi-core processor is used for parallel processing. The principle is to enable the new child process using NodeJS.
But then I upgraded the WebPack version to 5 and used the Terser-plugin, which is the same.
3. DLL plug-ins
DLL does the same thing as External, except that local dependencies are pre-packaged and then excluded, and if the dependencies change, they need to be repackaged.
DLL plug-ins can be used not only in production but also at build time, so they can be left alone. So I put the generated DLL files in the local directory,
And set the
Then introduce it on the page
4. Upgrade WebPack
Check for expired packages:
Upgrade webPack-related packages:
Install webpack after upgrade – CLI:
5. Use esBuild
However, esbuild-loader also has disadvantages:
(1) Decorators are not supported
OneOf uses a single loader for a single file. When returen is false, babel-loader + TS-loader is used to package files.
Determine if the input file has a decorator:
function hasDecorator(fileContent, offset = 0) {
const atPosition = fileContent.indexOf('@', offset);
if (atPosition === -1) {
return false;
}
if (atPosition === 1) {
return true;
}
if (["'", '"'].includes(fileContent.substr(atPosition - 1, 1))) {
return hasDecorator(fileContent, atPosition + 1);
}
return true;
}
Copy the code(2) Loading on demand is not supported
Development environment using esbuild-loader, ANTD can be fully introduced
(3) Can only be packaged as ES6
However, our projects are usually packaged as ES5, so there will be conflicts, so we can set the TS-Loader configfile in the development environment
8, exclude to include
This allows the Loader to be applied to the minimum number of necessary modules
Instead of
Use tools as little as possible
Each additional Loader /plugin has its own startup time.
For example, some plug-ins are only needed in the production environment, and the development environment can not use such as BannerPlugin, ExtractTextPlugin, UglifyJsPlugin and so on
10, Devtool
It is important to note that different devTool Settings can lead to performance differences.
- “Eval” has the best performance, but it doesn’t help you translate code.
- If you can live with a slightly worse map quality, you can use the cheap-source-map variant configuration to improve performance
- Incremental compilation using the eval-source-map variant configuration.
In most cases, the best option is eval-cheap-module-source-map
The development environment avoids additional optimization steps
Webpack performs additional algorithmic tasks to optimize the size and loading performance of the output. These optimizations are suitable for small code bases, but can be costly in large code bases:
12. The output of the development environment does not carry path information
Webpack generates path information in the output bundle. However, in projects where thousands of modules are packaged, this can cause performance pressure for garbage collection. Turn off in options.output.pathinfo setting:
13. Optimization of ts-Loader in development environment
You can pass in the transpileOnly option for the Loader to reduce build time when using TS-Loader. With this option, type checking is turned off. If you want to open type checking again, please use the ForkTsCheckerWebpackPlugin. Using this plugin moves the checking process to a separate process, speeding up TypeScript type checking and ESLint inserts.
Happypack or Thread-loader
Webpack3 can use happypack and Webpack5 uses thread-loader, but there are some limitations:
- These loaders cannot generate new files.
- These Loaders cannot use custom Loader apis (that is, they cannot be customized through plug-ins).
- These loaders cannot get the webpack configuration.
Each worker is an independent Node.js process, and its overhead is about 600ms. It also limits data exchange across processes.
Please use this loader only for time-consuming operations!
You can see several loaders that take time, so configure:
Encountered another pit:
The main reason is that JS files in the project use THE SYNTAX of TS, so I have to use TS-Loader to convert JS files, which is not rigorous, but there is no way for the old project.
15, noParse
If some third-party modules do not have AMD/CommonJS specification versions, noParse can be used to identify the modules, so that webPack will introduce these modules, but not transform and parse, thereby improving the build performance of WebPack, such as jquery and LoDash.
16, externals
The use of DLLS excludes some third-party dependencies, but there are also some external dependencies that can be excluded for each specific project, and externals can be used without recreating packaged DLLS. \
17. Optimize the resolve.extensions configuration
If an import statement does not have a file suffix, webPack will automatically apply the suffix to resolve. Extension to ask if the file exists.
- The resolve.extensions list should be as small as possible, and don’t put anything in the suffix attempts list that isn’t possible in the project.
- File suffixes with the highest frequency should be placed first in order to exit the search process as quickly as possible.
- When writing import statements in source code, use suffixes whenever possible to avoid the search process.
Removed. Web. Js
18. Enable caching
For example, if babel-loader is enabled on the cache, the cache will be read on the second build:
Webpack cache does not need to be configured. It is enabled in the default development environment.
HtmlWebpackPlugin, TerserPlugin, eslint-loader, babel-loader all add caching
In addition to these, there are cache-loader, HardSourceWebpackPlugin can be enabled caching. I haven’t done that yet, but you might want to try it.
19. Webpack compiles twice
When using Webpack3, there was often a problem with compiling twice, which seemed to be a problem with the lower version of the HTMl-webpack-plugin, but after I upgraded, the problem was gone.
Fourth, hit the pit
Pit 1:
Replace the META in the DLL’s JSON with buildMeta.
Pit 2:
JsonpFunction is replaced with chunkLoadingGlobal
Pit 3:
Replace loader with use.
Pit 4:
The reason is because of the custom WebPack plugin:
Webpack API changes should be rewritten as
5 the pit:
will
To:
6 the pit:
ContentBase to static
Pit 7:
DisableHostCheck allowedHosts instead: ‘all’
8 the pit:
Instead of logging: ‘warn’
Pit 9:
Webpack.optimize.Com to splitChunks monsChunkPlugin upgrade
10 the pit:
Change devServer’s before to onBeforeSetupMiddleware
Pit 11:
The plugin in the babel.rc file does not allow arrays to be configured
Pit 12:
The main reason is that the Babel preset has been changed to @babel/preset- ES2015
Install the new default package instead and overwrite the configuration items:
Pit 13:
Install webPack globally – CLI
Pit 14:
Remove the Babel configuration and change @babel/preset- ES2015 and @babel/preset-stage-0 to @babel/preset-env
Pit 15:
I’m going to write it this way
Pit 16:
to
Pit 17:
Use mini-CSs-extract-plugin instead of extract-text-webpack-plugin.
To:
18 the pit:
Upgrade ossPlugin.
Pit 19:
Install the @babel/plugin-proposal-decorators plugin.
20 the pit:
Since babel-plugin-import is used, the ts-import-plugin is redundant and can be removed.
Pit 21:
These errors are mainly due to the use of TS functions in JS, so
Js files should also use ts-loader
22 the pit:
Type assertions can only be used in TS files
Pit 23:
Zlib cannot be found because webpack5 used to have some polyfills of NodeJS built in and now needs to install the configuration itself.
Pit 24:
Or because the decorator’s conversion plug-in is missing descriptor,
The main reason was the way decorators were used in our project
The official way to use it is:
In our project we decorated the Propotery of the class, not the Class Method, so we didn’t end up converting the decorator plugin there at all.
There are only two solutions, one is to change the code in the project to be written in the same way as the official code, or to take the old decorator plug-in for the propetry section of the class and create a new custom plug-in.
Pit 25:
Uglifyjs webpack – the plugin to terser webpack — the plugin
Pit 26:
Install the Assert configuration
27 the pit:
Install and configure stream-Browserify
28 the pit:
Join ignoreOrder
29 the pit:
After using Esbuild, the speed was increased several times,
But the reslove module configured in Webpack could not find a reference
To fix this, you need to create an index.js file in the SRC file under crm-comps.
29 the pit:
There is a problem with the main setup. In the project, Babel was used to change the imported directory. Now using esbuild will cause an error
To:
5. Optimization effect
It all started in less than 10 seconds. Remember how long it took to start an entire project before optimization? A portion of 10 seconds, a 90% improvement in time!
Memory usage is reduced by 1.7GB