Webpack5 Built-in cache scheme exploration

background

Compilation and translation technologies like Babel, TypeScript, VueLoader, and Terser are becoming increasingly popular, and Webpack compilation times are ballooning. To optimize compilation speed, the community has two main solutions:

1. By caching loader processing results to the local disk, the secondary compilation is accelerated

2. Precompiling DLLS allows WebPack to skip compilation of some modules to speed up compilation

These solutions solved the problem of slow compilations to some extent, but with them came stacks of configurations that severely affected the Webpack experience and even created a “new profession” of “Webpack configuration engineers.”

At the same time, there are some new compilation technologies emerging in the community, such as Snowpack, which uses the browser’s native ES Module to compile in O(1) time, which is hardly appealing to those who are “old Webpack”.

But it was a challenge for Webpack, and Webpack5’s built-in cache solution finally came out in the face of a user market that could be cannibalized.

Persistent cache and compile cache

The long-term cache is a browser-level cache. Webpack uses Optimization’s splitChunks and runtimeChunk to provide stable hash names for compiled and output files so that the browser can reuse the cache effectively and safely for a long time to accelerate page loading.

A compile cache is a compile-time cache, and Webpack speeds up compilation by caching the results after the first compile and reusing the cache during subsequent compilations.

Webpack5’s built-in cache, as discussed in this article, refers to compilation caches.

Webpack4 caching scheme

When using Webpack4 and previous versions, we noticed something like this:

• Code hot updates quickly

• NPM run slow star

• NPM run build slowly

The essence of this phenomenon is that Webpack4 has a cache at runtime, but the cache is only in memory. So, once the Webpack runner is shut down, these caches are lost. This results in no cache available at all for our NPM run start/build.

So, the solution is to persist the cache generated during Webpack compilation to local disk, database, or cloud. There are two things involved: what to persist and where to persist.

Webpack already has a caching solution, but it’s not perfect enough to support persistence. From today’s point of view, it is obvious that we should go straight to the Core Webpack code, add persistent caching, and use one caching solution to solve all problems.

However, the community didn’t seem to understand the “what to persist” problem at the time, and instead of working on the Webpack core code, they chose to solve it externally. As a result, cache-loader, DLL and other technologies appeared, which solved the problem to a certain extent, but introduced too much complexity.

Webpack5 caching scheme

In fact, the question of “what to persist” is obvious from the start: it’s the cache that exists in memory at the Webpack runtime, not a loader artifact, let alone a DLL. Therefore, Webpackage 5 provides a set of persistent abstractions with several implementations:

• IdleFileCachePlugin: Persist to local disks

• MemoryCachePlugin: Persist to memory

Depending on the environment in which Webpack is run, MemoryCachePlugin is still used for dev development and IdleFileCachePlugin is used for build.

Webpack5 directly from the internal core code level, unified persistent cache scheme, effectively reduce the complexity of cache configuration. In addition, since all modules processed by Webpack will be cached, our NPM run start/build secondary compilation will be much faster than cache-loader, and the DLL can be eliminated.

In fact, Webpack5’s built-in caching scheme is better than cache-loader in terms of performance and security:

1. Performance: Because all modules processed by WebPack are cached, cache coverage is much higher

2. Security: Because cache-loader uses mtime-based cache validation mechanism (see: juejin.cn/post/684490…) In a CI environment, the cache often fails, but Webpackage 5 addresses this problem by switching to a cache validation mechanism based on file content ETAG.

Specific use Webpack5 configuration website has been given: webpack.js.org/configurati…

FAQ

1. Can Webpack4 use the Webpack5 caching scheme?

   You can use the hard-source-webpack-plugin, which is a plug-in based on the above Webpack5 implementation idea. But as you can also see in # Troubleshooting, there can be some pitfalls.

2. Can libraries like Snowpack and Vite replace Webpack?

   Given Webpack’s current ecosystem and the game-changer level module-federation capabilities Webpack5 will bring, I don’t think so, but this technology based on the browser’s native ES Module will probably be adopted by Webpack.