Look at webPack’s hash strategy from the source code

Recently, I found that different hashes are generated when the same code of WebPack is packaged on multiple machines. There is no article explaining the hash generation strategy in depth in the community, so I went through the WebPack source code and found that the absolute path of the project is included in the hash generation. Finally, I wrote a WebPack plug-in to solve this problem. This article mainly explains the use and principle of hash and how to solve the hash inconsistency of multiple machines.

Webpack’s hash strategy

It is well known that the browser will cache the static resource after it is loaded for the first time. The same resource will not be requested if the cache has not expired. So how to notify the browser of the change of the resource when it is updated? The hash of resource file names is designed to solve this problem; Webpack is now the mainstream build tool on the front end, so this article focuses on the hash strategy for webPack’s post-build file names; Webpack is divided into hash, Chunkhash, and Contenthash hashes. The use and principles of the three hashes are described in the following sections.

hash

One of the most commonly used types of hashes is when building with WebPack. After building with WebPack, the hash of js and CSS output files is the same for the entire project. For example, a project has six components. Components 1, 2, and 3 need to output a set of JS and CSS files as chunk, and components 4 and 5 need to output a set of JS and CSS files as chunk. The configuration of Webpack is as follows:

output: {
    path: path.resolve(__dirname, OUTPUT_PATH),
    filename: '[name].[hash].js'/ / usehash
    publicPath: '/dist/webpack/'
  }
Copy the code

The hash of the first set of JS and CSS files produced by webPack is the same, and the hash of the second set is the same as the hash of the first set. Here is the hash effect in the project:

So if one file is modified, the hash of all output files will change. Therefore, it has the disadvantage that once a single file is modified, the entire project’s file cache is invalidated.

chunkhash

Chunkhash has a smaller impact than hash. When chunkhash is used, each chunk corresponds to a hash. After a source file is modified, only the hash of the output file of the chunk in which the source file resides changes. For example, a project has six components. Components 1, 2, and 3 need to output a set of JS and CSS files as chunk, and components 4 and 5 need to output a set of JS and CSS files as chunk. The configuration of Webpack is as follows:

output: {
    path: path.resolve(__dirname, OUTPUT_PATH),
    filename: '[name].[chunkhash].js', // Use chunkhash publicPath:'/dist/webpack/'
  }
Copy the code

The two sets of hashes produced by the webpack are different, but each set has the same internal HASH for js and CSS. Here is what chunkhash looks like in the project:

contenthash

When using mini-css-extract-plugin, you can also use contenthash to hash the file. Contenthash has a smaller impact than Chunkhash. The JS and CSS output files in each chunk generate a hash independently. When the JS source file in a chunk is modified, only the HASH of the JS file output by the chunk changes. For example, a project has six components. Components 1, 2, and 3 need to output a set of JS and CSS files as chunk, and components 4 and 5 need to output a set of JS and CSS files as chunk. The configuration of Webpack is as follows:

output: {
    path: path.resolve(__dirname, OUTPUT_PATH),
    filename: '[name].[contenthash].js', // Use contenthash publicPath:'/dist/webpack/'
  }
Copy the code

The two sets of hashes output after the webPack is built are different, and each set has different Hashes for JS and CSS inside. Here is how contenthash looks in the project:

The differences between the three hashes

How WebPack hashes

Webpack hashing is implemented using the crypto encryption and hashing algorithm. Webpack provides hashDigest (the encoding used to generate the hash, which defaults to ‘hex’), hashDigestLength (the prefix length of the hash digest, Default is 20), hashFunction (hash algorithm, default is’ MD5 ‘), hashSalt (an optional value to add salt), etc. The following three hash generation strategies are described in turn.

Webpack’s three hash generation strategies are generated based on the source code content, but the source code has been packaged by WebPack to run in the WebPack environment, including the absolute path of each source file; Webpack generates a _buildHash for the corresponding module in the build phase based on the source code (which is then used to generate the hash for the module). You can see that the source code contains the absolute path as shown below.

Webpack generates three kinds of hashes in the SEAL phase, and determines which hash to use according to the output configuration. Webpack generates hashes by running the compilation.createHash function.

Hash and Chunkhash generation

The next section focuses on the hash generation process, which includes the chunkhash generation process. The first step in creating a hash for WebPack is to take all modules under the Compilation and generate a new hash value from the _buildHash generated by all modules in the build phase. Then, all chunks are obtained, and the hash of module contained in the chunk is used as the content to generate the hash of chunk. This hash is the hash value required when configuring chunkhash. The final hash is generated by taking the hash of all chunks as the content, as shown in the source code below.

// Non-source code, code has been deletedcreateHash() {// Generate a new module based on the _buildHash generated in the build phasehashValue const modules = this.modules;for (let i = 0; i < modules.length; i++) {
			const module = modules[i];
			const moduleHash = createHash(hashFunction); module.updateHash(moduleHash); } / /cloneneeded as sort below is inplace mutation const chunks = this.chunks.slice(); // Create one for each chunkshash
		for (let i = 0; i < chunks.length; i++) {
			const chunk = chunks[i];
			const chunkHash = createHash(hashFunction); try { chunk.updateHash(chunkHash); // For all modules contained in chunkhashGenerate one as contenthashValue of the template. UpdateHashForChunk (chunkHash, chunk, enclosing moduleTemplates. Javascript, enclosing dependencyTemplates); chunk.hash = chunkHash.digest(hashDigest); // put all chunks on chunkshashAs content hash. Update (chunk.hash); / / generated contentHash enclosing hooks. ContentHash. Call (the chunk); } catch (err) {}} // generatehash
		this.fullHash = hash.digest(hashDigest);
		this.hash = this.fullHash.substr(0, hashDigestLength);
	}
Copy the code

Contenthash Generation process

Contenthash is generated by the mini-css-extract-plugin and JavascriptModulesPlugin plugins, unlike the previous two types of hash generation. Mini-css-extract-plugin is a webPack build-in plug-in that separates CSS module types from each other and generates a separate hash for CSS files. It generates chunkhash from the hashes of all modules of type CSS /mini-extract in the chunk.

// mini-css-extract-plugin for the CSS filehashGeneration of hook function compilation. Hooks. ContentHash. Tap (pluginName, the chunk = > {const {outputOptions} = compilation; const {hashFunction, hashDigest, hashDigestLength } = outputOptions;
        const hash = createHash(hashFunction); // Extract all modules of type 'CSS /mini-extract' from chunkhashGenerated as contenthash
        for (const m of chunk.modulesIterable) {
          if (m.type === MODULE_TYPE) {
            m.updateHash(hash); } } const { contentHash } = chunk; ContentHash [MODULE_TYPE] = hash. Digest (contentHash[MODULE_TYPE] = hash.hashDigest).substring(0, hashDigestLength);
      });
Copy the code

When the contentHash hook is triggered, the contentHash event registered by the JavascriptModulesPlugin plug-in is invoked to generate hashes for all modules of function type in the chunk.

/ / JavascriptModulesPlugin plug-in generated contentHash hook function for js compilation. The hooks. ContentHash. Tap ("JavascriptModulesPlugin", chunk => { // ... There are cuts herefor (const m of chunk.modulesIterable) {
						if (typeof m.source === "function") {
							hash.update(m.hash);
						}
					}
					chunk.contentHash.javascript = hash
						.digest(hashDigest)
						.substr(0, hashDigestLength);
				});
Copy the code

Multi-machine build scheme

Although WebPack hash brings us great convenience, it also has some disadvantages; Webpack’s three hash strategies all rely on the module’s _buildHash value, and the _buildHash value depends on the module’s source file content and absolute path. Therefore, the same source code may not produce the same hash value on different machines unless the project path is identical on both machines. If there are multiple machines online to build and deploy the same project, the hash value may be different, resulting in a 404 appearance when accessing JS or CSS.

If you want to deploy hashes on multiple machines, here is the strategy for generating hashes on multiple machines:

• The hash generation of JS source file (excluding node_modules) in the project is not the source code encapsulated by WebPack, but the content of JS source file, to solve the hash inconsistency caused by the absolute path of the source code encapsulated by Webpack.
• The CSS source files in the project (excluding node_modules) hash to generate the CSS source code used, as before, there is no path problem.
• The hash generation of the CSS under node_modules uses the relative path of the CSS file plus the NPM package version number to resolve the sourceMap path problem of the style file in node_modules.
• The hash generation of js under node_modules uses the relative path of the JS file plus the version number of the NPM package to resolve the hash inconsistency of some NPM packages in node_modules.