One, foreword

Modularity has been around for a long time in many high-level languages such as Java, Ruby, Python, and even C with similar modularity, such as include statements introducing header files, various libraries, and so on. In the front end, JavaScript is the main language, and it was not designed to be modular. With the development of the Web, JavaScript has become more and more important, and the following major problems have emerged:

  • Code is hard to maintain
  • Scope contamination
  • Variables cannot be uniquely identified
  • .

This is a big pain point! But the majority of software engineers are not vegetarian, so solutions like AMD, CMD, ES Module, CommonJS have sprung up.

Now that AMD and CMD are out of sight, there are only two Module specifications we see: ES Module and CommonJS, the former for ECMAScript and the latter for Node.

The CommonJS specification is a very big concept. Like the ECMAScript specification, it is a whole language specification. Modularity is just one of the larger specifications, and I’m sure many people get confused.

The CommonJS specification implements the following specification:

  • ECMAScript (different versions have different support)
  • The module
  • binary
  • Buffer
  • I/O streams
  • .

I’m sure you can understand this, but I’ll introduce the CommonJS module specification that I’ve been studying.

Second, the content of the specification

It is mainly divided into three parts: module reference, module definition and module representation.

2.1 Module Reference

Node modules are divided into two types: core modules and file modules, and modules are introduced through the require method. The former is a built-in module in Node, while the latter is usually a user-defined module. The custom modules mentioned below are also file modules, just for clarification.

The code is as follows:

// Introduce the 'HTTP' built-in module
const http = require('http')

// Import the file module
const sum = require('./sum')

// Introduce third party package 'koa', which is a custom module
const koa = require('koa')
Copy the code

The basic function of the require command is to read and execute a JavaScript file and then return the exports object of that module. If no specified module is found, an error is reported.

2.2 Module Definition

In the CommonJS module specification, a file is a module, and variables or functions in a module are exported through module.exports and exports.

The code is as follows:

// Exports a 'sum' function through exports
exports.sum = (x, y) = > x + y;

Exports a 'sum' function through module.exports
module.exports = (a, b) = > a - b;
Copy the code

For convenience, Node provides an exports variable for each module, pointing to module.exports. Is equivalent to:

var exports = module.exports;
Copy the code

Exports will be disconnected from the address of module.exports if the exported variable type is a reference type such as a function. Module. exports is the ultimate export variable.

exports = function() {... };Copy the code

The graph looks something like this:

Similarly, if you export an object or function directly using module.exports, you can export a variable or function from the same address again.

// Store an A variable in the previously empty object
exports.a = function() {}

Exports a reference type variable directly through module.exports
// The previously exported variable is invalid

module.exports = {... }Copy the code

2.3 Module Identification

The module id is a parameter in the require method, which is the path to the imported module file. It may not have a suffix, but it must conform to the small hump naming convention.

In the module reference above, HTTP,./sum, and koa are module identifiers. There are the following categories:

  • The core module
  • In order to.or.Start the relative path module
  • In order to/Start the absolute path module
  • Custom modules, common as third-party packages

3. Module loading process

Importing modules from Node involves four main processes:

  • Cache loading
  • Path analysis
  • File location
  • Compile implementation

Let’s look at how they work.

3.1 Cache Loading

Whether it is a built-in module or a file module, it is compiled and executed and placed in the cache after the module is loaded for the first time. So that when the module is loaded again, it will directly go to the cache to find the corresponding module.

Unlike file modules, built-in modules are directly compiled into binary executables during Node source compilation. The core modules are loaded from memory and cached when the Node process is started. So the loading of built-in modules skips the steps of file location and compilation execution and takes precedence over file module loading.

Cache. If you want to remove a module’s cache, you can write it like this.

// Delete the cache for the specified module
delete require.cache[moduleName];

// Delete the cache for all modules
Object.keys(require.cache).forEach(function(key) {
  delete require.cache[key];
})
Copy the code

Note that the cache identifies modules by absolute path. If the module name is the same but stored in a different path, the require command will reload the module.

3.2 Path Analysis

Path analysis mainly analyzes module identifiers. Different rules are used to analyze paths based on different types of module identifiers.

The following is the comparison of module loading speed:

Core Module > File Module > Custom module.

Core modules are compiled into binaries when Node starts, so they load fastest. File modules with.,.. , / path identifier, specific identification of the location of the file, so the module loading speed is second only to the core module. Custom modules are the slowest of the three, for reasons explained below.

It is worth noting that if a custom module and a core module have the same name, the custom module will not be loaded because the core module takes precedence over the custom module.

How does Node find and load file modules and custom module paths? I’ll start with a very special object module.

3.2.1 Module Object Description

Node provides a Module builder function inside. All modules are instances of Module. Inside each module, there is a Module object that represents the current module.

Module

function Module(id, parent) {
  this.id = id;
  this.exports = {};
  this.parent = parent;
  // ...
Copy the code

To test, I set up a project structure:

sum.js 

module.exports = (x, y) = > x + y;
Copy the code

main.js

const sum = require('./sum')
const result = sum(1.2);

module. Exports = the result;console.log(module);
Copy the code

We print the module in main.js, which holds all the information about the current main.js module:

Module {
  id: '. '.path: 'e:\\web\\font-end-code\\Node\\01-Commonjs'.exports: 3.parent: null.filename: 'e:\\web\\font-end-code\\Node\\01-Commonjs\\main.js'.loaded: false.children: [
    Module {
      id: 'e:\\web\\font-end-code\\Node\\01-Commonjs\\sum.js'.path: 'e:\\web\\font-end-code\\Node\\01-Commonjs'.exports: [Function].parent: [Circular],
      filename: 'e:\\web\\font-end-code\\Node\\01-Commonjs\\sum.js'.loaded: true.children: [].paths: [Array]}],paths: [
    'e:\\web\\font-end-code\\Node\\01-Commonjs\\node_modules'.'e:\\web\\font-end-code\\Node\\node_modules'.'e:\\web\\font-end-code\\node_modules'.'e:\\web\\node_modules'.'e:\\node_modules']}Copy the code

Briefly describe each of its properties.

  • Id: The module’s identifier, usually the module’s file name with an absolute path.

  • Path: absolute path of the current module.

  • Export: indicates the output value of the module. I’ve derived a 3 here.

  • Parent: Returns an object representing the module that called the module. Return null if no

  • Filename: indicates the filename of the module with an absolute path.

  • Loaded: Returns a Boolean value indicating whether the module has completed loading.

  • Children: Returns an array representing other modules used by this module.

  • Paths: Array of absolute paths that the current module looks for. It follows certain module path query rules.

We can use the parent property to determine if the current file is an entry file:

if(!module.parent) {
	// do something
} else {
	// export something
}
Copy the code

After we understand the Module object, it is very helpful to analyze the following module path query rules.

3.2.2 Module path Query rules

We have seen above that there is an important property in the Module object called Paths, which holds an array of paths. Now write the sum module as a custom module:

main.js

const sum = require('sum');
const result = sum(1.2);

module.exports = result;

console.log('---------------main.js-----------')
console.log(module.paths);
Copy the code

Then create a node_modules directory in the parent directory of main.js and create a sum.js module:

node_modules/sum.js

module.exports = (x, y) = > x + y;

console.log('---------------node_module/sum.js-----------')
console.log(module.paths);

Copy the code

To perform the main. Js:

We found that sum.js introduced as a custom module has the same result as paths in the file module. So we can come up with a rule:

  • To query information in the current file directorynode_modulesThe path
  • Query the value of the parent directorynode_modulesThe path
  • Query information about the parent directory of the parentnode_modulesThe path
  • Keep recursing until you reach the root directorynode_modulesThe path

The node_modules directory is also printed in main.js, but the node_modules directory is not printed in main.js. Here’s a thought question.

3.3 Locating Files

After path analysis, the following specific location of the file, mainly divided into two steps: file extension analysis and directory analysis.

3.3.1 File extension name analysis

We can use require to import modules without adding file extensions. Such as:

const sum = require('./sum')
Copy the code

At this point, Node will analyze the file extension name. It will analyze the following three extensions in order:

  • .js
  • .node
  • .json

During analysis, Node synchronously blocks calls the FS module to determine whether the file exists. If the file is not found and a directory is found, directory (package) analysis is performed.

3.3.2 Directory (Package) Analysis

To test this, let’s tidy up the directory structure:

Node_modules = node_modules = node_modules = node_modules = node_modules

package.json

{
  "main": "sum.js"
}
Copy the code

The rest of the code stays the same. The module is loaded successfully if the result can be printed normally after executing main.js. The above is actually a process of directory analysis.

  • findsumThis directory (or package)
  • Judge whether or notpackage.jsonFile, if any, useJSON.parseparsingJSONObject, find the file name corresponding to the name of the main property, which I have heresum.jsIf the file name does not have an extension name, do the extension name analysis first, and then locate the module.
  • If there is nopackage.jsonFiles are searched in the current directoryindex.js,index.node,index.json
  • If none of the criteria are met, an exception is thrown

If you understand this process, you can also understand why the node_modules folder is automatically generated when NPM install is installed, and there are many packages in this folder. And then the way we introduce it is the way we introduce custom modules.

3.4 Module Compilation

Now that the module is found, we need to compile the module and execute the code in the module to expose any variables that need to be exposed. Different file extensions load different compilation methods.

  • .js: The fs module is loaded synchronously and compiled for execution.
  • .node: This is an extension file written in C/C ++ that needs to be compiled by calling dlopen().
  • .jsonParse the results using json. parse after synchronous loading by fs module.
  • The rest of the extensions are treated as JS files.

Every compiled Module caches the absolute path of the file as an index on the module. _cache object to improve secondary import performance.

3.4.1 Compiling A JSON File

Parse json. parse is used to read JSON file contents through Node synchronous call fs module, and then expose the parsed result in exports object. It is usually described as a configuration file, and Node usually loads pacakage.json itself. The processing code is as follows:

Module._extensions['.json'] = function(module, filename) { var content = NativeModule.require('fs).readFileSync(filename, 'utf-8'); try { module.exports = JSON.parse(stripBOM(content)); } catch(err) { err.message = filename + ':' + err.message; thow err; }}Copy the code

3.4.2 Node File Compilation

The node file is an extension of C/C++, which is already compiled and wrapped in the Libuv layer. It can be loaded and executed by calling the procee.dlopen method in Node. This one is more difficult, so I won’t go into it.

3.4.3 compiling JavaScript files

When compiling JavaScript files, add a layer of functions to the current module package and use closures to solve the problem of global variable contamination. Here is a simple implementation.

(function(exports, require, module, __dirname, __filename){
	var load = function (exports, module) {
	    // Read main.js code:
		const sum = require('./sum');
		const result = sum(1.2);
		module.exports.result = result;
	    // the main.js code ends
	    return module.exports;
	};
	var exported = load(module.exports, module);
	/ / save the module:
	save(module, exported);
})
Copy the code

Once wrapped, the variables that need to be exported are exported through module.exports. The other modules can only access the exported variables, the rest of the variables are not accessible

4. Differences between ES Module and ES Module

The loading mechanism for CommonJS modules is that the input is a copy of the output value. That is, once a value is printed, changes within the module do not affect that value. Let’s change the main.js code:

let sum = require('./sum');
sum = { a: 1 };
console.log(require('./sum'))
console.log(sum)
Copy the code

As you can see, the two modules are not affected.

This is different in ES Module, which is statically loaded. That is, module dependencies are identified during the code static parsing phase. In a word:

The CommonJS module prints a copy of the value, the ES6 module prints a reference to the value.

So modules interact with modules in ES Module.

Five, the summary

I had a vague understanding of CommonJS modularity before, and this article is a note for me. After all, most of the content is borrowed from the previous articles or books and followed their footsteps. Even so, I found it rewarding, and thanks again for their articles and books. Although there are still many imperfect places, I still have confidence in myself and strive to have more understanding of myself in the future.

Six, reference

[1] Simple Node.js edited by Piao Ling

[2] Ruan Yifeng CommonJS specification

[3] Liao Xuefeng module