An in-depth look at JavaScript module circular references

background

The online classroom platform of Vigorously Education provides the classroom SDK that encapsulates the core capabilities, and the business side develops the user-oriented online classroom App based on the classroom SDK. When I recently made a major change to the classroom SDK, I ran into a muddled problem. This problem took me about three days and at one point I was so stressed that I had a fever all over my body. At that time, the problem was solved, but the reason was not fully understood. It wasn’t until more than a month later that I had time to do some more in-depth analysis and write this article.

At the time, business side App projects could be compiled in TypeScript but would report errors at runtime. There are two different ways to use the classroom SDK. Figure 1 shows the error reported during debugging after the classroom SDK is normally installed in the App project of the business party. Figure 2 shows the error during debugging after the SDK of yarn Link classroom in the App project of the business side.

Figure 1

Figure 2

Before analyzing this problem, we need to first analyze the JS (JavaScript) module mechanism.

CommonJS vs ES6 modules

What is the difference between CommonJS and ES6 (ECMAScript 6) modules? The ECMAScript 6 Tutorial [1] points out three important differences between the two Module systems in the “Module loading implementation” section. Personally, I think these three differences are basically wrong and cause a lot of misunderstanding. Later on, I will talk about the reasons for doubt. Here are some differences I summarized:

The CommonJS module is implemented by the JS runtime, and the ES6 module is implemented by the JS engine. The ES6 module is the low-level implementation of the language level, and the CommonJS module is the upper layer where the underlying module mechanism was missing. This difference can be detected by the error message.
The CommonJS module loads and executes the module file synchronously, while the ES6 module loads and executes the module file in advance. The CommonJS module analyzes module dependency in the execution stage, and adopts depth-first traversal. The execution order is parent -> child -> parent. ES6 module analyzes module dependencies in the pre-processing stage and executes modules in the execution stage. Both stages adopt depth-first traversal, and the execution sequence is child -> parent.
Incorrect use of CommonJS module circular references generally does not cause JS errors; Improper use of ES6 module circular references generally results in JS errors.
The location of the CommonJS module’s import and export statements affects the execution of the module’s code. The location of the import and export statements of the ES6 module does not affect the execution results of the module code statements.

For the sake of convenience, this article divides JS code’s operation into two stages: preprocessing and execution. Note that there is no official statement like this. Here’s a more detailed analysis.

CommonJS module

In Node.js, the CommonJS module [2] is loaded by CJS /loader.js[3]. The module wrapper is one of the more ingenious designs.

In the browser, CommonJS modules are typically implemented by the runtime provided by the package manager, and the overall logic is similar to that of the Node.js module runtime, which also uses a module wrapper. The following analysis uses Node.js as an example.

Module usage error

If the CommonJS module is improperly used, an error is thrown by CJS /loader.js. Such as:

// Node.js
internal/modules/cjs/loader.js:905
  throw err;

  ^

Error: Cannot find module './none_existed.js'
Require stack:
- /Users/wuliang/Documents/code/demo_module/index.js
Copy the code

As you can see, the error is thrown through the throw statement.

Module execution sequence

CommonJS modules execute sequentially, loading and executing the code for the corresponding module when require is encountered, and then coming back to execute the code for the current module.

As shown in Figure 3, module A depends on modules B and C. Module A is divided into three sections from top to bottom by two require statements, denoted as A1, A2 and A3.

Figure 3

As shown in Figure 4, the code block execution sequence is A1 -> B -> A2 -> C -> A3.

Figure 4.

Module circular reference

From line L765, L772 and L784 of CJS/Loader.js, you can see that the corresponding module object is created and cached before the module is executed. The process performed by the module is actually calculating the variable attributes to be exported to the module object. As a result, the CommonJS module is already in a fetching state when it starts execution, which is a good solution to the problem of circular module references.

As shown in Figure 5, module A depends on module B, which in turn depends on module A. Module A and module B are divided into two sections respectively by require statement from top to bottom, denoted as A1, A2, B1 and B2.

Figure 5

As shown in Figure 6, the execution sequence of code blocks is A1 -> B1 -> B2 -> A2.

Figure 6.

Improper use of the problem

What happens if B2 uses a variable derived from A2? There is no attribute for this variable on the module object of module A, and the value is undefined. Getting undefined is not what you expect, but generally does not cause a JS error.

As you can see, the position of the CommonJS module’s import and export statements affects the execution result of the module’s code statements because the require statement directly splits the block of code that is executed.

ES6 module

ES6 module [4] is implemented with the help of JS engine. The JS engine implements the underlying core logic of the ES6 module, and the JS runtime needs to do adaptation at the upper level. The adaptation workload is not small, such as file loading. For details, please refer to a discussion I initiated [5].

Module usage error

When an ES6 module is improperly used, an error is thrown by the JS engine or the JS runtime adaptation layer. Such as:

// An error is reported in node.js
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException

                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module

// An error is reported in the browser
Uncaught SyntaxError: The requested module './child.js' does not provide an export named 'b'
Copy the code

The first is an internal error thrown by the Node.js adaptation (not thrown by a throw), and the second is a JS engine level syntax error thrown by the browser.

Module execution sequence

ES6 Module has five states, namely unlinked, linking, linked, evaluating and evaluated, which are represented by Status field of Module Environment Records [6]. The process of THE ES6 module includes two steps: link and evaluate. Only after the connection is successful can the evaluation be made.

The connection is implemented primarily by the function InnerModuleLinking[7]. InnerModuleLinking calls the function InitializeEnvironment[8], which initializes the module’s Environment Records [9], Setting the Execution context for a module [10], setting a binding for an imported module variable [11] and initializer for a submodule variable [12] Create bindings for var variables and initialize them to undefined, create bindings for function declaration variables and initialize them to the instantiated [13] value of the function body, and create bindings for other variables but do not initialize them.

For the module relationship in Figure 3, the connection process is shown in Figure 7. Adopting depth-first traversal connection phase, function HostResolveImportedModule [14] access module. The function InitializeEnvironment, which does the core operation, is executed after, so in effect, the child module is initialized before the parent module.

Figure 7.

The evaluation is mainly implemented by the function InnerModuleEvaluation[15]. The InnerModuleEvaluation function calls the ExecuteModule[16] function, which evaluates module code (evaluating module.[[ECMAScriptCode]]). Evaluating module code (evaluating module. The ES6 specification does not specify what the evaluation module code refers to. I went through the relevant parts of the ES6 specification at least a dozen times before I came up with a reasonable explanation. The Semantics of Evaluation should refer to the “Runtime Semantics: Evaluation” clause 13[17], clause 14[18] and Clause 15[19] in the order in which the code is executed. Evaluating scriptBody in ScriptEvaluation[20] should mean the same thing. As you can see, the ES6 specification, while a lot of design is done and logical and self-consistent, still has some ambiguity and does not reach a state of absolute perfection and watertight.

For the module relationship in Figure 3, the evaluation process is shown in Figure 8. And similar connection stage, evaluation stage and USES the depth-first traversal, through function HostResolveImportedModule access module. ExecuteModule, the function that performs the core operations, executes after, so in effect, the child module executes before the parent module.

Figure 8.

Since the connection phase will create bindings for the imported module variables and initialize them as the corresponding variables of the submodule, and the corresponding variables of the submodule will be assigned first in the evaluation phase, the imported module variables can obtain the same improvement effect as the function declaration variables. For example, code 1 works fine. Therefore, the location of the import and export statements of the ES6 module does not affect the execution results of the module code statements.

console.log(a) // Print the value of a normally
import { a } from './child.js'
Copy the code

Code 1

Module circular reference

In the case of circular references, the submodules are preprocessed and executed first. In addition to analyzing module dependencies, the connection phase also creates execution context and initialization variables, so the connection phase mainly includes analyzing module dependencies and preprocessing modules. As shown in Figure 9, for the module relationship in Figure 5, the processing sequence is: Preprocessing B -> Preprocessing A -> execute B -> Execute A.

Figure 9.

Improper use of the problem

Because the submodule is executed before the parent module, the submodule directly executes variables imported from the parent module will cause a JS error.

/ / file parent. Js
import {} from './child.js';
export const parent = 'parent';

/ / file child. Js
import { parent } from './parent.js';
console.log(parent); / / an error
Copy the code

Code 2

As shown in code 2, the import variable parent in child.js is bound to the export variable parent in parent.js. When the last line of child.js is executed, parent-.js has not been executed. The parent export variable in parent.js is not initialized, so the parent import variable in child.js is not initialized, resulting in a JS error. Var, let, const, function, etc.

console.log(parent)
            ^
ReferenceError: Cannot access 'parent' before initialization
Copy the code

If it is executed asynchronously, this is fine because the parent module has already been executed. For example, code 3 works fine.

// parent.js
import {} from './child.js';
export const parent = 'parent';

// child.js

import { parent } from './parent.js';
setTimeout(() = > {
  console.log(parent) / / output 'parent'
}, 0);
Copy the code

Code 3

Correcting tutorial ideas

The three major differences in ECMAScript 6:

The CommonJS module outputs a copy of the value, and the ES6 module outputs a reference to the value.
The CommonJS module is a runtime load, and the ES6 module is a compile-time output interface.
The require() of the CommonJS module loads the module synchronously, and the import command of the ES6 module loads asynchronously, with a separate module dependency resolution phase.

For point 1, both CommonJS and ES6 modules output variables, which are references to values. This point was also questioned in the section’s comments. For the second point, the first half of the sentence is basically correct, the second half of the sentence is basically wrong. The CommonJS module loads the submodule file during execution, and the ES6 module loads the submodule file during preprocessing. The ES6 module also loads the submodule file during execution, but it uses the preprocessing cache. From the formal CommonJS module overall export a object contains a number of variables, separate ES6 module export a single variable, if only the parent module, ES6 module’s father did in the pretreatment stage is binding child module export variables, but the pretreatment stage of sub-modules of the final value of the variable is not is derived, So it’s not really output. For point 3, the CommonJS module loads and executes the module file synchronously, and the ES6 module loads and executes the module file ahead of time. Asynchronous is generally understood to mean delayed execution of a time node, so it is a mistake to say asynchronous loading.

To analyze problems

Now that we have a better understanding of the JS module mechanism, let’s go back and analyze the problem I encountered.

Problem a

First analyze the error report in Figure 1. The engineering code for the business side App is packaged in WebPack, so the CommonJS module is actually running. The CommonJS module loop reference does not normally cause a JS error. This is because the improper use of circular references causes the value of the variable to be undefined, and our code uses extends[21], which does not support undefined. [23] [[inherits] [[inherits] [[inherits] [[inherits] [[inherits] [[inherits] [[inherits] Another typical unsupported case for undefined is object.create (undefined).

Question 2

Then analyze the error in Figure 2. In the App project of yarn Link classroom SDK of the business side, after packaging with WebPack, the CommonJS module is still running. Why does the JS engine level error occur? Isn’t this an ES6 module error? There are two reasons.

The Classroom SDK is packaged using Rollup[24]. Rollup packages multiple files into a single file, and the submodule code is put before the parent module. For example, code 2 is rolled up to become code 4.

console.log(parent); / / an error

const parent = 'parent';
export { parent };
Copy the code

Code 4

After the local YARN Link classroom SDK, the referenced classroom SDK package path is the soft connection, which is ignored in Babel transcoding. Therefore, the business App directly references the classroom SDK that Rollup packaged with ES6+ syntax. If a variable exported by the parent module is executed directly in a submodule, an error is reported. As shown in code 4, when the first line of code is executed, the parent variable is bound but not initialized.

To solve the problem

It is clear that the problem is caused by module circular reference and analyzes the specific reasons. So how do you find the modules with circular references in complex code projects?

webpack plugin

Circular -dependency- Plugin [25] is a WebPack plug-in that analyzes circular references to modules. Its source code is only about 100 lines, the principle is relatively simple. In the optimizeModules[26] hook, we recursively look for the dependent module from this module, and compare the debugId of the dependent module with that of this module. If the debugId is the same, we determine it to be a circular reference and return the circular reference chain.

Locate and resolve circular references

After introducing the circular-dependency-plugin in the business App project, you can see the circular reference module associated with the classroom SDK. The output module circular reference chain is relatively large, there are 112. How do you further locate the few circular references that are causing the problem? Locate the file based on the stack that reported the error, then find the circular references related to the file, hack off each of the circular references one by one and verify that the error was resolved. Finally, I solved the problem by cutting off two circular references. One of the circular reference chains is as follows:

Circular dependency detected:

node_modules/@byted-classroom/room/lib/service/assist/stream-validator.js ->

node_modules/@byted-classroom/room/lib/service/rtc/engine.js ->

node_modules/@byted-classroom/room/lib/service/rtc/definitions.js ->

node_modules/@byted-classroom/room/lib/service/rtc/base.js ->

node_modules/@byted-classroom/room/lib/service/monitor/index.js ->

node_modules/@byted-classroom/room/lib/service/monitor/monitors.js ->

node_modules/@byted-classroom/room/lib/service/monitor/room.js ->

node_modules/@byted-classroom/room/lib/service/npy-courseware/student-courseware.js ->

node_modules/@byted-classroom/room/lib/service/index.js ->

node_modules/@byted-classroom/room/lib/service/audio-mixing/index.js ->

node_modules/@byted-classroom/room/lib/service/audio-mixing/mixing-player.js ->

node_modules/@byted-classroom/room/lib/index.js ->

node_modules/@byted-classroom/room/lib/room/base.js ->

node_modules/@byted-classroom/room/lib/service/rtc/manager.js ->

node_modules/@byted-classroom/room/lib/service/assist/stream-validator.js
Copy the code

advice

Circular references in TypeScript projects are common, often adding a file dependency to the need to use a type. It is recommended to introduce a module circular reference detection mechanism in your project, such as the WebPack plug-in circular-dependency-plugin and esLint rule import/no-cycle, so that you can adjust the file or code structure in time to break the circular reference.

conclusion

This paper analyzes the JS module mechanism and circular reference in depth, and provides the method of locating and solving the problem of circular reference. Based on an interpretation of the ES specification, this article corrects several misconceptions in the ECMAScript 6 Introduction Tutorial.

reference

Introduction to ES6, ES6.ruanyifeng.com/
Modules: CommonJS Modules, nodejs.org/dist/latest…
CJS/loader. Js, github.com/nodejs/node…
ECMAScript Language: Scripts and Modules, TC39.es/ECMA262 /# SE…
What the features of V8 engine dosed Node. Js use when implementing ECMAScript modules, app.slack.com/client/T0K2…
Cyclic Module Records, Tc39.es/ECMA262 /# SE…
InnerModuleLinking tc39. Es/ecma262 / # se…
InitializeEnvironment tc39. Es/ecma262 / # se…
The Environment Records, tc39. Es/ecma262 / # se…
Execution Contexts, tc39. Es/ecma262 / # se…
CreateMutableBinding tc39. Es/ecma262 / # se…
InitializeBinding tc39. Es/ecma262 / # se…
The Runtime Semantics: InstantiateFunctionObject, tc39. Es/ecma262 / # se…
HostResolveImportedModule tc39. Es/ecma262 / # se…
InnerModuleEvaluation tc39. Es/ecma262 / # se…
ExecuteModule tc39. Es/ecma262 / # se…
ECMAScript Language: Expressions, TC39.es/ECMA262 /# SE…
ECMAScript Language: Statements and Declarations, TC39.es/ECMA262 /# SE…
ECMAScript Language: Functions and Classes, TC39.es/ECMA262 /#se…
ScriptEvaluation tc39. Es/ecma262 / # se…
Extends, developer.mozilla.org/en-US/docs/…
Babeljs, babeljs. IO /
_inherits github.com/babel/babel…
Rollupjs.org/guide/en/ rollup. Js
Circular dependency – the plugin, github.com/aackerman/c…
OptimizeModules v4.webpack.js.org/api/compila…

An in-depth look at JavaScript module circular references

background

CommonJS vs ES6 modules

CommonJS module

Module usage error

Module execution sequence

Module circular reference

Improper use of the problem

ES6 module

Module usage error

Module execution sequence

Module circular reference

Improper use of the problem

Correcting tutorial ideas

To analyze problems

Problem a

Question 2

To solve the problem

webpack plugin

Locate and resolve circular references

advice

conclusion

reference

Related Posts

What kind of team developed the game to play with the source code is called excellent

Protocol in WebRTC

Solve the problem that the file type is unrecognized or invalid when downloading files on H5 mobile terminal