The Online classroom Platform provides the classroom SDK that encapsulates core capabilities, and the business side develops user-oriented online classroom App based on the classroom SDK. When I recently made a major change to the classroom SDK, I ran into an awkward problem. This problem took me about 3 days, so that I was once so stressed that my whole body was hot. The problem was solved, but the cause was not well understood. It wasn’t until more than a month later that I had time to do some more in-depth analysis and write this article.
Business App projects can compile in TypeScript, but will report errors at run time. There are two types of errors reported for different ways of using the classroom SDK. Figure 1 shows the error reported during debugging after normal installation of classroom SDK in the App project of the business side. Figure 2 shows the error reported during debugging after YARnlink classroom SDK in the App project of the business side.
Figure 1
Figure 2
Before analyzing this problem, we need to analyze the module mechanism of JS (JavaScript).
CommonJS vs ES6 modules
What is the difference between CommonJS and ES6 (ECMAScript 6) modules? The ECMAScript 6 Tutorial points out three major differences between the two Module architectures in the chapter “Loading implementation of Modules”. Personally, I think these three differences are basically wrong, causing a lot of misunderstanding. Here are some of the differences I’ve summarized:
- CommonJS module is implemented by JS runtime, ES6 module is implemented by JS engine. The ES6 module is a low-level implementation at the language level, and the CommonJS module is the upper layer to compensate for the lack of a low-level module mechanism. The difference can be detected by the error message.
- The CommonJS module analyzes module dependencies at execution stage, using depth-first traversal in the order of parent -> child -> parent. ES6 module analyzes module dependencies in the pre-processing stage and executes modules in the execution stage. Depth-first traversal is adopted in both stages, and the execution order is child -> parent. To put it simply, CommonJS modules load and execute module files synchronously, while ES6 modules load and execute module files prematurely.
- Incorrect use of CommonJS module circular references generally does not cause JS errors; Improper use of cyclic references in ES6 modules generally leads to JS errors.
- The position of the import/export statement of the CommonJS module will affect the result of the module code execution. The positions of import and export statements in the ES6 module do not affect the execution results of code statements in the ES6 module.
For the convenience of explanation, this article puts the JS code running roughly divided into pre-processing and execution of two stages, there is no such official saying. Here’s a more detailed analysis.
CommonJS module
In Node.js, CommonJS modules are loaded by CJS /loader.js. Among them, module wrapper is a more clever design. In browsers, CommonJS modules are typically implemented by a runtime provided by the package manager, and the overall logic is similar to node.js’s module runtime, using a module wrapper. The following analysis uses Node.js as an example.
Module usage error
CJS /loader.js throws errors when CommonJS modules are used incorrectly. Such as:
// Node.js
internal/modules/cjs/loader.js:905
throw err;
^
Error: Cannot find module './none_existed.js'
Require stack:
- /Users/wuliang/Documents/code/demo_module/index.js
Copy the codeAs you can see, the error is thrown by the throw statement.
Module execution sequence
CommonJS modules are executed sequentially, loading and executing the corresponding module’s code when require is encountered, and then coming back to execute the current module’s code. As shown in Figure 3, module A depends on modules B and C. Module A is divided into three sections from top to bottom by two require statements, denoted as A1, A2 and A3.
Figure 3
As shown in Figure 4, the code blocks are executed in the order A1 -> B -> A2 -> C -> A3.
Figure 4.
Module circular reference
As you can see from the L765, L772, and L784 codes of CJS /loader.js, the corresponding module objects are created and cached before module execution. The process a module performs is actually calculating the variable properties to the module object that need to be exported. As a result, CommonJS modules are already in a fetching state when they start execution, which is a good solution to the problem of circular reference of modules.
As shown in Figure 5, module A depends on module B, which in turn depends on module A. Modules A and B are divided into two sections from top to bottom by require statement, denoted as A1, A2, B1 and B2 respectively.
Figure 5
As shown in Figure 6, the code blocks are executed in the order A1 -> B1 -> B2 -> A2.
Figure 6.
The problem of improper use
What if B2 uses a variable derived from A2? The attribute corresponding to this variable does not exist on the module object of module A, and the value obtained is undefined. Getting undefined is not expected, but generally does not cause JS errors. As you can see, the position of the import/export statements of the CommonJS module affects the result of the execution of the module’s code statements, since the REQUIRE statement directly splits the code block executed.
ES6 module
ES6 module is implemented with JS engine. JS engine implements the bottom core logic of ES6 module, JS runtime needs to be adapted in the upper layer. Adaptation workload is not small, such as the implementation of file loading, specific can see a discussion I initiated.
Module usage error
The ES6 module is used incorrectly, and errors are thrown by the JS engine or JS runtime adaptation layer. Such as:
// Node.js reported an error
internal/process/esm_loader.js:74
internalBinding('errors').triggerUncaughtException
^
Error [ERR_MODULE_NOT_FOUND]: Cannot find module
// An error occurred in the browser
Uncaught SyntaxError: The requested module './child.js' does not provide an export named 'b'
Copy the codeThe first is an internal error triggered by the Node.js adaptation layer (not thrown by a throw), and the second is a JS engine-level syntax error thrown by the browser.
Module execution sequence
ES6 Module has five states, namely unlinked, linking, linked, evaluating and Evaluated, which are represented by the Status field of Module Environment Records. ES6 module includes two steps: link and evaluate. Evaluation will not be conducted until the connection is successful.
The connection is primarily implemented by the function InnerModuleLinking. The InnerModuleLinking function calls the InitializeEnvironment function, which initializes the module’s Environment Records, Creating Execution context for modules, binding imported module variables and initializing the corresponding variables of submodules Create bindings for var variables and initialize them to undefined, create bindings for function declaration variables and initialize them to the instantiated value of the function body, and create bindings for other variables but do not initialize them.
For the module relationship in Figure 3, the wiring process is shown in Figure 7. Adopting depth-first traversal connection phase, function HostResolveImportedModule acquisition module. The function InitializeEnvironment that does the core operation is executed postponely, so in effect, the child module is initialized before the parent module.
Figure 7.
The evaluation is primarily implemented by the function InnerModuleEvaluation. The InnerModuleEvaluation function calls the ExecuteModule function, which evaluates the module code (evaluating the module.[[ECMAScriptCode]]). The ES6 specification does not specify exactly what the evaluation module code refers to here. I went through the relevant parts of the ES6 specification at least a dozen times before I came up with a reasonable explanation. Here the Evaluation module code should mean Runtime Semantics: Evaluation which executes the corresponding sections of Clause 13, clause 14, and Clause 15 in the order in which the code statements are executed. The evaluating scriptBody in a ScriptEvaluation should mean the same thing. As you can see, the ES6 specification, while well-designed and logically clear and consistent, still has some ambiguities and does not reach a state of absolute perfection and invocability.
For the module relationship in Figure 3, the evaluation process is shown in Figure 8. And similar connection stage, evaluation stage and USES the depth-first traversal, through function HostResolveImportedModule access module. The ExecuteModule function, which does the core operations, is executed behind, so in effect, the child module is executed before the parent module.
Figure 8.
Since the link stage creates bindings for imported module variables and initializes them as corresponding variables of submodules, which are assigned values first in the evaluation stage, imported module variables get the same promotion effect as function declaration variables. For example, code 1 is fine. Therefore, the position of the import and export statements of the ES6 module does not affect the execution result of the module code statements.
console.log(a) // Print the value of a normally
import { a } from './child.js'
Copy the codeCode 1
Module circular reference
For cyclic reference scenarios, submodules are preprocessed and executed first. In addition to analyzing module dependencies, the connection stage also creates execution context and initialization variables, so the connection stage mainly includes analyzing module dependencies and preprocessing modules. As shown in Figure 9, for the module relationship in Figure 5, the processing sequence is: pre-processing B -> pre-processing A -> executing B -> executing A.
Figure 9.
The problem of improper use
Since the child module is executed before the parent module, executing variables imported from the parent module directly by the child module causes JS errors.
/ / file parent. Js
import {} from './child.js';
export const parent = 'parent';
/ / file child. Js
import { parent } from './parent.js';
console.log(parent); / / an error
Copy the codeCode 2
As shown in Code 2, the import variable parent in child.js is bound to the export variable parent of parent-js. When the last line of child.js is executed, parent-js has not yet been executed. The export variable parent in parent-js is not initialized, so the import variable parent in child.js is not initialized, resulting in js errors. Var, let, const, function, etc.
console.log(parent)
^
ReferenceError: Cannot access 'parent' before initialization
Copy the codeIf it is executed asynchronously, this is fine because the parent module has already been executed asynchronously. For example, code 3 is fine.
// parent.js
import {} from './child.js';
export const parent = 'parent';
// child.js
import { parent } from './parent.js';
setTimeout(() = > {
console.log(parent) / / output 'parent'
}, 0);
Copy the codeCode 3
Correcting tutorial views
The book ECMAScript 6 Tutorial highlights three major differences:
-
The CommonJS module prints a copy of the value, the ES6 module prints a reference to the value.
-
The CommonJS module is run time loaded, and the ES6 module is compile time output interface.
-
The CommonJS module require() is a synchronously loaded module, while the ES6 module import command is asynchronously loaded, with a separate module-dependent parsing phase.
For point 1, both CommonJS and ES6 modules output variables, which are references to values. Some of the comments in the chapter also question this point. For the second point, the first half of the sentence is basically true, and the second half is basically false. The CommonJS module loads submodule files in the execution phase, the ES6 module loads submodule files in the preprocessing phase, and of course the ES6 module loads submodule files in the execution phase, but uses the preprocessing phase cache. From the formal CommonJS module overall export a object contains a number of variables, separate ES6 module export a single variable, if only the parent module, ES6 module’s father did in the pretreatment stage is binding child module export variables, but the pretreatment stage of sub-modules of the final value of the variable is not is derived, So it’s not really output. For point 3, CommonJS modules load and execute module files synchronously, ES6 modules load and execute module files ahead of time. Asynchrony is usually understood as a delay of execution by one time node, so it is incorrect to say asynchrony.
To analyze problems
With a deeper understanding of the JS module mechanism, let’s return to the problem I encountered.
Problem a
First, analyze the error reported in Figure 1. The engineering code for the business App is packaged with WebPack, so you actually run the CommonJS module. The CommonJS module loop reference will not cause JS error. The CommonJS module loop reference will not cause JS error. This is because circular references are misused and the value of the variable is undefined. Our code uses extends, which does not support undefined. Error reported by gasket _inherits because bable was used for transcoding. Another typical case where undefined is not supported is object.create (undefined).
Question 2
Then analyze the error in Figure 2. In the App project of the business side, yarn Link classroom SDK is packaged with WebPack, but still runs CommonJS module. Why do JS engine level errors occur? Isn’t this an ES6 module error? There are two reasons for this. The classroom SDK is packaged using rollup. Rollup packages multiple files into a single file, and the submodule’s code is placed in front of the parent module. For example, code 2 is rollup packed to become code 4.
console.log(parent); / / an error
const parent = 'parent';
export { parent };
Copy the codeCode 4 After the local YARN Link classroom SDK, the referenced classroom SDK package path is soft connection, which is ignored when Babel transcodes. Thus, the business App directly references the classroom SDK for the ES6+ syntax packaged by Rollup. If a variable exported by the parent module is executed directly in a child module, an error is reported. As shown in Code 4, when the first line of code executes, the parent variable is bound but not initialized.
To solve the problem
It is clear that the problem is caused by module circular reference, and the specific reasons are analyzed. How do you find a module in a complex code project that has circular references?
webpack plugin
Circular -dependency-plugin is a Webpack plug-in that analyzes circular references to modules. Its source code is only about 100 lines, the principle is relatively simple. In optimizeModules hooks, recursively search for dependent modules from this module, and compare the debugids of dependent modules and this module. If they are the same, they are judged as circular references and the chain of circular references is returned.
Locate and resolve circular references
After introducing circular-dependency plugin in the business App project, you can see the circular reference module related to the classroom SDK. The output module circular reference chain is quite large, there are 112. How do you further locate several circular references that cause problems? Find the file that reported the error according to the stack, and then find out the loop references related to this file, cut these loop references one by one by hack to verify whether the error is resolved. In the end, I solved the problem by cutting off the two circular references. One of the circular reference chains is as follows:
Circular dependency detected:
node_modules/@byted-classroom/room/lib/service/assist/stream-validator.js ->
node_modules/@byted-classroom/room/lib/service/rtc/engine.js ->
node_modules/@byted-classroom/room/lib/service/rtc/definitions.js ->
node_modules/@byted-classroom/room/lib/service/rtc/base.js ->
node_modules/@byted-classroom/room/lib/service/monitor/index.js ->
node_modules/@byted-classroom/room/lib/service/monitor/monitors.js ->
node_modules/@byted-classroom/room/lib/service/monitor/room.js ->
node_modules/@byted-classroom/room/lib/service/npy-courseware/student-courseware.js ->
node_modules/@byted-classroom/room/lib/service/index.js ->
node_modules/@byted-classroom/room/lib/service/audio-mixing/index.js ->
node_modules/@byted-classroom/room/lib/service/audio-mixing/mixing-player.js ->
node_modules/@byted-classroom/room/lib/index.js ->
node_modules/@byted-classroom/room/lib/room/base.js ->
node_modules/@byted-classroom/room/lib/service/rtc/manager.js ->
node_modules/@byted-classroom/room/lib/service/assist/stream-validator.js
Copy the codeadvice
The problem of circular references in TypeScript projects is common, often adding a file dependency because of the need to use a type. It is recommended to introduce circular reference detection mechanism in the project, such as the webPack plug-in circular-dependency-plugin and esLint rule import/no-cycle, so that the file or code structure can be adjusted in time to cut off circular references.
conclusion
Based on an error encountered during development, this paper makes a deep analysis of JS module mechanism and cyclic reference, and provides a method to locate and solve the problem of cyclic reference of modules. Based on an interpretation of the ES specification, this article corrects several of the mistakes made in the ECMAScript 6 Tutorial.
reference
- ECMAScript ® 2022 Language Specification
