We usually write third-party dependencies used in our projects in package.json files and use popular dependency management tools such as NPM, CNPM, or YARN to manage them for us. But how do they manage these dependencies, what are the differences between them, and how to resolve circular dependencies if they occur.
Before we answer these questions, let’s look at semantic versioning rules.
Semantic version
When using third-party dependencies, you usually need to specify the version range of the dependency, for example
"dependencies": {
"antd": "Granted"."react": "~ 16.0.1"."redux": "^ 3.7.2." "."lodash": "*"
}
Copy the codeThe package.json file above shows that the version number of ANTD used in the project is 3.1.2, but what’s the difference between 3.1.1 and 3.1.2, 3.0.1, 2.1.1? The semantic version rule specifies that the version format is major version number. Minor version number. Revision number, and the increment rule of the version number is as follows:
- Major version number: when you make incompatible API changes
- Secondary version number: when you make a backward-compatible functional addition
- Revision number: When you make a backward compatible problem fix
Updating a major version usually means major changes and updates. Upgrading a major version can cause errors in your application, so be careful, but it can also lead to better performance and experience. A minor update usually means that some feature has been added, for example, antD has been upgraded from 3.1.1 to 3.1.2, and the Select component did not support search, but now supports search. An update to the revision number usually means some bug fix has been made. Therefore, the release number and revision number should be kept up to date, so that your previous code does not report errors and you can get the latest features.
However, instead of specifying the exact version of the dependency, we specify the version range, such as the react, redux, and Lodash dependencies in the package.json file above, which each use three symbols to indicate the version range of the dependency. The semantic version scope specifies:
- ~ : Updates only the revision number
- ^ : Upgrade version number and revision number
- * : Upgrade to the latest version
Therefore, the package.json file installed above has the following range of dependent versions:
- React @~16.0.1: >[email protected] && < [email protected]
- Redux @^3.7.2: >[email protected] && < [email protected]
- Lodash @ * : lodash @ latest
Semantic versioning rules define an ideal version number update rule that you want all dependency updates to follow, but there are often many dependencies that do not follow these rules strictly. Therefore, it is very important to manage these dependencies, especially the versions of these dependencies, otherwise you can accidentally fall into various problems caused by inconsistent versions of dependencies.
Dependency management
In project development, NPM, YARN, or CNPM are commonly used to manage dependencies in a project. Let’s take a look at how they help us manage these dependencies.
npm
NPM has gone through three major versions to date.
npm v1
The earliest versions of NPM took a very simple approach to managing dependencies. We call this a nested pattern. For example, in your project you have the following dependencies.
"dependencies": {
A: "1.0.0".C: "1.0.0".D: "1.0.0"
}
Copy the codeThese modules all depend on B modules, and they depend on different versions of B modules.
[email protected] - > [email protected] [email protected] - > [email protected] [email protected] - > [email protected]Copy the codeBy running NPM install, the node_modules directory generated by NPM v1 is as follows:
Node_modules ├ ─ ─ [email protected] │ └ ─ ─ node_modules │ └ ─ ─ [email protected] ├ ─ ─ [email protected] │ └ ─ ─ node_modules │ └ ─ ─ [email protected] └ ─ ─ [email protected] └ ─ ─ Node_modules └ ─ ─ [email protected]Copy the codeObviously, there is a node_modules directory under each module that holds its direct dependencies. Module dependencies there is also a node_modules directory to store the dependencies of module dependencies. This dependency management is obviously straightforward, but the problem is that in addition to nesting the length of the node_modules directory too deeply, it can also create the problem of storing multiple copies of the same dependency. For example, [email protected] above stores two copies, which is obviously a waste. As a result, NPM dependency management changed significantly after THE release of NPM V3.
npm v3
For the same dependencies, the node_modules directory generated by executing the NPM install command using NPM v3 is as follows:
Node_modules ├ ─ ─ [email protected] ├ ─ ─ [email protected] └ ─ ─ [email protected] └ ─ ─ node_modules └ ─ ─ [email protected] ├ ─ ─ [email protected]Copy the codeObviously, NPM V3 uses a flat model, placing all modules and module dependencies in the node_modules directory at the top level. In case of a version conflict, dependencies are stored in the node_modules directory. The reason for this is based on the packet search mechanism. The package search mechanism means that when you require(‘A’) directly in A project, you first search the current path to see if node_modules exists, and if it doesn’t, you search up the path to node_modules. Because of this, NPM V3 was able to flatterthe previous nesting structure and put all dependencies in node_modules at the root of the project, thus avoiding the problem of the node_modules directory being too deeply nested. In addition, NPM V3 will resolve multiple versions of module dependencies into one version. For example, if A depends on B@^1.0.1 and D depends on B@^1.0.2, only one [email protected] version will exist. Although NPM V3 solves these two problems, there are still many problems with NPM at this time, and the most criticized one is its uncertainty.
npm v5
What is certainty. In the context of JavaScript package management, determinism means that a consistent node_modules directory structure is always available for a given package.json and lock files. Simply put, NPM install gets the same node_modules directory structure in any environment. NPM V5 is designed to solve this problem. The node_modules directory generated by NPM V5 is the same as that generated by V3. The difference is that V5 generates a package-lock.json file by default to ensure that the dependencies of the installation are deterministic. For example, for a package.json file like this
"dependencies": {
"redux": "^ 3.7.2." "
}
Copy the codeThe corresponding package-lock.json file contains the following contents:
{
"name": "test"."version": "1.0.0"."lockfileVersion": 1."requires": true."dependencies": {
"js-tokens": {
"version": "3.0.2"."resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-3.0.2.tgz"."integrity": "sha1-mGbfOVECEw449/mWvOtlRDIJwls="
},
"lodash": {
"version": "4.17.4"."resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.4.tgz"."integrity": "sha1-eCA6TRwyiuHYbcpkYONptX9AVa4="
},
"lodash-es": {
"version": "4.17.4"."resolved": "https://registry.npmjs.org/lodash-es/-/lodash-es-4.17.4.tgz"."integrity": "sha1-3MHXVS4VCgZABzupyzHXDwMpUOc="
},
"loose-envify": {
"version": "1.3.1"."resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.3.1.tgz"."integrity": "sha1-0aitM/qc4OcT1l/dCsi3SNR4yEg="."requires": {
"js-tokens": "3.0.2"}},"redux": {
"version": "3.7.2"."resolved": "https://registry.npmjs.org/redux/-/redux-3.7.2.tgz"."integrity": "sha512-pNqnf9q1hI5HHZRBkj3bAngGZW/JMCmexDlOxw4XagXY2o1327nHH54LoTjiPJ0gizoqPDRqWyX/00g0hD6w+A=="."requires": {
"lodash": "4.17.4"."lodash-es": "4.17.4"."loose-envify": "1.3.1"."symbol-observable": "1.1.0"}},"symbol-observable": {
"version": "1.1.0"."resolved": "https://registry.npmjs.org/symbol-observable/-/symbol-observable-1.1.0.tgz"."integrity": "sha512-dQoid9tqQ+uotGhuTKEY11X4xhyYePVnqGSoSm3OGKh2E8LZ6RPULp1uXTctk33IeERlrRJYoVSBglsL05F5Uw=="}}}Copy the codeAs you can see, the package-lock.json file records the exact version of each dependency installed, so that the same dependency can be installed from this file the next time you install it.
yarn
Yarn was made open source on October 11, 2016. Yarn was introduced to solve some problems in NPM V3 before NPM V5 was released. Yarn is defined as fast, secure, and reliable dependency management.
- Fast: global cache, parallel download, offline mode
- Security: Verify the integrity of the installation package before it is executed
- Reliable: Lockfile files, deterministic algorithms
The node_modules directory structure generated by YARN is the same as that of NPM V5, and a yarn.lock file is generated by default. For the example above, the yarn.lock file generated by redux’s dependencies is as follows:
DO NOT EDIT THIS FILE DIRECTLY. # yarn lockfile v1 js-tokens@^3.0.0: The version "3.0.2" resolved "Http://registry.npm.alibaba-inc.com/js-tokens/download/js-tokens-3.0.2.tgz#9866df395102130e38f7f996bceb65443209c25b" Lodash - es @ ^ 2: The version "4.17.4" resolved "Http://registry.npm.alibaba-inc.com/lodash-es/download/lodash-es-4.17.4.tgz#dcc1d7552e150a0640073ba9cb31d70f032950e7" Lodash @ ^ 2: The version "4.17.4" resolved "Http://registry.npm.alibaba-inc.com/lodash/download/lodash-4.17.4.tgz#78203a4d1c328ae1d86dca6460e369b57f4055ae" Loose - envify @ ^ 1.1.0: Version 1.3.1 resolved "Http://registry.npm.alibaba-inc.com/loose-envify/download/loose-envify-1.3.1.tgz#d1a8ad33fa9ce0e713d65fdd0ac8b748d478c8 46 "dependencies: tokens "^3.0.0" The version "3.7.2" resolved "Http://registry.npm.alibaba-inc.com/redux/download/redux-3.7.2.tgz#06b73123215901d25d065be342eb026bc1c8537b" dependencies: Lodash "^4.2.1" lodash-es "^4.2.1" loose-envify "^1.1.0" symbol-Observable "^1.0.3" symbol-Observable @^1.0.3: Version 1.1.0 resolved "Http://registry.npm.alibaba-inc.com/symbol-observable/download/symbol-observable-1.1.0.tgz#5c68fd8d54115d9dfb72a8472054 9222e8db9b32"Copy the codeThe yarn.lock file differs from the package-lock.json file generated by NPM V5 in the following points:
- The file format is different. NPM V5 uses JSON format, and YARN uses a custom format
- The dependency versions recorded in package-lock.json are determined and do not display the semantic version range symbol (~ ^ *), whereas the semantic version range symbol still appears in yarn.lock
- The package-lock.json file is more informative. NPM V5 only needs package.lock to determine the node_modules directory structure. Json and yarn.lock to determine the node_modules directory structure
An official YARN article explains why these differences exist, the deterministic algorithm for YARN, and the differences from NPM V5. Due to the limited space, I will not repeat it here. If you are interested, you can go to my translation Yarn Certainty.
In addition to improving the installation speed, the biggest contribution of YARN is to ensure the certainty of installation dependencies through lock files, ensuring that the same package.json file can be installed in any environment and on any machine to get the same result, that is, the same node_modules directory structure. This largely avoids some “what works on my computer, fails on other machines” bugs. However, you must pay attention to the following three points when using YARN for dependency management.
- Do not manually modify the yarn.lock file
- The yarn.lock file should be committed to the version control repository
- Used when upgrading dependencies
yarn upgradeCommand to avoid manually modifying package.json and yarn.lock files.
cnpm
CNPM’s domestic users should be quite large, especially for those who have the need to build private warehouses. CNPM uses Npminstall to install dependencies. Simply put, CNPM uses the link installation mode to maximize the installation speed, and the generated node_modules directory adopts a different layout from NPM. Packages installed with CNPM are named in the node_modules folder with the version number @package name, and then soft link to the folder named by the package name only. Similarly, the node_modules directory structure generated when using CNPM to install only redux dependencies is as follows:
The biggest difference between CNPM, NPM, and YARN is the generated node_modules directory structure, which can cause problems in some scenarios. Lock files are also not generated, which results in less deterministic installation than NPM and YARN. However, CNPM uses a link installation that saves disk space and keeps the node_modules directory structure clean, which strikes a balance between nested and flat modes.
NPM, YARN, and CNPM all provide good dependency management to help us manage the various dependencies and versions used in the project. However, what should we do if a dependency is called in a loop?
Circular dependencies
Cyclic dependency means that module A’s execution depends on module B, and module B’s execution depends on module A. Loop dependencies can lead to recursive loading and, if not handled properly, can render the program unexecutable. Before we dive into circular dependencies, let’s take a look at the module specification in JavaScript. Because different specifications deal with circular dependencies differently.
Currently, there are three common JavaScript specifications: CommonJS, AMD, and ES6.
The module specification
CommonJS
Since node.js appeared in 2009, CommonJS module system has gradually gained popularity. A module in CommonJS is a script file that loads the module with the require command and uses the interface exposed by the module. Load-time execution is an important feature of the CommonJS module, meaning that script code executes the code in the module when required. This feature is fine on the server side, but introducing a module and waiting for it to complete before it can execute the following code can be problematic on the browser side. Hence the AMD specification to support the browser environment.
AMD
AMD stands for “Asynchronous Module Definition”. It adopts asynchronous loading mode to load modules, and the loading of modules does not affect the running of its following statements. All statements that depend on this module are defined in a callback function that will not run until the load is complete. The most representative implementation is Requirejs.
ES6
Unlike CommonJS and AMD’s module loading solutions, ES6 implements module functionality at the JavaScript language level. The idea is to be as static as possible so that module dependencies can be determined at compile time. When the module load command import is encountered, the module is not executed, but only a reference is generated. Wait until you really need to use it, and then go to the module to value. This is the biggest difference from the CommonJS module specification.
CommonJS loop dependency solution
Look at the following example:
a.js
console.log('a starting');
exports.done = false;
const b = require('./b.js');
console.log('in a, b.done = %j', b.done);
exports.done = true;
console.log('a done');
Copy the codeb.js
console.log('b starting');
exports.done = false;
const a = require('./a.js');
console.log('in b, a.done = %j', a.done);
exports.done = true;
console.log('b done');
Copy the codemain.js
console.log('main starting');
const a = require('./a.js');
const b = require('./b.js');
console.log('in main, a.done=%j, b.done=%j', a.done, b.done);
Copy the codeIn this example, module A calls module B, which in turn calls module A, creating a loop dependency between a and B, but when we execute Node main.js the code doesn’t get stuck in an infinite loop. Instead, it prints something like this:
$ node main.js
main starting
a starting
b starting
in b, a.done = false
b done
in a, b.done = true
a done
in main, a.done=true, b.done=true
Copy the codeWhy didn’t the program report an error, instead of printing something like this? This is because of two features of the CommonJs module. First, execute on load; Second, loaded modules are cached and not reloaded. Let’s analyze the execution process of the program:
- Execute main.js and print main starting
- Js load a.js, execute a.js and print a starting, export done = false
- A. js load B. js, execute B. js and print B starting, export done = false
- B. js load a.js, because a.js has been loaded once before, so it will not be loaded again. In the cache, a.js export done = false, therefore, b.js output in B, a. Tone = false
- B. js export done = true and output b done
- After b.js is executed, the execution authority is handed back to A. js, execute A. js, and output in a, b. Tone = true
- A. js export done = true and print a done
- After the execution of A. js is completed, the execution authority is handed back to main.js, and main.js loads B. js. Since B. js has been loaded once before, the execution will not be repeated
- In main, a.tone =true, b.tone =true
In the CommonJS specification, when a require() statement is encountered, the code in the Require module will be executed and the result will be cached. The next time it is loaded, it will not be executed again, but will fetch the cached result directly. Because of this, there is no case of infinite loop calls when loop dependencies occur. While this module-loading mechanism can avoid the problem of cyclic dependency reporting errors, it is possible to make code that does not perform as expected if you are not careful. Therefore, careful planning is needed when writing code to ensure that cyclic module dependencies work correctly. Careful Planning is required to allow Cyclic Module dependencies to work correctly within an application).
Is there anything you can do to avoid circular dependencies other than careful planning? A less elegant approach would be to export each module that the loop depends on first and then require, using CommonJS caching to export its contents before requiring () other modules. This ensures that other modules can get the correct value when used. Such as:
A.js
exports.done = true;
let B = require('./B');
console.log(B.done)
Copy the codeB.js
exports.done = true;
let A = require('./A');
console.log(A.done)
Copy the codeThis writing method is simple, the disadvantage is to change the writing method of each module, and most students are used to write require statement at the beginning of the file.
As a matter of personal experience, loop dependencies should be considered only when writing code. Most of you will rarely encounter the need to handle loop dependencies manually when writing Node.js, and most of you will probably never think about it.
Solution of cyclic dependence in ES6
To understand the solution to loop dependencies in ES6, you must first understand the module loading mechanism in ES6. We all know that ES6 uses the export command to specify the external interface of modules and the import command to load modules. So what happens when you encounter import and export? The module loading mechanism of ES6 can be summarized as four words: one quiet move.
- Istatic: import is executed statically
- Move: export Dynamic binding
Import statically executed means that the import command is statically parsed by the JavaScript engine, taking precedence over the rest of the module. Export dynamic binding indicates that the interface of export command output is dynamically bound with its corresponding value. Through this interface, the value inside the module can be obtained in real time.
Let’s look at the following example:
foo.js
console.log('foo is running');
import {bar} from './bar'
console.log('bar = %j', bar);
setTimeout((a)= > console.log('bar = %j after 500 ms', bar), 500);
console.log('foo is finished');
Copy the codebar.js
console.log('bar is running');
export let bar = false;
setTimeout((a)= > bar = true.500);
console.log('bar is finished');
Copy the codeNodefoo.js will output the following:
bar is running
bar is finished
foo is running
bar = false
foo is finished
bar = true after 500 ms
Copy the codeIs it different from what you expected? When we execute Node foo.js, the first line prints not the first console statement in foo.js, but the first console statement in bar.js. This is because the import command is executed at compile time and is statically parsed by the JavaScript engine before the code runs, so it takes precedence over foo.js’ own content. At the same time, we also see that the updated value of bar can be obtained after 500 milliseconds, which also indicates that the interface of export command output is dynamically bound to its corresponding value. This design allows the program to determine module dependencies at compile time, which is a major difference from the CommonJS module specification. It is also important to note that since import is executed statically, it has the effect of boosting the program, i.e. the position of the import command does not affect the output of the program.
Now that we’ve looked at ES6’s module loading mechanism, let’s take a look at how ES6 handles circular dependencies. Modify the above example:
foo.js
console.log('foo is running');
import {bar} from './bar'
console.log('bar = %j', bar);
setTimeout((a)= > console.log('bar = %j after 500 ms', bar), 500);
export let foo = false;
console.log('foo is finished');
Copy the codebar.js
console.log('bar is running');
import {foo} from './foo';
console.log('foo = %j', foo)
export let bar = false;
setTimeout((a)= > bar = true.500);
console.log('bar is finished');
Copy the codeNodefoo.js will output the following:
bar is running
foo = undefined
bar is finished
foo is running
bar = false
foo is finished
bar = true after 500 ms
Copy the codeFoo. js and bar.js form loop dependencies, but the program does not report an error if it falls into a loop call. Why is this? Because import is executed at compile time, it allows the program to determine module dependencies at compile time. Once a cyclic dependency is found, ES6 itself will not execute the dependent module, so the program can end normally. This also shows that ES6 has built-in support for circular dependencies, which ensures that programs don’t get stuck with infinite calls. Still, try to avoid loop dependencies in your program, because things can get confusing. Bar.js export foo = false, bar.js export foo = false, bar.js export foo = false, bar.js export foo = false This is the confusion of circular dependencies. In large, complex projects, circular dependencies are hard to see with the naked eye, which can make troubleshooting extremely difficult. For projects built using WebPack, the circular-dependency plugin is recommended to help you detect all circular dependencies in your project. Spotting potential circular dependencies early can save you a lot of trouble in the future.
summary
Hopefully this article has given you a better understanding of dependency management in JavaScript and how to deal with circular dependencies in your projects.