first post on http://blog.xgheaven.com/2018/05/03/npm-to-yarn-to-npm/

NPM package management is an integral part of getting in touch with the Node environment, but due to the various issues with the official NPM, there are many different versions, and the twists and turns are probably only known from the past.

Give up NPM?

In ancient times

In older versions of NPM, the installation policy was not flat, meaning that if you were to install express, you would only find an Express folder under node_modules. The projects that Express depends on are in its folders.

- app/
  - package.json
  - node_modules/
    - express/
      - index.js
      - package.json
      - node_modules/
        - ...
Copy the code

This is a problem that Windows users may be aware of, because in this installation environment, the directory hierarchy is extremely high. For Windows, the maximum path length is limited to 248 characters (see more here), plus the word node_modules is extremely long. So you get it, ha ha ha. Solution what of their own search, anyway, now estimated that no one will use the ancient version.

In addition to the problems Windows users have, a more serious problem is that the modules are separate. For example, path-to-regexp under Express and Path-to-regexp under CONNECT are two different modules. So what’s the impact of this? In fact, in use, there is no too big impact, but the memory occupation is too large. Multiple instances can occur because many of the same modules are under different modules (see Node module loading for why multiple instances are loaded). When you think about it, it’s the same function. Why do you want to do it so many times? Can’t you just load it once and reuse the instance?

The shortcomings of the ancient NPM can be said to be many:

  • The directory nesting level is too deep
  • The module instance cannot be shared
  • The installation was slow because of directory nesting and installation logic problems. Because NPM requests one module and then another, only one module is being downloaded, parsed, and installed at any one time.

Soft chain era

Behind, someone in order to solve the problem of directory nesting level is too high, the introduction of soft link scheme.

Simply put, all packages are installed flat into one location and then combined into node_modules via soft links (Windows shortcuts).

- app/ - node_modules - .modules/ - [email protected]/ - node_modules - connect -> .. /.. /[email protected] - path-to-regexp -> .. /.. /[email protected] - ... - >.. /.. /[email protected] - [email protected]/ - [email protected]/ - ... others - express -> ./.modules/[email protected]Copy the code

This has the advantage of reducing the overall logical hierarchy to just a few layers. Moreover, for node module resolution, it can solve the problem of loading multiple instances caused by different locations of the same module, thus leading to the situation of memory occupation.

Based on this scheme, npminstall and the package PNPM implement this scheme. CNPM uses NPminstall. However, their implementation methods are different from what I have mentioned above. Simply put, they don’t have.modules. For more, see Npminstall’s README.

In general, this solution has the following benefits:

  • Good compatibility
  • Two of the above problems (directory nesting and multi-instance loading) are solved while keeping the directory concise.
  • Installation speed is very fast, because the use of soft connection and multi-threaded request, multiple modules at the same time download, analysis, installation.

Then shortcomings can be quite fatal:

  • In general, third-party libraries implement this function, so there is no guarantee of the same behavior as NPM, so if there is a problem, you have to go to the author to submit it, and wait for the fix.
  • Does not work well with NPM. It is best to use either NPM alone or CNPM/PNPM alone, the combination of the two can be quite bizarre.

Npm3 era

The biggest change is to change the directory level from nested to flat, which can be said to be a good solution to the above nested level too deep and instances do not share. However, nPM3 does not choose the soft connection mode in the flat scheme, but instead installs all modules directly under node_modules.

- app/
- node_modules/
  - express/
  - connect/
  - path-to-regexp/
  - ...
Copy the code

If there are different versions of dependencies, such as package-a depending on the [email protected] version and package-b depending on the [email protected] version, then the solution remains as nested as before.

- app/
- node_modules/
  - package-a/
  - package-c/
    - // 0.x.x
  - package-b/
    - node_modules/
      - package-c/
        - // 1.x.x
Copy the code

As for which version is outside, which version is inside, seems to be based on the order of installation, I will not verify the specific. If anyone knows, please let me know.

After this release, most of the problems were solved and NPM entered a new world. But there is also a problem is that its installation speed is still very slow, compared to CNPM. So he still has a lot of room for improvement.

The birth of the yarn

As the Node community grows, more and more people are applying Node to enterprise projects. This also exposes NPM to a number of problems:

  • There is no guarantee that the versions installed will be identical. As you all know, NPM installs applications with semantic version numbers. You can limit the version number of the modules you install, but you can’t limit the version number of the modules that you install. Shrinkwrap exists, but few people use it.
  • The installation speed is slow. As mentioned above, in some large projects, thousands of packages may be relied upon, even including C++ Addon, which can take 10 minutes or even half an hour to install in severe cases. This is obviously intolerable, especially with CI/CD.
  • NPM does not support offline mode by default, but in some cases the company’s network may not support Internet connections, and it is convenient to build applications using caching. And network requests can be greatly reduced.

Therefore, YARN was born to solve the above problems.

  • The yarn.lock file is introduced to manage dependency version issues and ensure that each installation is consistent.
  • Cache and parallel download ensure the installation speed

At that time, I was still using CNPM. I made a comparison and found that CNPM was faster, so I continued to use CNPM because it was enough for me. But later found that YARN is really more and more popular, coupled with CNPM does not update for a long time. I also tried yarn, and after trying it, I gave up CNPM completely. And until now, it doesn’t seem to have added lock functionality.

Of course, YARN has more than just these advantages in terms of user usage:

  • Provides very succinct commands that group related commands, for exampleyarn globalThe commands below are all related to the global module. And the hints are so complete that you can see what they mean at a glance. It’s not like NPM,npm --helpIt’s just a bunch of strings, and I don’t have to explain what it’s for.
  • The installation is saved to Dependencies by default and does not need to be added manually like NPM-Sparameter
  • The very convenient YARN run command not only automatically looks at the contents under scripts in package.json, but also looks upnode_modules/.binExecutable file under. This is the highest frequency I can use YARN. Let’s say you installyarn add mochaAnd then you can go throughyarn run mochaRun directlymocha. Without the need for./node_modules/.bin/mochaRun. Is one of my favorite features
  • Interactive versions depend on updates. NPM you can only pass firstnpm outdatedLook at which packages need to be updated and passnpm update [packages]Updates the specified package. In YARN, you can use interactive mode to select those that need to be updated and those that do not.
  • Management of global modules. NPM manages global modules directly in the/usr/lib/node_modulesInstall below, then connect to via soft connection/usr/local/binDirectory. Yarn, on the other hand, selects a directory where global modules are installed, and manages all global modules as a project. The advantage is that you can directly back up package.json and yarn.lock files in this directory, making it easy to restore the global modules you installed in another location. As for the question of this directory, yesyarn global dirCommand can be found, MAC is under the~/.config/yarn/global/I haven’t tested Linux.

It can be said that YARN is very comfortable to use, but the only disadvantage is that it is not officially issued by NPM, and the update strength and compatibility will be poor. But that doesn’t stop the popularity of Yarn in the Node community. Soon, everyone switched from NPM to YARN.

Regain the NPM 5

After the impact of YARN, NPM officials decided to address these shortcomings and released a version of NPM5 to counter (that’s a term I coined) Yarn.

  1. Package-lock. json has been introduced, and is added by default, to act as yarn.lock, and replace NPM shrinkwrap.
  2. By default, the installation adds dependencies automatically; you do not need to write them manually-Sparameter
  3. The installation speed is improved, but it is slightly slower than YARN

At this point, the difference between YARN and NPM is very narrow. The difference is more in user experience. The only functions I can use yarn are global module management, module interactive update, and YARN run.

However, the introduction of NPX made me give up using yarn run. It is not that NPX is better than YARN, but that NPM integrates this functionality so there is no need to use third-party tools. NPX also supports temporary installation modules, which are used only once and then deleted.

Then I found the nPm-check tool, which I used instead of yarn interactive update.

However, the advent of NPM6 with the addition of caching, and further speed, can be said to be close to YARN.

So yarn only has a global module management function left for me. My whole development process and switch from YARN back to NPM. Maybe I’ll let NPM take over global module management at a later date and give up using YARN. But I will still install YARN, after all, some old projects still use YARN.

conclusion

I went through a cycle from NPM -> CNPM -> YARN -> (NPM + NPm-check + NPX) and witnessed the development of the NPM community step by step. In addition, the update frequency of YARN is very slow, maybe once a month, which gradually made me give up using YARN.

Sometimes feel, the third party is the third party after all, or no original easy to use and convenient, and use at ease.