Recently, The release of Yarn by Facebook has caused a tsunami of front-end nodes. Yarn is faster than NPM and supports offline installation. The biggest impact on NPM is that YARN provides lock by default. That is, after a dependency is installed through YARN, if yarn upgrade is not executed, the version of the module after the dependency is deleted does not change. Shrinkwrap is also available in NPM, but requires users to perform NPM Shrinkwrap to manually lock the version.
After Yarn came out, many developers talked about how the locked-in feature could help improve the stability of online code. While maintaining CNPM/Npminstall, Shrinkwrap has also received a significant number of inquiries due to the optimization we made to install’s process to no longer support Shrinkwrap. So I thought I’d take this opportunity to talk about why I’m not using Shrinkwrap (lock).
Before we dive into this topic, let’s presuppose that shrinkWrap won’t save your project if you choose credible open source modules. In general, reliable modules have the following characteristics:
- Release strictly following semver’s principles of semantic versioning. Avoid modules that do not follow semver’s publishing.
- NPM has a high number of downloads, and when you have a module problem, the more widespread it is, the faster it can be fixed.
- Github provides quick feedback on issues, or is maintained by some well-known developers.
- Patch bit changes are not released much (indicating that there are not many bug fixes).
NPM is different from module managers in other languages (Java and Ruby) by nested dependency. Each module is responsible for its dependencies, resulting in a project directly dependent on a dozen modules, but ultimately indirectly dependent on thousands of modules. It is very difficult to manage a list of thousands of modules.
NPM has introduced the semver semantic version mechanism to help developers manage dependencies. Developers can import modules in package.json either ^1.1.0 or ~1.0.0. If developers trust the modules they depend on, Developers can lock a large version of a module through ^, so that they can enjoy all the new features and bug fixes of the package every time a dependency is re-installed or packaged. If this module follows semver’s principles, there is no need to worry that it will introduce incompatible changes and cause unknown exceptions to the project. In the end, the only thing developers need to care about is whether the modules they rely on directly are reliable enough.
We maintain a very complex Node Web framework within the company (Alibaba/Ant Financial), which has 932 dependency modules scattered in different plug-ins and base libraries and maintained by different students. Our way through the loose (^) to manage the dependence, any plugin or the underlying module provides a new function or a bug fix, don’t need to go we take the initiative to promote business to upgrade, the business just need to trust our framework, the big version specified framework, the next time the reinstall dependence or packaging will be automatically updated.
As framework maintainers, we probably don’t know how many online and offline businesses are relying on our code, and if all business developers are shrinkwrap locking version numbers, there are hidden bugs that may be hard to fix. However, most business development students do not have a thorough understanding of NPM in general, and it is not realistic to expect all business students to update Shrinkwrap frequently. And while maintaining business priorities, it is more likely that it is “stable” to not move any dependent versions without upgrading them during business development. Little do they know that this instead left a bigger hidden trouble.
For example, some time ago, mongodb was used in a project. Due to an internal network outage exercise, a bug of mongodb- Core, the underlying dependent library of Node-mongodb – Native, was triggered, resulting in memory leakage OOM. If you’re using Semver dependencies normally, the bug will be fixed the next time you install a module, but if you’re locking dependencies with Shrinkwrap, you could be the bug’s next victim.
With Semver dependencies, shrinkWrap is a wall in the path of fast forward delivery of new features and bug fixes in a benign environment, and while it may block the introduction of new bugs, it’s not worth the cost. What we really want to do is provide a better environment (relying on solid modules).
There are some problems with not using Shrinkwrap, all of which may be fixable:
If you don’t use ShrinkWrap, you may have dependencies packaged in code packages that run online that are inconsistent with development and testing.
If you are worried about this, it is more important to think about how the overall o&M publishing system can be optimized. The best solution is to start with a package at the start of beta, then test and regression the package until it goes live, ensuring that the code that goes live (including all dependencies) is fully tested. Otherwise, just because you’re shrinkwrap locked doesn’t mean that the code you’re publishing is the same as the code in development and testing. Dependencies that introduce modules that are not in NPM Registry (git, remote URL), or build environments that are inconsistent can cause code to run online and tests to run inconsistently.
After all, testing cannot cover all scenarios. What if there are still some failures due to the upgrade of the underlying module that you rely on?
The first thing to do in the event of an online failure is always to roll back code (if you find that code can’t be rolled back directly to the last release, it’s time to rethink the o&M release process). If there is a good testing and regression process, it is believed that this time the failure is not particularly fatal, and timely rollback to stop the bleeding, to a large extent to reduce the impact of such problems.
Even if the bleeding is stopped in time, there is no way to stop the underlying module from relying on the problem version if the author of the problem module does not fix the bug.
If this is the case, I believe that your development team is already large and it is time to set up a private source within the team. Private sources are completely under the team’s control and can be removed by removing the offending module version or by resetting the latest tag to prevent the offending module from being installed. With CNPM, a fully controlled private source of NPM can be built internally at a very low cost.
We do not have a private source, so we cannot control it internally. What should we do if there is no one to deal with the issue submitted by the module with problems for a long time?
Of the hundreds of thousands of modules on NPM, there are not many that are really popular and well maintained all the time. If we’re going to use Node for some serious business, it’s important to carefully select the modules that are popular in the community, preferably with a good understanding of the modules you’re using. Usually these popular modules have a relatively low incidence of such problems and can be fixed quickly if they do occur. Believe that good modules in the community are self-healing. In the few experiences we’ve had, they can be fixed within half an hour. If such a problem does occur, you need to think back to why you chose such an unreliable module.
If you still have doubts about the above points, welcome to Ali/Ant Financial to experience how the JS/Node world works without Shrinkwrap. 🙂