Quick summary

Locking files is useful if you are building an application that resembles a Web server. However, if you publish libraries or CLI to NPM, never publish lock files. Using lock files means that your users and you may be using different versions of dependencies.

What is a lock file?

A lock file describes the entire dependency tree as it is resolved at creation time, including nested dependencies with a particular version. In NPM, these are called package-lock.json and in YARN, they are called yarn.lock. In NPM and YARN, they are placed next to package.json.

Package-lock. json looks like this:

{
 "name": "lockfile-demo"."version": "1.0.0"."lockfileVersion": 1."requires": true."dependencies": {
   "ansi-styles": {
     "version": 3.2.1 ""."resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz"."integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA=="."requires": {
       "color-convert": "^ 1.9.0"}},"chalk": {
     "version": "2.4.2"."resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz"."integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ=="."requires": {
       "ansi-styles": "^ 3.2.1." "."escape-string-regexp": "^ 1.0.5." "."supports-color": "^ 5.3.0." "}}}}Copy the code

The yarn.lock file format is different, but contains similar information:

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1


ANSI - styles @ ^ 3.2.1:
  version 3.2.1 ""
  resolved "https://registry.yarnpkg.com/ansi-styles/-/ansi-styles-3.2.1.tgz#41fbb20243e50b12be0f04b8dedbf07520ce841d"
  integrity sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==
  dependencies:
        color-convert "^ 1.9.0"

Chalk @ ^ 2.4.2:
  version "2.4.2"
  resolved "https://registry.yarnpkg.com/chalk/-/chalk-2.4.2.tgz#cd42541677a54333cf541a49108c1432b44c9424"
  integrity sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==
  dependencies:
        ansi-styles "^ 3.2.1." "
        escape-string-regexp "^ 1.0.5." "
        supports-color "^ 5.3.0." "
Copy the code

Both documents record some important information:

  1. The actual version of each dependency installed
  2. Dependencies for each dependency
  3. A parsed package that includes a checksum to verify package integrity

So, if all dependencies are listed in the lock file, why list them in package.json? Why do we need two files?

package.json vs. Lock File

The purpose of the Dependencies field in the project “package.json” file is to show the dependencies that should be installed, not the dependencies of those dependencies. Dependencies can specify the exact version or within the Semver version number range. For those using semver version number rules, NPM or YARN will select the version most appropriate for installation.

This means that if you run NPM Install twice during a new release, you might actually get versions of different dependencies. For example, if you use NPM Install Twilio to install a dependency called Twilio. Json might have a similar entry:

{
  "dependencies": {
     "twilio": "^ 3.30.3"}}Copy the code

If you look at semver’s documentation on the NPM page, you will see that ^ actually means that any version greater than 3.30.3 and less than 4.0.0 is valid. So, if any new version is released and you don’t have a lock file in place, NPM or YARN will install that new version and will not automatically update package.json. However, this is not the case if there are lock files.

If NPM or YARN finds their respective lock files, they will use them for module installation. This is particularly useful in situations such as continuous integration (CI) on platforms where you need to ensure that tests run in a predictable environment. For this use case, you can use special commands or flags with the corresponding package manager:

npm ci The contents of package-lock.json will be installed exactly
yarn install --frozen-lock-file # Install the contents of yarn.lock exactly. Do not update lock
Copy the code

This is very useful when building applications such as Web applications or servers, because in a CI environment, we want to simulate the behavior of users. So, if we start tracking lock files (such as Git) in source control, we can ensure that every developer, server, build system, and CI system uses the same version of the dependency.

So why wouldn’t we want to do the same thing when we’re writing libraries or anything else we want to publish to an NPM repository? To answer this question, let’s first talk about how outsourcing works.

How to publish modules

Contrary to popular belief, content posted to NPM is not always the same as content on GitHub, nor is it always the same as content in the project as a whole. The way modules are published is that NPM will determine which files to publish by examining the files configuration and.npmignore files in the package.json file. If you do not have a. Npmignore file, use a. Gitignore file. Some files are always included and others are always excluded. You can find a complete list of these files on the NPM page. For example, the.git folder is always ignored.

After that, NPM takes the list of files and packs them into a tarball using the NPM Pack. If you want to see which files are packed, you can run the NPM pack –dry-run command, which will output all files:

The tarball will then be uploaded to the NPM package repository. When you run this command, you may notice that package-lock.json is not pushed into the package. This is because package-lock.json is always ignored, as specified by the list in the NPM document.

This means that if other developers install your distributed packages, they will never download your package-lock.json. Therefore, lock files are not affected at all during installation.

This can accidentally lead to a “good on my machine” effect, because your CI and developer environments may get different version dependencies. So what should we do?

Disable lock files and Shrinkwrapping files

First, we should make sure to stop tracking lock files. If you are using Git, add the following to your project. Gitignore file:

yarn.lock
package-lock.json
Copy the code

The yarn official documentation says that even if you write a library, you should register a yarn.lock file. However, if you want to ensure that you have the same experience as the user, I recommend adding yarn.lock to.gitignore.

You can turn off the generation of package-lock.json files. By creating or adding the following to the.npMRc file in your project

package-lock = false
Copy the code

For yarn, you can use the yarn install –no-lockfile command to not generate a lockfile.

However, just because we don’t use package-lock.json files doesn’t mean we can’t lock dependencies and child dependencies. We can also use another file called npm-shrinkwrap. Json.

It is basically the same as package-lock.json. Generated by the NPM shrinkwrap command, and actually packaged and published to the NPM repository.

Therefore, by adding NPM Shrinkwrap to NPM scripts as pre-packaged scripts or even Git commit hooks, you can ensure that the same version of the dependency is used in your development environment, your users, and your CI.

It’s important to be careful when you use it. By installing dependencies with the Shrinkwrap file, you can lock in the exact version, which can be great, but it can also prevent people from getting critical patches. NPM strongly advises against using Shrinkwrap for libraries and recommends using shrinkwrap only in CLIs or similar scenarios.

How can I get more information

Although there is a lot of information about this in the NPM documentation, it is sometimes hard to find. If you want to get a better idea of what to install or package, a common command is dry-run. Running this command does not affect applications. For example, NPM install –dry-run does not actually install dependencies into the project directory, and NPM publish –dry-run does not actually publish packages.

Here are some commands you may want to check out:

npm ci --dry-run # Simulate installation with package-lock.json or nPM-shrinkwrap
npm pack --dry-run # lists all files to be packaged along with meta information
npm install <dep> --verbose --dry-run # Run the installation of the package in verbose mode without actually installing it on the file system
Copy the code

Links to some useful documentation on this topic are as follows:

  • What is the difference between NPM CI and NPM Install
  • Package-lock. json and NPM-shrinkwrap.
  • A list of files that are always packaged and always ignored

Much depends on how NPM handles packaging, publishing, and installing dependencies, and this may change at some point in a changing environment. I hope this article has given you more insight into this complex topic.

When Not to Use Lock Files with Node.js