preface

Hi ~ It’s been a while. The stateof JS 2021 survey was released a few days ago, and this year it added a survey report on Monorepo Tools (link: 2021.stateofjs.com/en-US/libra…). .

PNPM is the most popular monorepo tool chain in 2021. At the same time, the breadth of user use and other aspects have also achieved good results.

Just some time ago, I debugged PNPM for a while and contributed some code, so I have some understanding of the structure of PNPM. This article will give you a code structure analysis within the scope of my understanding. If you have any questions, you can directly point out and discuss and learn together.

Introduction to source code structure

First of all, PNPM codes are mainly concentrated in the packages directory under the root directory. PNPM itself uses PNPM workspace as the management tool of Monorepo. Some modules in PNPM workspace are stored as independent sub-packages under the Packages directory.

Because the PNPM itself monorepo main management are some of the tools related to library package, so its use of contract awarding scheme is changesets, specific can refer to my previous articles: mp.weixin.qq.com/s/DkXmsAGcT…

The directory structure for subpackages can refer to the following structure:

// Rely on the installation of the core logic code package ├─ core // core log output package ├─ core loggers // log print package ├─ default-reporter // rely on the package path of the package (including soft link, etc.) ├─ Dependency path // Flag Exercises for filter-workshop-packages // Flag Exercises for workshop-packages // Flag Exercises for workshop-packages // Flag Exercises for workshop-packages ├─ Find - Workshop-dir ├─ Find - workshop-dir ├─ Git -fetcher ├─ Git -resolver ├─ Find - workshop-dir ├─ git-fetcher ├─ git-resolver ├─ find- Workshop-dir ├─ git-fetcher ├─ git- Hoist // lifecycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle cycle Lockfile-utils ├─ Lockfile-walker ├─ Normal-registries ├─ Npm-registries Agent ├── Parse-wanted dependency ├── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ─ Pkgs-graph // plugin-commands ├─ plugin-commands-audit ├─ plugin-commands-env ├─ pkgs-graph // plugin-commands ├─ plugin-commands-env └ ─ Plugin-commands-installation ├─ plugin-commands-listing ├─ plugin-commands-outdated ├─ plugin-commands-publishing ├─ Plugin-commands-script-runners ├─ plugin-commands-server ├─ plugin-commands-setup ├─ plugin-commands-store // PNPM ├─ PNPM // Read The.pnPMfile.cjs package ├─ PnPMFile // Read the pkg.json tool ├─ read-project-manifest // To improve the PNPM ├─ Real -hoist // Visual Output Dependency On peerDep Problem ├─ render-peer-issues Dependency Dependency Use ├─ Flag Flag flag flag Flag Flag flag flag flag Flag Flag Flag ├─ Sort Packages ├─ Store-Connection-Manager ├─ Store-controller-types ├─ symlink-dependency ├─ Tarball -fetcher ├─ tarball-resolver └ ─ symlink- Dependency ├─ tarball-fetcher ├─ tarball-resolver └ ─ The write - project - the manifest └ ─ ─...Copy the code

PNPM itself has a lot of packages in it, and IN the tree above, I have omitted some of the less used or nearly obsolete packages (even so, there are still many, many packages…). .

Here I will introduce the structure of the code based on the command line on the PNPM website. In fact, there are many commands that encapsulate the code using the same module. For example, install, update, and add.

The main entrance

First of all, the main entry file of PNPM is packages/ PNPM. This package is also called PNPM. The main. Then according to the different parameters after processing, do a delivery execution work for each command, and then send the command parameters to each package to execute the corresponding logic inside.

The @pnpm/parse-cli-args package takes the command line arguments passed in by the user and processes them into a unified format within PNPM. For example, the user enters the following command:

pnpm add -D axios
Copy the code

Some of the arguments passed in here will be handled by the parseCliArgs method:

For example, add is handled in the CMD field, some bare arguments such as axios are placed in the cliParams array, and -d is placed in the cliOptions array. These processed variables and parameters are used to determine subsequent code execution logic in the main entry file. The specific judgment logic can be encountered in the debugging, and then go to the corresponding entry logic to judge the debugging, there is no specific introduction.

Other internal packages used in the entry package are @pnpm/ find-workshop-Packages and @pnpm/ filter-workshop-Packages.

  • findWorkspacePackagesSome information (such as names, paths, and so on) for finding all the packages in the PNPM Workspace (for the Monorepo project) in the entry file.
  • filterPackagesThis is a relatively key package, and PNPM has an official document dedicated to it--filterThis functional module (see:PNPM. IO/filtering),…The PNPM command provides a very simple and useful filter function to filter subpackages under Monorepo based on the filter parameters passed in by the user.Output the corresponding package selected by the manager and related information.

In main.ts, commands are distributed by calling the method (pnpmCmds) in the CMD directory under the current package.

  • If CMD isadd 、install 、updateIf these involve packages related to the dependent installation, they will go@pnpm/plugin-commands-installationThis package contains the corresponding subcommand logic (basically all of PNPM’s core modules revolve around dependency installation).
  • If CMD ispack 、publishThis category involves packages that are packaged and published, and goes@pnpm/plugin-commands-publishingLogic of this package.
  • If CMD isrun 、exec 、 dlxMethods that are associated with command execution will go@pnpm/plugin-commands-script-runnersLogic of this package.

See the command mount details in PNPM/SRC/CMD for more logic here.

Below I will explain the logic involved according to CLI Commands on the official website.

Dependency management

This part can be said to be the most core part of the whole PNPM, which involves PNPM install, PNPM add

and other dependency management related core commands.

The logic mentioned in the previous section is mainly implemented in the @pnpm/plugin-commands-installation package under PNPM. This is just a brief introduction of the logic and the referenced package without specific discussion. Because the PNPM principle of dependency installation really combined with the code to introduce the principle, you can write a whole article.

The core logic of dependency management is in the SRC /installDeps directory of the corresponding package directory. The last logic of almost all dependency related commands is transferred and executed here. It can be seen that the core logic of install, add, and update commands is executed in this section. The logic conversion is based on the parameters passed in by the user:

 const result = await mutateModules([
  {
    ...mutatedProject,
    dependencySelectors,
    manifest: updatedImporter.manifest,
    peer: false,
    targetDependenciesField: 'devDependencies',
  },
], installOpts)
Copy the code

The mutateModules method here comes from the package @pNPM /core, which is the core package of the entire PNPM project. Some key core logic (such as dependency installation, etc.) is implemented here. For details, you can refer to the source code.

Dependency management involves a few other packages:

  • Used to process the lifeCycle method@pnpm/lifecycle
  • To output logs (for example, depending on log printing during installation)@pnpm/core-loggers 、@pnpm/logger
  • Depends on the pnpm-lock.yaml file generated and updated during installation@pnpm/lockfile-file
  • Dependencies that resolve dependencies and pull dependency packages during installation@pnpm/resolve-dependencies

When I debug a bug in PNPM update, I pick it up step by step from plugin-command-installation to resolve-dependencies, and finally find the problem in the statement processing of a library function. For details, please refer to pr: github.com/pnpm/pnpm/p…

Debugging technique

If you want to debug PNPM, there is a CONTRIBUTING. Md document in the source repository for PNPM. The recommended way is to use PNPM run compile to compile the package as a whole. Then debug through node < rePO_dir >/packages/ PNPM [command].

However, in fact, this approach is inefficient. In many cases, the code is modified, and the debugging does not meet the expectations. After the modification is completed, the code needs to be modified again for recompilation.

I used to debug PNPM for a period of time. Here I would like to share some of my personal debugging experience with you.

There is a pnpm. CJS file in the packages/ PNPM bin directory where the require method specifies which piece of logic PNPM should use when executing:

The default logic here is the packaged code logic in the dist directory, PNPM compile the default directory is in the dist directory, but if we just debug here, we can not use the code logic in the dist directory at all.

The author mentioned a PR to PNPM before, and added a section of local product code to the bottom. As can be seen in the screenshot above, when debugging, you only need to comment out the logic of dist code and then go to the code in lib directory:

At the same time, most of the sub-packages being maintained under PNPM are developed using typescipt. I also added TSC –watch command to some libraries:

So if you want to debug PNPM source code in a just-in-time compile way, you can directly go to the corresponding subpackage below the corresponding subpackage start command to run. Then do a debugging job for different subpackages. The following is a debugging process for the author, which can be provided for reference.

Debugging process

For example, debug a subpackage under PNPM, using @pnpm/plugin-commands-installation as an example.

First, you can perform a full compilation of the entire package code, in case the local product is not updated after some package code synchronization, directly in the entire project root directory:

pnpm run compile
Copy the code

This time it may take a bit longer, but it will ensure that the package referenced later and we don’t debug the package is up to date, so as to prevent some packages from requiring problems.

Then go directly to the directory where you want to debug the package, and run the main package. Note that you need to modify the entry code mentioned in the previous section. Here I usually start multiple terminal processes, and then compile the package ts run:

cd packages/pnpm && npm run start
cd packages/plugin-commands-installation && npm run start
Copy the code

Now you can find a real PNPM project to debug.

For example, in the case of the naive- UI (www.naiveui.com/) project, which uses PNPM as dependency management, you can break the plugin-commands-installation code that needs to be debugable. Then use vscode’s debug terminal to debug:

# in the debug directory of the project, for example, the author here is naive - UI node ~ / path/to/PNPM/packages/PNPM installCopy the code

The command will be distributed to the corresponding code logic, and the breakpoint will take effect immediately. Refer to the figure below:

This can be relatively simple and can debug directly against the source code, if there is code modification can also be modified in the source code directly after debugging.

However, such debugging also has a disadvantage, for example, when debugging a library with a deep dependency level, many processes may start at the same time. For example, the following figure shows the breakpoint observation of each library when debugging the PNPM dependency installation process:

There are six processes in total, but overall it’s a lot cleaner than trying to debug and find problems in a build.

conclusion

So far, PNPM has made great achievements in 2021, and we expect more surprising features in 2022 as well. At the same time, more and more contributors are expected to participate in the source code construction of PNPM, and the joint construction may be the most promising package management tool in the future.

Also like this article can bring harvest to everyone, look forward to better and better ~