• Replacing Lerna + Yarn with PNPM Workspaces
  • Raul Melo
  • The Nuggets translation Project
  • Permanent link to this article: github.com/xitu/gold-m…
  • Translator: CarlosChenN
  • Proofreader: XYJ1020 FinalWhy

Replace Lerna + Yarn with PNPM Workspaces

In recent years, the Monorepo architecture has become increasingly popular, which is understandable given the problems it solves. However, the biggest challenge is finding an easy-to-use tool to handle similar architectures.

If you search for “Monorepo Tool javascript”, you’ll find a number of articles that show us some of the most popular tools of the moment, but it’s odd that each of these tools solves problems that Monorepo solves in a very different way.

From our current options, some tools (such as Lerna) have been around for a while but are no longer maintained on an ongoing basis; Others, such as Bolt, have yet to be approved. There are also tools that work, but only for a specific kind of project.

Unfortunately, we don’t have a good tool for all JavaScript/Typescript projects and all team sizes, but that’s understandable.

However, there is now a (” new “) option that might help us in most scenarios: PNPM WorkSpaces.

But before I get to PNPM, let me tell you about my monorepo/ Workspaces usage and how I tried to work around the problems I had with Monorepo.

My blog

When I first started my blog, I created a Next. Js application, put it in a Git repository, and committed (or pushed) the scaffolding code into the same repository.

After that, I need to set up a CMS (content management system) to hold my content. Then, I create a Strapi application, put it in another Git repository, and push it to another Github repository.

Then, I decided to fork a library called MDX-PRISM to fix some of its glitches and make it deploy automatically. Similarly, I created a new Git repository with the mdX-PRISM code and its initial configuration.

I have 3 Git repositories, which means I have 3 ESLint, prettier, Jest, Babel, and typescript configurations, which means I have 3 ESLint, prettier, Jest, Babel, And typescript configuration, but I maintained it for a while.

I soon became frustrated with every dependency (like TypeScript) update because I needed to update three repositories at a time, which meant I had to pull three times. Every time I learned something new, like new rules for ESLint, I had to change code in three code bases, etc.

My first thought was:

What if I put all my projects into a single folder and repository, created my base configuration, and then used it to extend to each project configuration?

Unfortunately, I can’t simply put the files together, extend the configuration and hope it works, because it’s a lot more complicated than that. These tools require module/file parsing, and I don’t want to release all projects before deploying the service.

At this point, I realized THAT I needed a Monorepo tool to connect disparate projects and make my experience better.

I tried some solutions, the easiest way to build and run is Lerna + Yarn Workspaces.

Of course, in the build process, I have some insight, such as understanding why some builds fail (not all apps like to promote dependencies), having to match my pipelines, and how I deploy each project. Still, I manage everything and have a proper configuration.

With the simplest setup I had before, I started creating smaller stand-alone modules/applications to reuse, extend, and experiment with new tools without affecting my existing code. This is the amazing moment I saw it in action in a Monorepo.

About Lerna + Yarn Workspaces

Lerna is an advanced Monorepo tool that provides abstractions for managing one or more applications/packages simultaneously.

You can control all of your projects by running a single command (e.g. Build, test, Lint, etc.), or filter specific projects by using the –scope tag if you need it.

Yarn Workspaces is a low-level tool that handles package installation, creating symbolic links between projects, and assigning modules under root and controlled project folders.

You can use Lerna or Yarn Workspaces to manage your warehouse, but you may notice that the two tools are more complementary than they are exclusive. In other words, they work well together.

Until now, this combination has been a good choice for practicing monorepo, but some “problems” may pop up:

  • Yarn Workspaces (V1) are no longer maintained (last updated in 2018);
  • Lerna documentation is fine (but not detailed enough) and you need to figure out a lot on your own;
  • The Lerna publishing system is not as simple as it seems, especially when automatic publishing is generated with Commit Lint.
  • You may not really understand what the command you need to run means or you may not know what is running while you are running other commands;
  • Lerna CLI has some issues like you can’t install multiple dependencies at the same time.
  • Lerna CLI --scopeUnreliable and difficult to understand and use;
  • There is a Wizard to help us with common tasks, but it’s more like it’s maintained outside of the main repository.
  • Lerna is currently not maintained;

Lerna was created in 2015, the emergence of this tool helps us to solve the current situation of the lack of tools to manage JS Monorepos, and it does a good job of managing JS Monorepos.

However, Lerna is slowly stepping back from managing JS Monorepos because it doesn’t have a dedicated team or staff to update and plan for the tool’s future.

I’m not here to blame creators and maintainers, there are many problems in the open source world, but that’s the subject of another article.

Now you might be thinking something like this:

If Lerna were in this era, what options do we have now?

PNPM profile

If you don’t know, NPM, like Yarn and PNPM, is also a package management tool for JavaScript projects. It does the same thing, only more efficiently.

The biggest benefit of using PNPM is that PNPM solves the problem of how NPM is introduced and also copied by Yarn, that is, the way NPM and Yarn install dependencies.

PNPM aims to solve two big problems:

Disk space

Let’s say you have five projects that include [email protected] as a dependency.

When you install NPM or Yarn in all projects, each project will have its own copy of React in node_modules. Given that a React package is about 6.9kB in size, we would have 34.5kB of the same dependencies on disk across five repositories.

This example may seem small, but as anyone who uses JS knows, sometimes node_modules directories can easily reach gigabytes in size.

If PNPM is used to install dependencies, it first downloads the dependencies to its own “repository” (~/.pnpm-store). After that, PNPM will create a hard link to that module in node_modules in your project.

To take the previous example, PNPM will install [email protected] in its own storage folder, and when we install the project’s dependencies, it will first check to see if the 17.0.2 version of React has been saved. If so, it creates a hard link (pointing to a file on disk) in the project’s node_modules.

Compared to keeping 5 copies of [email protected] (34.5kB) on disk, Currently we only have one version of the STORE in PNPM (6.9KB) and a hard link in each project that has the same functionality as the React copy.

As a result, we save a lot of disk space, and if our new project uses dependencies that we already have installed, it will be faster to install them.

Ghost rely on

When we install dependencies with NPM, it packs all dependencies and everything in them into the node_modules folder. This approach is known as “flattening.”

Let’s see this in practice. Here is package.json:

{
  "dependencies": {
    "unified": "10.1.0"}}Copy the code

After running NPM install, node_modules looks like this:

Node_modules ├─ @Types ├─ Extention ├─ IS-Buffer ├─ IS-Plain-OBJ ├─ Unified ├─ Unist - util - stringify - position ├ ─ ─ vfile └ ─ ─ vfile - the messageCopy the code

This approach has worked for many years, but it can lead to a problem we call “ghost dependency.”

(for example) The only dependency declared in our project is Unified, but we can still reference the IS-Plain-OBj module (Unified’s dependency) in our project code:

import ob from "is-plain-obj";

console.log(ob); // [Function: isPlainObject]
Copy the code

Since this can happen, the dependency we declared and the dependency’s dependencies can also have the problem of importing a dependency from node_modules without declaring a dependency as a dependency or peerDependency.

Now, let’s see how PNPM handles this.

Using the same package.json, and then running PNPM install, we’ll have the following node_modules:

├──.pnpm ├── @types ├──.pnpm/[email protected]/node_modules/unified ├─.modulesCopy the code

As you can see, node_module has the unified dependency only, but an arrow indicates that the module is a symbolic link.

Then, let’s see what’s inside.pnpm:

Node_modules ├ ─ ─ the PNPM │ ├ ─ ─ @ types + [email protected] │ ├ ─ ─ [email protected] │ ├ ─ ─ [email protected] │ ├ ─ ─ [email protected] │ ├ ─ ─ [email protected] │ ├── Node_modules │ ├─ [email protected] │ ├─ [email protected] │ ├─ [email protected] │ ├ ─ ─ [email protected] │ ├ ─ ─ [email protected] │ └ ─ ─ the lock. The yaml ├ ─ ─ @ types │ └ ─ ─ unist - >.. /. PNPM /@[email protected]/node_modules/@types/unist ├─ unified ->.pnpm/[email protected]/node_modules/unified ├─ .modules.yamlCopy the code

PNPM will install each dependency in the corresponding directory of.pnpm (package name + version) and then “move” those dependencies that you have explicitly defined in your project’s package.json. You actually create a symbolic link pointing to the corresponding module in.pnpm) to the node_modules of your project.

Now, if we try to write the same code as before, we’ll get an error because IS-plain-obj is not installed in node_modules:

internal/process/esm_loader.js:74
 internalBinding('errors').triggerUncaughtException(
 ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'is-plain-obj' imported from /Users/raulmelo/development/sandbox/test-pnpm-npm/pnpm/index.js
Copy the code

Although node_modules are more reliable to install in this way, it can break the compatibility of PNPM with applications built on flat node_modules structures.

One example is Strapi V3. As you can see here, they are aware of this and will work it out in the future.

Fortunately, THE PNPM considers these issues and shamefully provides a flag called the hoist to address them.

When we use this tag, project dependencies are installed in a “flat fashion,” which makes applications like Strapi work.

pnpm Workspaces

PNPM introduced the Workspace feature in V2.

It is designed to fill the void of monorepo tools that are easy to use and well maintained today.

Because they already have the underlying tool (package manager), they aim to add a new module to PNPM to handle the workspace, as long as you create a pnpm-workshop.yaml file in the root of your project.

It is almost the same configuration as Lerna + Yarn Workspaces, but has three significant advantages:

  1. We control disk space repair from PNPM;
  2. The PNPM CLI is nice to use (it’s built-in);
  3. It addresses many of the problems with Lerna CLI, such as filtering and installing multiple versions of the same package.

The –filter identifier is supported by (almost) all PNPM commands. I think the meaning of this identifier is self-explanatory, but it means that this command will only be executed for the filtered repository.

Suppose you have two complete applications, each with its own pipeline. With Lerna + NPM or Yarn, when we perform the installation, we install the dependencies individually for each project.

This means that, in some cases, we download 1GB of dependency files instead of 300MB.

With PNPM, I can simply run the following command:

pnpm install --filter website
Copy the code

For now, only the root dependencies and my site dependencies will be installed.

The filter command is convenient enough, but it does more than that and provides more flexibility.

I highly recommend you read PNPM’s “Filtering” document and see how amazing it is.

Another suggestion: “PNPM vs Lerna: Filtering in a multi-package warehouse”

It may seem like an incredibly small thing, but when you’re working in a different environment, it’s those little details that can make a big difference.

The migration

If you look at the PR I have merged that contains all the migration changes, you can see it here. I’ll just highlight all the changes I need to show.

Replace the command

I have many scripts to execute yarn CLI. For these, I just need to replace them with PNPM or PNPM run ;

Example Remove the Yarn Workspace configuration

In my package.json, I’ve defined the workspace directory for Yarn and defined some packages that don’t need to be promoted to the root node_modules directory;

{
"workspaces": {
   "packages": [
     "packages/*"."apps/*"]."nohoist": [
     "**/netlify-lambda"."**/netlify-lambda/**"]}}Copy the code

All of these Settings are past tense.

withpnpm-workspace.ymlInstead oflerna.json

I have removed the following configuration:

{
  "version": "independent"."packages": ["packages/*"."apps/*"]."npmClient": "yarn"."useWorkspaces": true."command": {
    "version": {
      "allowBranch": "main"
    },
    "publish": {
      "conventionalCommits": true."message": "chore(release): publish"."ignoreChanges": ["*.md".! "" packages/**/*"]}}}Copy the code

Replaced by:

prefer-workspace-packages: true
packages:
  - 'packages/*'
  - 'apps/*'
Copy the code

Adapter Pipelines, Dockerfiles, and master platforms

One thing I had to change in my Github Actions, Docker images, and Vercel installation scripts was to make sure PNPM was installed before installing project dependencies:

npm install -g pnpm && pnpm install --filter <project-name>
Copy the code

This is an essential step because most environments contain YARN out of the box, not PNPM (I hope this will change soon).

removeyarn.lockfile

This file is no longer needed. Pnpm creates its own pnpm-lock.yaml lock file to control dependent versions.

Adaptive build command

When I run lerna Run Build for my site, it also automatically builds those packages that I use in my site project.

For PNPM, I must be clear:

pnpm run build --filter website # Build websites only

pnpm run build --filter website... # First build all the dependencies used by the website project, and then start to execute my website project build
Copy the code

These declarations are important because I do not publish all packages in NPM.

add.npmrc

PNPM receives a bunch of identifiers and options through the CLI. If I don’t want to keep passing them, we can define them in a.npmrc file.

The only option I added there was:

shamefully-hoist=true
Copy the code

As I said earlier, Strapi doesn’t match the way PNPM installs node_modules, which is awkward.

By submitting these files, I guarantee that the dependencies will be installed correctly wherever I run PNPM Install.

Replace semantic-Release with Changesets

I have to confess, I haven’t fully tested this yet.

In summary, in my previous setup, I was forced to write commit information in a specific way so that semantic publishing could check my changes, automatically recognize what changed by reading commit information, change the version number, and publish my package.

It has been working fine, but there are still some issues, especially considering the way the Github Actions environment works.

So Pnpm suggested we use Atlassian’s Changesets.

This is a little bit different. Now if I commit a change, I have to create a.md file with some meta information and description, based on which Changesets will know how to generate a change log and which version to change.

I still need to finish this setup and possibly write an article about it. 😅

conclusion

This is all the basics I need to replace Lerna + Yarn Workspaces with PNPM Workspaces.

To be honest, it was easier than I originally thought.

The more I use PNPM, the more I enjoy it. The project is solid and the user experience is enjoyable.

reference

  • pnpm.io
  • github.com/lerna/lerna
  • Classic.yarnpkg.com/lang/en/doc…
  • Medium.com/pnpm/pnpm-v…
  • Github.com/raulfdm/rau…
  • medium.com/@307/hard-l…

If you find any mistakes in your translation or other areas that need to be improved, you are welcome to the Nuggets Translation Program to revise and PR your translation, and you can also get the corresponding reward points. The permanent link to this article at the beginning of this article is the MarkDown link to this article on GitHub.

The Nuggets Translation Project is a community that translates quality Internet technical articles from English sharing articles on nuggets. The content covers Android, iOS, front-end, back-end, blockchain, products, design, artificial intelligence and other fields. If you want to see more high-quality translation, please continue to pay attention to the Translation plan of Digging Gold, the official Weibo, Zhihu column.