Original text: classic.yarnpkg.com/blog/2017/0…

Posted by Konstantin Raev on August 2, 2017

Projects tend to grow over time, and sometimes parts of projects will be useful in other projects as well. For example, Jest as a general-purpose testing tool produced many packages, one of which was Jest Snapshot, which is now used in other projects such as Snapguidist and Chai Jest Snapshot.

Monorepos

Anyone who has tried to split a project into multiple packages knows how difficult it is to make changes in multiple packages at once. To simplify this process, some large projects have adopted the Monorepo approach or multi-package repositories, which reduces the burden of writing code across packages.

Several projects that JavaScript developers use every day are managed as Monorepo: Babel, React, Jest, Vue, Angular.

However, separating the parts of a project into their own folders is sometimes not enough. Testing, managing dependencies, and publishing multiple packages quickly became complicated, and many of these projects adopted tools like Lerna to simplify monorepos.

Lerna

Lerna is a tool that optimizes the workflow for managing multiple package repositories using Git and NPM. Internally, it boots a project using Yarn or NPM CLI (that is, installs all third party dependencies for each package). In short, Lerna calls YARN/NPM Install for each package in the project, and then creates symbolic links between packages that reference each other.

As a wrapper for the package manager, Lerna cannot efficiently manipulate the contents of the node module:

  • Lerna calls YARN Install multiple times for each package, which incurs overhead because package.json is considered independent and they cannot share dependencies with each other. This results in a lot of duplication in each node_modules folder, which often uses the same third-party packages.
  • After the installation is complete, Lerna manually creates links between packages that reference each other. This introduces inconsistencies within the node_ module that the package manager may not be aware of, so running YARN Install from the package may break the Lerna managed meta-structure.

Such problems convinced us, as package manager developers, that we should support multi-package repositories directly. Starting with Yarn 0.28, we are pleased to share that we support such repositories under the Workspace feature.

Yarn Working Area

The Yarn workspace is a package.json file that allows users to install all dependencies in one go from multiple package.json subfiles to a single root file.

A native with a workspace set as Yarn enables a faster and lighter installation by preventing package replication across workspaces. Yarn can also create symbolic links between interdependent workspaces and ensure consistency and correctness of all directories.

Setting up Workspaces

Starting with Yarn 1.0 The workspace is enabled by default and you may not need to set the following configuration. Please refer to the following location update steps at https://yarnpkg.com/lang/en/docs/workspaces/

To start, the user must enable the workspace in Yarn by running the following command:

yarn config set workspaces-experimental true
Copy the code

It adds the experimental workspace to the.yarnrc file in the operating system home folder. When we collected feedback from the community, the YARN workspace was still considered experimental.

Let’s take Jest as an example and set up the YARN workspace for this structure. In fact, this has already been done in a PR, and Jest has been booting its packages with Yarn for some time.

Jest’s project structure is typical of the open source JavaScript Monorepo.

| jest/
| ---- package.json
| ---- packages/
| -------- jest-matcher-utils/
| ------------ package.json
| -------- jest-diff/
| ------------ package.json
...
Copy the code

The highest level of package.json defines the root directory of the project, along with the other package.json folders that are the workspace. Workspaces are typically published to registries such as NPM. While the root should not be used as a package, it usually contains glue code or business-specific code that is not useful for sharing with other projects, which is why we mark it as “private.”

The following example is a simplified root package.json that enables the workspace for the project and defines the third-party packages that the project needs to build and test the environment.

{" private ": true," name ":" jest ", "devDependencies" : {" chalk ":" ^ 2.0.1 "}, "workspaces:" [] "packages / *"}Copy the code

For simplicity, I’ll describe two small workspace packages:

  • Jest-matcher-utils workspace:

    { "name": "jest-matcher-utils", "description": "..." , "version": "20.0.3", "License ": "..." , "main": "..." , "browser": "..." , "dependencies" : {" chalk ":" ^ 1.1.3 ", "pretty - format" : "^ 20.0.3"}}Copy the code

  • Jest-diff workspaces depend on jest-matcher-utils:

    {" name ":" jest - diff ", "version" : "20.0.3", "license" : "..." , "main": "..." , "browser": "..." , "dependencies" : {" chalk ":" ^ 1.1.3 "and" diff ":" ^ 3.2.0 ", "jest - the matcher - utils" : "^ 20.0.3", "pretty - format" : "^ 20.0.3"}}Copy the code

Wrappers like Lerna first run YARN Install for each package.json and then run Yarn Link separately for interdependent packages.

If we use this approach, we get a folder structure like this:

| jest/ | ---- node_modules/ | -------- chalk/ | ---- package.json | ---- packages/ | -------- jest-matcher-utils/ | ------------ node_modules/ | ---------------- chalk/ | ---------------- pretty-format/ | ------------ package.json | -------- jest-diff/ | ------------ node_modules/ | ---------------- chalk/ | ---------------- diff/ | ---------------- jest-matcher-utils/ (symlink) -> .. /jest-matcher-utils | ---------------- pretty-format/ | ------------ package.json ...Copy the code

As you can see, the third-party dependencies are redundant.

With a workspace enabled, Yarn can generate a more optimized dependency structure, and when you run a normal Yarn installation anywhere in your project, you get the following node_modules.

| jest/ | ---- node_modules/ | -------- chalk/ | -------- diff/ | -------- pretty-format/ | -------- jest-matcher-utils/  (symlink) -> .. /packages/jest-matcher-utils | ---- package.json | ---- packages/ | -------- jest-matcher-utils/ | ------------ node_modules/ | ---------------- chalk/ | ------------ package.json | -------- jest-diff/ | ------------ node_modules/ |  ---------------- chalk/ | ------------ package.json ...Copy the code

Packages such as diff, Pretty Format, and Symlink to Jest Matcher Utils are promoted to the node_modules directory of the root node, making installation faster and smaller. However, package Chalk could not be moved to the root directory because the root directory already depended on another incompatible chalk version.

Both structures are compatible, but the latter is more optimized and still the correct node.js module parsing logic.

For Lerna users, this is similar to the –hoist flag to boot code.

If you run the code in the Jest Diff workspace, it will be able to resolve all of its dependencies:

  • /node_modules/chalk
  • Require (‘ diff ‘) parse from.. /.. /node_modules/diff
  • Require (‘ pretty-format ‘) parse from.. /.. /node_modules/pretty-format
  • Require (‘ jest-matcher-utils’) parse from.. /.. /node_modules/jest-matcher-utils this is a symbolic link.. /packages/jest-matcher-utils

Manage workspace dependencies

To modify workspace dependencies, simply run the corresponding command in the Workspace folder:

$CD packages/jest-matcher-utils/ $yarn add left-pad ✨ Done in 1.77s. $git status modified: package.json modified:.. /.. /yarn.lockCopy the code

Note that workspaces do not have their own workspace yarn.lock file and the root directory yarn.lock contains all dependencies for all workspaces. When you want to change a dependency in the workspace, the root yarn.lock changes at the same time as the workspace package.json.

With Lerna integration

Will YARN workspaces make Lerna obsolete?

Not at all. The YARN workspace is easily integrated with Lerna.

Lerna offers more than just guiding a project, it also has a community of users around it who fine-tune Lerna to suit their needs.

Starting with lerna2.0.0, when you pass the flag — useworkSpaces when running Lerna, it will use Yarn to boot the project, and it will also use package.json/workspaces fields to find packages instead of lerna.json/ packages.

Here is how Lerna configures Jest:

{"lerna": "2.0.0", "npmClient": "yarn", "useWorkspaces": true}Copy the code

Jest relies on Yarn to boot the project and Lerna to run publish commands.

What’s next?

The Yarn workspace is the first step for the package manager to manage Monorepos, as Monorepos becomes a more common solution for code sharing.

At the same time, we don’t want to put all the possible Monorepo features in Yarn. We want to keep Yarn focused and lean, which means Yarn and projects like Lerna will continue to work together.

Our next goal is to finalize Yarn 1.0, which means taking stock of what we’ve done with Yarn over the past year and recognizing how reliable Yarn has become. We will also share our thoughts on what Yarn should do next.

Stay tuned.