What is a Monorepo

Monorepo is a code organization idea that requires us to use only one code repository to manage all the resources and subprojects of a large project. Simply put, code that used to reside in multiple REPOs can now reside in one REPO.

Multirepo (traditional multi-REPo project) splits a project into responsibilities, business modules, and then creates different code repositories to manage it. Different teams can focus on a single repository code submission, compilation, and release.

advantages

  • You can see all the code, as well as new commits for other projects.
  • Easy to work collaboratively, not delayed by dependent project releases, dependencies are available as soon as they are submitted.
  • Code reuse has improved. Internal references can be easily made to each other, once committed, and projects in the REPO that depend on this are instantly synchronized to the latest dependencies.
  • Common dependencies are installed once, all modules in Monorepo are shared, and there is no problem with inconsistent versions of dependencies for multiple projects.
  • Unify the tests and avoid the complexity of cross-REPO testing caused by project dependencies.
  • Unified CI, CD process, a set of configuration to solve the test and release of all modules.

disadvantages

  • When the code size becomes too large, it can cause performance issues such as code loading, version control, and so on, which can cause the code editor to fail to load or load slowly
  • Code size increases, lots of commits, branches, tags, and version control becomes difficult. For example, you want to commit logs to do revert, blame, etc.
  • Increased coupling due to arbitrary cross-referencing between projects, and the lack of clarity of responsibility and organization of code due to code being placed together may also result
  • Permissions issue, all code visible.

Who is using Monorepo

  • Babel, React, Angular, Ember, Meteor, and Jest are all managed using a single library.
  • Google has 86 terabytes of more than 2 billion lines of code, and Google started using Monorepo on a large scale in 2015.
  • Monorepo is also used by well-known foreign companies such as Facebook, Twitter and Airbnb.

How do I decide whether to Monorepo or not

  • Select Monorepo when the dependencies need to be unified and copied independently to different containers when microservices share the dependency library code.
  • When developing an automated test framework using the CLI tool library, a single automated test framework can test all projects.
  • In the multi-project development process, the responsibilities of the project are clearly divided, and there are different testing and release processes, so the use of Monorepo is not considered.

Difficulties in using Monorepo in Git

  • Git keeps track of the state of the entire commit tree at every commit, which can slow git down as projects multiply.
  • Git tags will become irrelevant, such as a release tag for an Android project in repo that has no real meaning for a Web project in repo.
  • Git uses directed acyclic graphs to record history. A surge in commits causes git log, Git blame to slow down.
  • Git branches can also proliferate due to project surges, which can also slow down Git Branch.
  • Large File Storage can affect the performance of the entire repO (this is addressed in Git LFS (Large File Storage)).
  • Git Clone, Git Fetch, Git push, Git Status, and so on are all slowed down by file count surges.

Use Lerna at the front end for Monorepo project management

Managing Monorepo through Git does create a lot of difficulties once the code becomes large, but it doesn’t stop the front-end world from wanting monorepo. Because of monorepo’s code management ideas, there are many tools for project management based on Monorepo, such as Rush, NX, Bit, Lerna.

Lerna is an open source software for javascript monorepo project management using Git + NPM. Lerna was created to solve the multi-package problem with Babel to optimize the workflow for managing multi-package repositories using Git and NPM.

A simple Lerna project file structure is as follows:

| - my - lerna - '| | ─ ─ package. The json | | ─ ─ lerna. Json | └ ─ ─ packages | | ─ ─ package - 1 | | └ ─ ─ package. The json | └ ─ ─ package - | 2 └ ─ ─ package. JsonCopy the code

The key idea

One Git repo, multiple JS projects

All JS projects that were once in different REPOs are now in one REPO, as shown in the folders above, under packages. These REPOS can reference each other directly through the capabilities provided by Lerna. A code commit operation is no different from a commit in a REPO, in that a commit may contain changes to multiple subprojects.

Shared node_modules

The capability provided by Lerna (dependency enhancement) makes node_modules dependencies installed only once, and all subproject dependencies are installed into the project root directory.

Unified version release management

Lerna supports automatic version increment, automatic release, changelog modification, and release tag. A publish command automatically publishes all subitems that make changes under the REPO. The Lerna publishing mode also allows you to decide whether to automatically publish all current updates or to select the subprojects that make changes to be published.

Unified trigger command

Running commands with Lerna triggers commands of the same name configured in all subprojects by configuring the script command in the root package.json directory. Suppose project A has A start command to start A server and project B has A start command to start A web page in the subproject. Running the start command with Lerna will automatically run the instructions for projects A and B. The same applies to other commands such as test and publish.

Several commonly used Lerna commands

Here is just a brief description of the main functions, details can be found on Lerna’s official website.

  • lerna init

Create a new Lerna repO. Generate a directory containing lerna.json and Pacakge. json, along with an empty Packages folder.

  • lerna clean

The node_modules folder is generated after the subproject runs the NPM install XXX dependencies. In lerna repo, you only need the dependency declaration in the package.json of the subproject. Lerna provides the ability to share node_modules, so node_modules are not required in subprojects. The following lerna bootstrap command can be used to install dependencies and implement dependency sharing in subprojects. Delete the node_modules folder in all subprojects at once with the lerna clean directive (this will be demonstrated in the practical demonstration below)

  • lerna bootstrap

This command installs dependencies declared in the package.json subprojects and links the dependencies of the subprojects through soft links (e.g., project A is dependent on project B, After lerna bootstrap, A soft chain /node_modules/A will be created at the repo root to point to the A project folder, as if the A project was installed. After this operation, all dependencies installed by project A can also be referenced directly by project B. This command can specify the use of NPM or YARN as the package management tool.

  • lerna version

This command will identify local git commits for existing subprojects since the last release (managing the commit information for a release to be released) and increment the NPM package.json version number for these code updates. Modify changelog, create git Release tag, create commit, and push to git server.

  • lerna publish

Running this command releases all subprojects that git has not yet released commit to NPM (this command also performs lerna version action internally). This command triggers NPM to publish the associated lifecycle functions, which can be declared in lerna.json to execute the required procedure.

  • lerna run

Similar to NPM run, lerna run will execute commands of the same name defined in package.json in all subprojects. For example, if all subprojects define the test command, lerna run test will execute tests for all subprojects.

Practice demonstration

This demonstrates the use of Lerna to create and manage a Monorepo project, which demonstrates a traditional front – and back-end separated project containing a front-end Web app, a back-end service, and a business common module.

Create lerna monorepo

Create a new project directory, my-lerna-project, and execute lerna init in that directory

mkdir my-lerna-project
cd my-lerna-project
lerna init --yes
Copy the code

After lerna init is executed, the following structure is generated under my-Lerna-project

├─ My-First-class Exercises ── ├─ Package. json ├─Copy the code

Open lerna.json with the following content:

{
  "packages": [
    "packages/*"]."version": "0.0.0"
}
Copy the code

The packages field represents all subpackages managed by LERNA. Packages /* means all projects in the Packages directory (projects not included in the declaration list are not affected and configured commands are not executed when lerna directives are run).

The package.json file is no different from NPM package.json and is used to declare dependencies for the entire REPO.

Create a subproject

Start by creating a React project in the Packages directory as our front page. After creating the project through the create-react-app scaffolding, commands like start and test have been configured.

cd ./packages
npm init react-app my-web-app

npm start
Copy the code

Verify that running NPM run start can run the page.

Then create a Node Expree project under the Package directory as a back-end service. Once the project is created through the create-Express-app scaffolding, commands like start and test are automatically configured.

cd ./packages
npm init express-app my-backend
Copy the code

Verify that running NPM run start can run the page. The scaffolding example provides the default corresponding GET request to return {hello:’world’}.

Ascension node_modules

At this point, our REPO has two separate subprojects.Now let’s use lerna clean to clean the node_modules installation in the subproject.

Lerna Clean Lerna Notice CLI V4.0.0 LERna Info Removing the following directories: lerna info clean packages\my-backend\node_modules lerna info clean packages\my-web-app\node_modules ? Proceed? Yes lerna info clean removing \my-first-lerna\packages\my-backend\node_modules lerna info clean removing \my-first-lerna\packages\my-web-app\node_modules lerna success clean finishedCopy the code

Lerna bootrap –hoist installs dependencies to the root to achieve shared node_modules for subprojects.

Lerna Bootstrap --hoist LERna notice CLI V4.0.0 LERna info Bootstrapping 2 Packages LERna info Installing External dependencies lerna info hoist Installing hoisted dependencies into root lerna info hoist Pruning hoisted dependencies lerna info hoist Finished pruning hoisted dependencies lerna info hoist Finished bootstrapping root lerna info Symlinking packages and binaries lerna success Bootstrapped 2 packagesCopy the code

After dependency promotion, the REPO directory structure is as follows: if the dependency package versions of the subprojects are different, packages of the same version are promoted to the outermost layer. Node_modules under subprojects retain their own dependent versions. Not all dependencies are promoted to the outer layer. For example, the executable provided in devDependencies remains in the node_modules of the child project to ensure that the executable runs correctly in the child project.

You can see that my-backend and my-web-app still have node_modules folders, but they actually only contain executables. React and Express dependencies refer to outer node_modules.

  1. Configure tests and run commands

The scaffolded project is already configured with test and run commands by default. Let’s change the test command in the subproject to the following to facilitate verification:

// my-web-app/package.json. ."scripts": {
    "start": "react-scripts start"."build": "react-scripts build"."test": "echo tests running in my-web-app"."eject": "react-scripts eject"},...// my-backend/package.json. ."scripts": {
    "start": "node ./bin/app"."test": "echo test running in my-backend-app"},...Copy the code

Now modify the package.json in the root directory and configure the lerna directive

. ."scripts": {
    "start": "lerna run start"."test": "lerna run test"},... .Copy the code

Now execute NPM run test and see that the tests for both projects are executed as follows:

 npm test 

> test
> lerna run testLerna notice CLI V4.0.0 LERNA INFO Executing Command in 2 packages: "NPM run test" lerna info run Ran NPM script 'test' in 'my-backend' in 1.5s:
> [email protected] test
> echo test running in my-backend-appTest running in my-backend-app lerna info run Ran NPM script 'test' in 'my-web-app' in 1.5s:
> [email protected] test
> echo tests running in my-web-appTests running in my-web-app lerna success run NPM script 'test' in 2 packages in 1.5s: lerna success - my-backend lerna success - my-web-appCopy the code

The same goes for NPM run Start, which starts both the front-end and back-end services. In this way, when the sub-project joint adjustment can be done by an instruction.

  1. Submit a version

We first need to commit the code changes to Git, and then use Lerna for unified release version management.

git add .
git commit -m "first commit"

lerna version

Copy the code

After running lerna version, the command line will prompt you to select the new version number (add a version number for all the changes). After selecting the corresponding version number, the lerna version command will automatically create a tag corresponding to the version number and push it to the remote git server.

On the Git remote Web management page, you can see that the version number increases in packakge.json after lerna version is run.

The lerna version command will intelligently check the items that are currently changed and uniformly manage the version numbers for those items that have changed. For example, in A (1.0.1) and B (1.0.2), only B has the code fixed and updated, so after running lerna version, only B’s version number will increase to 1.0.3, and A will not change. If both A and B have code changes at the same time, then after running lerna version, both A and B will be incremented to the same version 1.0.3 (again different in Lerna Independent mode).

To post a new version, lerna version — Converntional commits (Conventional Commits definition). This command automatically modifies Changelog.md to reflect the change. If conventional commits were specified, Changelog. md automatically generated commits. For example, we made the following commit:

 git commit -m "feat: add new funciton 1"
 git commit -m "fix: fix funciton 2 bug" 
 git commit -m "perf: optimize funciton 3"
Copy the code

Then Changelog. md will automatically record the following:

  1. Release NPM package

The above shows how to create and submit monorepo projects through Lerna. If we are developing NPM packages like Babel, then we need to learn how to use Lerna for package distribution. For details on how to publish NPM packages, see this article on how to publish an NPM package to the NPM official Registry.

If we want to publish to NPM Registry after committing the code, we can run Lerna publish directly after git commits the code, Lerna publish automatically runs lerna version internally to increase the version number, tag, and push the remote Git server. In addition, Lerna Publish automatically runs NPM publish internally, pushing new releases to the NPM Registry (this works only if we have configured NPM and Git authentication information).

conclusion

Just a little bit about what Monorepo is, what Lerna is, and how lerNA is basically used. If you want to use Monorepo on the front end, lerna is pretty handy. Understand the basic operation, is to use the time to encounter problems to scratch the official documents.

Reference:

www.perforce.com/blog/vcs/wh… www.atlassian.com/git/tutoria… medium.com/@mattklein1… About.gitlab.com/direction/m… Blog. Bitsrc. IO / 11 – tools – to…