0. πŸ§‰ preface

In the recent project development, there appeared a situation that troubled me. The project A I am developing depends on the project B that has been released online. However, with the continuous development of project A, it is necessary to modify the code of project B from time to time (these modifications do not need to be released online for the time being). How can I synchronize the changes in project A after modifying the code of project B? After the release of project A, how to solve the version synchronization problem after the upgrade of project A and B in an elegant way? After some research, I found that the best solution to these problems is the Monorepo strategy I will introduce in this article.

1. πŸ€” What is the Monorepo strategy?

Monorepo is a software development strategy for storing multiple project code in a repository (“mono” comes from the Greek ΞΌΟŒΞ½ΞΏΟ‚ meaning single, while “repo”, obviously, is an abbreviation for repository). Putting the code of different projects in the same code repository may seem strange at first, but in fact, this kind of code management method has many benefits, whether it is the world-class Internet enterprises Google, Facebook, Both Babel (pictured below), a well-known open source project team, use the Monorepo strategy to manage their code.

Babel uses the Monorepo policy to manage code

What are the benefits of using the Monorepo strategy for code managers and application developers? ** How can we try to implement the Monorepo strategy at work? ** This is exactly what this article wants to discuss. I hope that through my introduction, you can have a more complete understanding of the Monorepo strategy. The tools and ideas introduced in this article can actually help you and your team.

2. Advantages and disadvantages of πŸŒ— Monorepo strategy

By organizing your code with the MonorePO policy, the directory structure of your code repository would look like this:

.β”œ ─ β”œβ”€ package. β”œβ”€ β”œβ”€ β”œ.json β”œβ”€ β”œ.txtThis is where all the child repo directories will be storedβ”œ ─ ─ project_1 / β”‚ β”œ ─ ─ index. The js β”‚ β”œ ─ ─ node_modules / β”‚ β”” ─ ─ package. The json β”œ ─ ─ project_2 / β”‚ β”œ ─ ─ index. The js β”‚ β”œ ─ ─ node_module / β”‚ β”” ─ ─ package. Json...Copy the code

At first glance, the so-called Monorepo strategy is just a collection of different project directories under one directory, but in practice there is more to consider than meets the eye. By analyzing the advantages and disadvantages of using Monorepo strategy, we can more intuitively feel the hidden knowledge points involved in this.

2.1 Advantages of MonorePO scheme

  1. Code reuse is easy: since all project code is centralized in a code repository, it’s easy to isolate common business components or tools across projects and reference them in-code via TypeScript, Lerna, or other tools;
  2. Dependency management becomes very simple: similarly, because the reference paths between projects are internalized in the same repository, it is easy to track which projects are affected when code changes are made in one project. With some tools, it is easy to manage version dependency and upgrade version numbers automatically;
  3. Code refactoring will become very convenient: think about what’s to stop you to refactor the code, a lot of times, the reason from the “uncertainty”, you are not sure whether modify for other projects of a project is a “fatal”, out of fear of the unknown, you will tend to refactor the code, this will cause the entire project code decay degrees will be growing at an alarming rate. Under the guidance of monorepo, you can clearly know the scope of your code, and you can conduct uniform testing on affected projects, which encourages you to constantly optimize your code;
  4. It promotes an open, transparent, and shared organizational culture that is conducive to the growth of developers and the improvement of code quality: Under monorepo strategy, each developer is encouraged to view, modify the code of others (if necessary), at the same time, will also arouse developers to maintain code, and the writing of unit tests of responsibility (before visiting a friend, after all, we never mind your house how messy), it will form a benign technology atmosphere, Thus ensuring code quality throughout the organization.

2.2 Disadvantages of Monorepo scheme

  1. Project-level permission management becomes very complex: neither Git nor other VCS systems have A satisfactory solution for project-level permission management in support of monorepo policies, which means that it is difficult for A project in A to avoid being seen by developers in B. (Fortunately, we can practice the Monorepo strategy at the “project level,” which is the subject of this article and will be clarified again later);
  2. Higher learning costs for new employees: Unlike the one-project-one-codebase model, where new employees only need to be familiar with the code logic of a specific codebase, under monorepo, new employees may have to spend more effort to clarify the logic of each codebase. Of course, this cost can be covered by new employee documentation. But keeping the documents fresh requires extra manpower;
  3. A corporate-level Monorepo strategy requires a dedicated VFS system, supported by automated refactoring tools: imagine how an enterprise like Google could store a billion lines of code in a repository? How long do developers have to wait each time they pull code? How to implement authority management and agile release between each project code? Any simple strategy multiplied by enough scale can work wonders (for better or worse), and for small and medium enterprises, without the human resources of Google and Facebook, the hope of having all the project code in the same repository is just a castle in the air.

2.3 Summary: How to choose?

True, there has never been a “silver bullet” in software development. The Monorepo strategy is not perfect, and I have found in practice that it takes more than excellent programming skills and patience to apply the Monorepo strategy to an organization. It’s the collision of team agendas, organizational culture, and individual influence that determines whether or not the idea gets implemented.

But don’t get discouraged too soon, because while it may be difficult to get organizations to change and adopt a unified Monorepo strategy, that doesn’t mean we need to say goodbye to the Monorepo strategy (or I’ll stop here). We can also put monorepo strategy practice at this level “project”, that is, from logic to determine the correlation between projects and project, then the associated project integration under the same warehouse, normally, we won’t have too many interrelated projects, that means we can get all the benefits of monorepo strategy, free And can refuse to pay interest on large Monorepo architectures.

The rest of this article is a summary of the “project-level MonorePO practice,” and even if you don’t ultimately choose the MonorePO strategy to organize your code, some of the engineering tools or ideas provided in this article will still help you.

3. πŸ§‘πŸ»πŸ’» Monorepo scheme practice

3.1 Lock Environment: Volta

Volta is a JavaScript tool manager that allows you to easily lock down versions of Node, NPM, and YARN in your project. After installing Volta, run the Volta pin command in the root directory of your project, and Volta will automatically switch to whatever version of Node or NPM (YARN) you are currently using.

So, in addition to using Docker and showing versions of Node and NPM (YARN) declared in documentation, you have another powerful tool for locking your environment.

Volta also has an attractive feature compared to NVM: When your project’s CLI tool is inconsistent with the global CLI tool, Volta can automatically recognize it in the project root directory and switch to the project specified version, all by Volta without the developer having to worry about anything.

3.2 Reuse packages: workspace

After using monorepo strategy, the two biggest benefits are:

  1. Avoid reinstalling packages, thus reducing the footprint of disk space and reducing build time;
  2. Internal code can refer to each other;

Both of these benefits can be accomplished by a full-fledged package management tool, which for front-end development is YARN (1.0 +) or NPM (7.0 +) through a feature called workspaces (⚠️ note, The NPM that supports workspaces is still not the LTS version.

To achieve the two benefits mentioned earlier, you need to do three things in your code:

  1. Adjust the directory structure to place related items in the same directory, and the recommended name ispackages;
  2. In the project root directorypackage.jsonFile, SettingsworkspacesProperty, the value of which is the directory created previously;
  3. Similarly, inpackage.jsonFile, SettingsprivateProperties fortrue(in order to avoid our misoperation to release the warehouse);

After modification, your project directory should look like this:

. β”œ ─ ─ package. Json β”” ─ ─ packages / β”œ ─ ─ @ mono/project_1 /# Use '@< project name >/< subproject name >'β”‚ β”œ ─ ─ index. Js β”‚ β”” ─ ─ package. The json β”” ─ ─ @ mono/project_2 β”œ ─ ─ index. The js β”” ─ ─ package. The jsonCopy the code

When you execute NPM install or YARN install in the project root directory, you will find the node_modules directory in the project root directory, and this directory not only has the NPM package shared by all the sub-projects, but also contains our sub-projects. Therefore, we can introduce code from other subprojects in the same way as a regular NPM module through various module import mechanisms.

Note that the name of the subproject starts with @

/. This is a community best practice that not only makes it easier for users to understand the architecture of the entire application, but also makes it easier for you to find the subproject within the project.

At this point, we’ve completed the core of the Monorepo strategy. It’s pretty easy, isn’t it? But as the old saying goes, we still have some way to go before we can gracefully build a Monorepo project.

3.3 Unified Configuration: Merge similar items – Eslint, Typescript, and Babel

You’ll agree that code should follow the DRY principle (short for Don’t Repeat Yourself). Then, of course, we should try to avoid placing duplicate eslintrc, tsConfig, and other configuration files in multiple subprojects. Fortunately, Babel, Eslint, and Typescript all provide features that allow us to reduce self-repetition.

3.3.1 TypeScript

We can place the tsconfig.settting.json file in the Packages directory and define the generic TS configuration in the file. Then, in each subproject, we can introduce the generic configuration through the extends property. And set up the compilerOptions.com the email.composite value is true, ideally, the component of tsconfig file should only include the following contents:

{
  "extends": ".. /tsconfig.setting.json".// Inherits the generic configuration under packages
  "compilerOptions": {
    "composite": true.// Helps TypeScript quickly determine the location of the output file referencing the project
    "outDir": "dist"."rootDir": "src"
  },
  "include": ["src"]}Copy the code

3.3.2 rainfall distribution on 10-12 Eslint

We can do the same for the Eslint configuration file by defining the.eslintrc file contents for the subproject:

{
  "extends": ".. /.. /.eslintrc".// Notice the difference here
  "parserOptions": {
    "project": "tsconfig.json"}}Copy the code

Notice that for the generic ESLint configuration, we don’t place it in the Packages directory, but in the root of the entire project, because some editor plug-ins will only look for.eslintrc files in the project root directory. So for our projects to maintain good “development environment consistency,” it is important to place the common configuration files in the root directory of the project.

3.3.3 Babel

Babel configuration files are merged in the same way as TypeScript, or even simpler, by declaring it in a.babelrc file in a subproject:

{
  "extends": ".. /.babelrc"
}
Copy the code

When everything is ready, our project catalog should look something like this:

. β”œ ─ ─ package. Json β”œ ─ ─ the eslintrc β”” ─ ─ packages / β”‚ β”œ ─ ─ tsconfig. Settings. The json β”‚ β”œ ─ ─ the babelrc β”œ ─ ─ @ mono/project_1 β”‚ β”œ ─ ─ Index. Js β”‚ β”œ ─ ─. Eslintrc β”‚ β”œ ─ ─ the babelrc β”‚ β”œ ─ ─ tsconfig. Json β”‚ β”” ─ ─ package. The json β”” ─ ─ ─ @ mono/project_2 β”œ ─ ─ index. The js β”œ ─ ─ .eslintrc β”œβ”€.β”œ ─ β”œ.json β”œβ”€.jsonCopy the code

3.4 Unified command script: scripty

In the previous step, we abstracted all configuration files as much as possible, simplifying the code and improving consistency throughout the project. As a result, our entire warehouse has “more intense monorepo flavor β˜•οΈ”. But if you look closely at our entire project file, there is one glaring flaw and some annoying bad taste, and when you look closely at your numerous package.json files, you’ll know what I’m talking about — scripts.

If you have enough subprojects, you might find that the scripts properties in each package.json file are pretty much the same, and some scripts are riddled with various Linux syntax, such as pipe operators, redirection, or directory generation. We need to solve the problems of inefficiency caused by repetition and incomprehensibility caused by complexity.

The solution presented here is to use Scripty to manage your script commands. In a nutshell, scripty allows you to define script commands in files and refer to them directly by filename in package.json files. This allows us to:

  1. Reuse script commands between subprojects;
  2. Scripting commands as code, no matter how complex, and calling them as functions;

By using Scripty to manage our Monorepo application, the directory structure would look like this:

.β”œ ── Package. json β”œβ”€.Eslintrc β”œβ”€ scirpts/This is where all the scripts are storedβ”‚ β”‚ β”œ ─ ─ packages /Package level scriptingβ”‚ β”‚ β”‚ β”œ ─ ─ build. Sh β”‚ β”‚ β”‚ β”” ─ ─ test. Sh β”‚ β”” ─ ─ ─ β”” ─ ─ workspaces /# global scriptβ”‚ β”œ ─ ─ build. Sh β”‚ β”” ─ ─ test. Sh β”” ─ ─ packages / β”‚ β”œ ─ ─ tsconfig. Settings. The json β”‚ β”œ ─ ─ the babelrc β”œ ─ ─ @ mono/project_1 β”‚ β”œ ─ ─ Index. Js β”‚ β”œ ─ ─. Eslintrc β”‚ β”œ ─ ─ the babelrc β”‚ β”œ ─ ─ tsconfig. Json β”‚ β”” ─ ─ package. The json β”” ─ ─ @ mono/project_2 β”œ ─ ─ index. The js β”œ ─ ─ .eslintrc β”œβ”€.β”œ ─ β”œ.json β”œβ”€.jsonCopy the code

Note that our scripts are divided into two classes, “package level” and “workspace level”, and are placed in two folders. The advantage of this is that we can execute global scripts in the project root directory as well as specific scripts for individual projects.

By using scripty, the scripts properties in the package.json file of a subproject become very compact:

{..."scripts": {
    "test": "scripty"."lint": "scripty"."build": "scripty"
  },
  "scripty": {
    "path": ".. /.. /scripts/packages" // Note that we specify the path to scripty here},... }Copy the code

And you’re done! πŸŽ‰ So far, we have done our best to remove duplicate code from the entire project, making it clean, clean and highly reusable.

πŸ§‰ Tips:

Don’t forget to use the chmod -r u+x scripts command to make all shell scripts executable, and don’t forget to include this tip in your readme.md file!

3.5 Unified Package Management: Lerna

Image source: https://github.com/lerna/lerna

I sometimes wonder at my lack of inspiration for a fabulous, self-explanatory name like Lerna. You can imagine a scenario where each dragon of the nine heads is managing a sub-project for you, and you just need to ride on the dragon to give orders. This is basically the intuitive feeling when we use Lerna.

This is why we almost have to mention Lerna when we talk about the Monorepo strategy, which really provides a very convenient way to manage monorepo projects. The more subprojects there are, the more power Lerna can exert.

When multiple subprojects are placed in a code repository and depend on each other, we face two thorny problems:

  1. If we need to execute the same command in multiple subdirectories, we need to manually enter each directory and execute the command;

  2. When a subproject is updated, we can only manually track and upgrade the versions of other subprojects that depend on the project.

With Lerna, these thorny problems are eliminated.

Json file will be added to the root directory of the project after initialization with NPX lerna init. The default content is:

{
  "packages": ["packages/*"]."version": "0.0.0"
}
Copy the code

Let’s change this file slightly so that it becomes:

{
  "packages": ["packages/*"]."npmClient": "yarn"."version": "independent"."useWorkspaces": true,}Copy the code

Notice that we explicitly declared our package client (npmClient) as YARN and let Lerna track the directory we set up for workspaces, so we still have all the features of previous Workspaces (subproject references and generic package promotion).

Another interesting change is that we specify the version attribute as a keyword independent, which tells Lerna that the version numbers of each subproject should be considered independent of each other. When lerna publish is run after a subproject code is updated, LERNA will monitor the subproject whose code has changed and let the developer decide the version number that needs to be upgraded in an interactive CLI mode. The version number of the associated subproject will not be automatically upgraded. On the contrary, when we fill in a fixed version number, the code of any subproject will change. Causes the version numbers of all subprojects to be upgraded based on the currently specified version number.

Lerna provides a number of CLI commands to meet our various needs, but following the 2/8 rule, you should focus on the following commands first:

  • lerna bootstrap: equivalent tolerna link + yarn installFor creating compliant links and installing dependencies;
  • lerna runNPM script is executed in all subprojects like a for loop, and it is smart enough to recognize dependencies and execute commands from the root dependency.
  • lerna exec: likelerna runSimilarly, commands are executed in dependent order, except that it can execute any command such as shell scripts.
  • lerna publish: Release packages with code changes, so first you need to use them before using Lernagit commitCommand to submit code so that Lerna has a baseline;
  • lerna add: Add local or remote packages as dependencies to the current Monorepo repository. This command is important because it allows Lerna to identify and track dependencies between packages.
#Add @mono/project1 to @mono/project2 and @mono/project3Lerna add @ mono/project1 '@ mono/project {2, 3}'Copy the code

3.5.1 Lerna Advanced Commands

In addition to the common commands described above, Lerna also provides some parameters for our more flexible needs, such as:

  • --concurrency <number>: Parameters can enable Lerna to utilize multiple cores on the computer and run concurrently, thus improving construction speed;
  • --scope '@mono/{pkg1,pkg2}':--scopeThe parameter specifies the environment in which Lerna commands are run. By using this parameter, Lerna will no longer be able to execute commands in all warehouses, but will execute commands precisely in the warehouses we have specified. It also supports the template syntax in the example.
  • --stream: This parameter enables us to view the command execution information of Lerna runtime.

3.5.2 Local distribution of NPM package: Verdaccio

Seeing this, you may want to experience for yourself what it’s like to manage/publish monorepo projects using Lerna. It is somewhat frustrating to quickly discover that it is not a good idea to publish the sample code to a real world NPM repository, but don’t worry, you can use Verdaccio to create a local NPM repository as a proxy and experience all the power of Lerna.

Installing and running Verdaccio is very simple, you just run:

npm install --global verdaccio
Copy the code

Install the Verdaccio application globally and type in the shell:

verdaccio
Copy the code

To access your local proxy NPM repository, use localhost:4837. Create a.npmrc file in your project root directory and rewrite the NPM repository address to your local proxy address:

registry="http://localhost:4873/"
Copy the code

Done πŸ™Œ! Whenever you perform lerna publish, the package built by the subproject will be published in the local NPM repository, and when you perform Lerna Bootstrap, Verdaccio will release you to successfully pull the corresponding code from the remote NPM repository.

3.6 Formatting Commit Information

Now that we’ve mastered all the cutting edge techniques for organizing a project-level MonorePO repository, let’s look at one final area that can be optimized: binding commit information when code commits.

A MonorePO repository can be submitted by different developers for different subprojects, and without the normalized COMMIT information, it can be a disaster during troubleshooting or version rollback without accident. Therefore, don’t underestimate the importance of formatting commit messages (as well as code comments!). .

To keep track of every code change at a glance, we use the CommitLint tool as the commitlint tool of choice for formatting commit messages.

Commitlint as the name suggests, commitLint helps us check commit information by forcing us to specify a type at the beginning of a commit to indicate the general intent of the commit. Supported type keywords include:

  • feat: adds a new feature.
  • chore: indicates that you did some “housework” unrelated to features and fixes.
  • fix: indicates that a Bug is fixed.
  • refactor: indicates that the commit is due to code refactoring.
  • style: indicates code beautification or formatting;
  • .

I strongly recommend that you follow this specification when writing your Commit messages. Don’t be lazy. Stick to it, and your Git logs will look neat, organized, and expressive, and you’ll receive compliments from your peers who will be proud to work with an elegant engineer like you.

In addition to limiting the commit information type, CommitLint also supports (though not required) a display that specifies the name of the subproject to which we are committing this time. If we have a subproject named @mono/project1, we can write the commit information for that project as follows:

Git commit -m "feat(project1): add a babe buttonCopy the code

No doubt, this will make our COMMIT messages more expressive.

We can install commitlint and peripheral dependencies with the following command:

npm i -D @commitlint/cli @commitlint/config-conventional @commitlint/config-lerna-scopes commitlint husky lerna-changelog
Copy the code

Notice that? I secretly installed Husky, which helps us automatically run commitLint for checking when committing, but before doing so, we need to add something to the package.json file in the root directory, like this:

{..."husky": {
    "hooks": {
      "commit-msg": "commitlint -E HUSKY_GIT_PARAMS"}}... }Copy the code

To make commitlint aware of our subproject name, add the file commitlint.config.js to the project root directory and set the file content to:

module.exports = {
  extends: [
    "@commitlint/config-conventional"."@commitlint/config-lerna-scopes",]};Copy the code

At this point, we unified and standardized the COMMIT information of monorePO project, and finally the last piece of the jigsaw puzzle of the whole MonorePO engineering was put together!

(by the way, you can use the command line to perform echo “build (project1) : change something” | NPX commitlint command to verify whether your commit information through commitlint check.)

4. How do 🚚 migrate from Multirepo to use the Monorepo policy?

Now that we’ve learned the best practices for organizing your project code using the Monorepo strategy, you may already be itching to try out the techniques mentioned above. Build a Monorepo project from 0, no problem! But what about taking an existing project and turning it into a project using the Monorepo strategy?

Remember? At the end of a hundred miles, you still have some holes to tread. But you can still get my help here, don’t mention it!

As you may have noticed, Lerna provides us with the Lerna import command, which imports our existing packages into the Monorepo repository and also keeps all the commit information for that repository. In practice, however, this command only supports importing local projects, and does not support importing branches and labels of projects πŸ™ƒ.

So what if we want to import a remote repository, or get a branch or label? The answer is to use Tomono, whose content is a shell script.

Using Tomono to import the remote repository, there are only two things you need to do:

  1. Create a text file containing all the REPO addresses that need to be imported;
  2. Run the shell command:cat repos.txt | ~/tomono/tomono.shHere we assume that your text file is namedrepos.txtAnd you download Tomono in the user root directory;

The following is an example of the repo file content:

Git repository address 2. Sub-project name 3. Migrated path [email protected]/backend.git @mono/ Backend Packages /backend [email protected]/frontend.git @mono/frontend packages/frontend [email protected]/mobile.git @mono/mobile packages/mobileCopy the code

At this point, we also know how to migrate existing projects to monorepo projects. By this time, you are no longer a layman in the monorepo world!

Congratulations!! πŸŽ‰

5. πŸŽ“ subtotal

In this article, we learned about “what is Monorepo strategy” and “pros and cons of Monorepo strategy”, and learned and practiced some best practices of Monorepo strategy together. You must also be aware that even if your work scenario is not currently able to implement the Monorepo strategy, the methods, tools, and ideas you learn from reading this article can be applied to your current work.

Of course, the methods and ideas presented in this article will always be out of date, and the community will never stop looking for ways to better practice the Monorepo strategy, so you may come up with a better idea to fill a gap in an area after a while. Hopefully, you’ll have an article ready to contribute to the JavaScript community. Don’t forget to leave a comment in the comments section and let me share your achievements.

On monorepo, I will take you to explore here for the time being, see you later πŸ™‚

6. πŸ“ References

  1. πŸ“Ή JavaScript and TypeScript Monorepos
  2. πŸ“„ Why you should use a single repository for all your company’s projects
  3. πŸ“„ Advantages of monorepos
  4. πŸ“„ Lerna best practices for managing front-end Packages
  5. πŸ“„ Monorepo workflow based on Lerna and Yarn Workspace
  6. πŸ“„ Monorepos in the Wild
  7. πŸ“„ Monorepos: both Please don ‘t!
  8. πŸ“„ Monorepo: both please do!
  9. πŸ“„ the Introduction to Lerna
  10. πŸ“„ Monorepo migration practices

7. πŸ‘€ Read more

  1. Introduction to practice monorepo ecology: awesome-Monorepo
  2. Why Google Stores Billions of Lines of Code in a Single Repository
  3. Monorepo had Disadvantages such as Advantages and Disadvantages of a Monolithic Repository. Monorepo was a storage Repository

8. Recruitment information at πŸ™Œ

Alibaba Tao department user growth team is eagerly looking for like-minded partners, if you are ready to meet the moderate challenge, let more people like mobile shopping, but also let yourself grow quickly, welcome to send your resume to my email: [email protected], I look forward to receiving your message.

  • Cover Photo by Tetiana SHYSHKINA on Unsplash
  • This article only support paid reprint, reprint please contact the author to negotiate fees