Today, many tools can run “NPM Install” and “NPM Run Build” in 20 different folders. However, not all tools promote the correct monorepo.
Facilitating a proper singleton development means addressing challenges such as running test and build processes for separate modules, being able to release modules independently from the project, and managing the partial impact of changes on each affected dependent module in the project.
The list of challenges goes on, including even the “trivial” things like how you manage issues and PRs, which can become difficult depending on the size of your development.
Please note that a Monorepo is not a whole application (!). It is not built or deployed once, it is a set of applications developed separately.
What is monorepo?
Uvu opened the completed source code of VUe3.0 on October 5 during the National Day, and also adopted the management mode of MonorePO. It seems that MonorePO does have its unique advantages.
Monorepo is a code management pattern that puts multiple packages in a single REPO, rather than the traditional pattern of multiple packages and multiple repOs.
Babel, React, Angular, Ember, Meteor, Jest, and many other open source projects use this mode to manage code.
Problem solved
- Multiple REPOs are difficult to manage, and the editor needs to open multiple projects;
- When a module is upgraded, other modules that depend on the upgraded module need to be upgraded manually, which is easy to be neglected.
- Common NPM packages are installed repeatedly and occupy a large amount of disk capacity, such as the packaging tool WebPack, which is installed once per project;
- Friendly to newcomers, one command can complete the dependent installation of all modules, and the whole project module does not need to find each warehouse;
Problems brought about
- All package codes are concentrated in one project, and the volume of a single project is large.
- All package codes are visible to all, and permission management is not possible;
Those of you who are not aware of Monorpo can read the following article:
- Close reading of The Advantages of Monorepo
- The realization of MONorePO management mode in Vue3.0
- Lerna package management
In this roundup, I’ve collected some of the best tools in the world for building a “Monorepo” where you can build multiple modules in a single project and have a nice developer experience that can be extended.
This list is not ranked and is intended to outline the strengths of each tool in terms of its strengths. Hopefully it will help you save time and find the right tool.
Feel free to share your thoughts in the comments below.
1. Yarn Workspaces
The goal of Yarn Workspaces is to simplify working with Monorepos to address one of the primary use cases of Yarn Link in a more explicit way. Your dependencies can be linked together, which means your workspace can depend on each other while always using the latest code. This is also a better mechanism than YARN Link because it only affects your workspace tree and not your entire system.
Workspaces help solve some of the problems that make it a good piece of equipment for individual soldiers.
- It sets up a single
node_modulesThere is no need to duplicate or clone dependencies across different packages in a project. - All your project dependencies will be installed together, giving Yarn more room to optimize them better.
- Yarn will use a single lock file instead of a different lock file for each project, which means fewer conflicts and easier reviews.
- It allows you to make changes to the code in one of your packages and let other packages that use it see the changes immediately. Any changes made to the source code of one package are immediately applied to the other packages.
As a result, Yarn Workspaces is a very powerful combination that can be used with almost any tool in the list, especially tools like Bit, Nx, and Lerna, as the underlayer of your MonorePO management abstraction.
However, you can also publish directly with WorkSpaces. When a workspace is packaged into an archive, it dynamically replaces any Workspace: dependencies with a version of a package, so you can publish the resulting package to the remote registry without running an intermediate step — consumers will be able to use the published workspace just like any other package. Cool!
Reference: Monorepo workflow based on Lerna and YARN Workspace
2. Bit
Bit is the next generation tool for building modular projects. This is a new and exciting single-repository approach, in which modules managed by the same project (the same Bit workspace) are actually distributed in different scopes, regardless of the repository.
Bit lets you split up your module development in a completely decoupled way and enjoy a simple, holistic development experience to coordinate everything.
With bit, you can decouple components in your project so that each component is independently developed, built, tested, and released. Each component is developed and built using special environments that are extensible and reusable so you can quickly customize and reuse them.
The Bit’s workspace manages the relationships between all the components in the project. When you make changes to any component, Bit builds and tests it separately and propagates the changes to the dependency diagram.
Components can be distributed in bulk to the NPM and/or bit.dev platforms as standalone packages for collaboration, consumption, and documentation.
Bit’s UI helps you see how your MonorePO is developing. As you write code, every component is recorded, tested, built, etc., and you can visually see what’s happening with real-time feedback and hot overloading.
Bit provides a decoupled development environment — reusable and customizable modules that bind together the different service configurations and “bundles” required by individual components throughout their life cycle, such as compilation, bundling, testing, running-in, documentation, and so on.
Master component diagrams – Bit defines, manages, and helps you leverage the relationships between all the components in your project.
Graphically-driven builds – When you make changes to a component, Bit automatically detects other components that depend on it and “knows” to build only affected graphics that depend on that component.
“Graphically-driven build” also means that in the event a component is marked for a new release (before being exported to Bit’s cloud), Bit not only runs the build on each affected component, but also ensures that they are marked for a new release.
Isolated testing and builds – Each component is built and tested isolated from the project, so you can see exactly the impact of the changes.
Component Build pipeline — You can build jobs in a reusable pipeline that can be applied to all components in a project or all projects.
Bulk distribution – Each component developed in Bit Monorepo can be distributed as a separate package. Bit removes all the overhead of configuring “package.json” and other Settings files for each component. All you have to do is run the ‘bit Tag ‘, and the bit will automatically patch all the components that have changed (support semver rules) and release the changes in bulk.
Reusable Document Templates – Each component is documented using reusable and customizable templates, and Bit does most of the work for you automatically. Working with MDX? Maybe add some visual examples? no problem
Combination of independent renders – Each component is rendered completely independently, completely outside of the project, and the visual effects of the rendering (reloaded hot when writing the code) become part of each component’s documentation.
3. NX
NX is an advanced set of extensible development tools for Monorepos with a strong emphasis on modern full-stack Web technology.
The goal of NX is to provide a holistic development experience through the CLI (with editor plug-ins) and to provide controlled code sharing and consistent code generation. It also offers incremental builds, so it doesn’t rebuild and retest everything with every commit you make, speeding up build times.
With Nx, you can use your favorite framework and integrate with modern tools you might already be using. For example, NX lets you use out-of-the-box integration with Cypress, Jest, Typescript, Prettier, and other tools.
The NX team also provides the NX Cloud, which helps teams using NX deliver faster with smart computing memory and faster builds in the cloud.
4. Rush
Rush is a powerful Monorepo infrastructure open-source by Microsoft + that is designed to help you build and distribute many packages in one repository.
Some of the major features of Rush include a single NPM installation (which can also be used with Yarn and PNPM), so you can install all dependencies for all projects into a common folder, rebuilding an exact “node_modules” folder for each project using isolated symbolic links.
This also helps to ensure that there are no phantom dependencies, so you won’t accidentally import a library that’s missing in package.json, and you won’t find 10 dependency duplicates of lib in node_modules.
Automatic local linking means that all your projects automatically symlink to each other, and when you make a change, you can see the effect downstream without having to post anything and without any NPM link hassle.
Rush’s unique installation strategy generates a single shrink/lock file for a quick installation for all your projects. Rush will detect your dependency graph and build your projects in the right order, so if there are no direct dependencies between two packages, Rush will build them in parallel as separate processes.
If you’re only going to use a few items in your REPO, Rush offers subset and incremental builds, so Rush rebuild –to
Rush supports bulk publishing when you want to publish, so it detects which packages have changed, automatically jumps to all relevant version numbers, and runs NPM Publish in each folder.
Rush also contributes to the implementation and implementation of development policies. For example, when creating PR, you can ask developers to provide major/minor/patch log entries for affected projects, which will then be aggregated into a change log file at release time. It can also help you perform things like pre-release reviews, specific versions of dependencies, etc.
5. Lerna
Lerna (named after the home of Hydra, the many-headed beast) is a “tool for managing JavaScript projects with multiple packages”.
Lerna was created to solve the multi-package problem with Babel to optimize the workflow for managing multi-package repositories using Git and NPM. It is essentially a tool and script that can efficiently manage and publish many independent versions of packages in a Git repository.
my-lerna-repo/
package.json
packages/
package-1/
package.json
package-2/
package.json
Copy the codeThe two main Lerna commands are Lerna bootstrap and Lerna publish. Bootstrap wires the dependencies in the REPO together, and publish helps publish any updated packages.
You can use one of two modes to manage projects: Fixed or Independent.
Fixed-mode Lerna projects operate as a single version line, which is the version key stored in the lerna.json file at the root of your project. When you run Lerna Publish, if a module has been updated since the last release, it will be updated to the new version you published. This is the model Babel currently uses.
The Standalone Mode Lerna project allows maintainers to increase package versions independently of each other, and with each release you will receive a prompt for each package that has changed to specify whether it is a patch, small, large or custom change. Standalone mode lets you update the version of each package more specifically, which makes sense for a group of packages.
The “lerna.json” file is a list of globs that match the directory containing package.json, which is how Lerna identifies the “leaf” package (as opposed to managing development dependencies and scripts for the entire REPO). Example:
{
"version": 1.1.3 ""."npmClient": "npm"."command": {
"publish": {
"ignoreChanges": ["ignored-file"."*.md"]."message": "chore(release): publish"."registry": "https://npm.pkg.github.com"
},
"bootstrap": {
"ignore": "component-*"."npmClientArgs": ["--no-package-lock"]}},"packages": ["packages/*"]}Copy the codeEven if you don’t plan to publish to NPM, Lerna can still help manage versioning and common development tasks in Monorepo.
6. Bazel Build System (Google)
Google has launched Bazel Build System, an open source build and test tool similar to Make, Maven, and Gradle that uses a human-readable high-level build language. Bazel supports projects in multiple languages and builds output for multiple platforms. It supports a large code base in a large single repository or across multiple repositories and a large number of users.
Uber developers use Bazel to build their Go Monorepo. Uber writes most of its back-end services and libraries in Go, and in 2018, these services and libraries were all grouped into a large Go Monorepo that now has more than 100,000 files. Bazel allowed the project to expand, shorten build times, and support its growth.
This is a nice little open source project with Bazel as a demo: Thundergolfer/exemplar-bazel-Monorepo
Bazel is designed to work on a large scale and supports incremental sealed builds across distributed infrastructures that are necessary for large code bases. With Bazel’s remote cache, build servers can also share their build artifacts. Bazel caches all previously done work and tracks changes to file contents and build commands. Build and test packages only when packages or package dependencies change.
Bazel runs on Linux, macOS, and Windows. Bazel can build binaries and deployable packages from the same project for multiple platforms, including desktops, servers, and mobile devices. Supports many languages, and you can extend Bazel to support any other language or framework.
7. Buck Build System (Facebook)
Buck is a build system that encourages the creation of small reusable modules of code and resources, supporting a variety of languages on different platforms.
It was developed and used by Facebook as the official build system for FB units, and gained a reputation for being used by teams such as Uber developers to greatly shorten the build time. AirbnbEng’s team has sped up construction by 50 percent and shrunk the app by 30 percent.
Buck was designed to build a MonorePO, and the support for the Monorepo design inspired Buck to support the cell and the project.
Facebook’s experience is that keeping all dependencies in the same repository makes it easier to ensure that all developers have the correct version of the code and simplifies the atomic commit process.
Buck is commonly used for Android and iOS development.
8. Pants Build System (Twitter)
In 2014, Twitter launched its Monorepo build system called Pants. Today, with version V2, Pants aims to be a fast, scalable build system to accommodate an ever-growing code base. For now, it focuses on Python and will soon support other languages.
Pants uses fine-grained workflows and isolates each unit of work from side effects, so it can leverage all available kernels. Some of Pant’s best features include explicit dependency modeling, fine-grained invalidation, shared result caching, concurrent execution, remote execution, and extensibility and customisability through plug-in apis.
The Pants engine is written in Rust for performance. The build rules are written in typed Python 3 for familiarity and simplicity. The engine is designed so that fine-grained invalidation, concurrency, sealing, caching, and remote execution occur naturally without the intervention of the rule author.
9. Please build the system
Please is a cross-language build system that emphasizes high performance, portability, extensibility, and correctness.
Make sure that the build steps are performed in your own sealed environment and that you can only access the files and env variables that have been granted permission. Incremental build means that it builds only what it needs, and it also provides task parallelism, as well as distributed caching, to enable reliable and high-performance build systems on a large scale.
Please also aims to focus on the development experience, so you can enjoy a commonly used CLI and define aliases for common tasks that are automated.
Please, written in Go, provides all of this user experience with no runtime dependencies. Also, there is no single large workspace file that needs to handle much configuration.
10. Oao
Oao isn’t the most mature, rich, or easy-to-use tool on this list, but it’s fun nonetheless. It is a YARn-based, opinion monorepo management tool, p provides Monorepo functions such as installing all dependencies, adding/removing/upgrading subpackage dependencies, verifying version numbers, determining updated subpackages, publishing everything at once, updating the change log, etc.
Oao lets you run commands or package.json scripts on all subpackages, serial or parallel, optionally following the reverse dependency tree. Furthermore, it supports YARN workspaces, optimizes the Monorepo dependency tree as a whole, simplifies bootstrap and dependency addition/upgrade/removal.
Support for non-single package publishing: Benefit from oAO’s pre-release checking, tagging, version selection, change log updates, and can also be used in your single and non-single package. Note that Oao uses a synchronous version scheme, so a major version is configured in the package.json at the root level, and the sub-packages will also be synchronized with that version. You can try it here.
11. Bolt
Boltpkg is intended to be a “super-functional JavaScript project management tool”.
Bolt implements the concept of workspaces on top of Yarn. Bolt CLI is largely a replacement for Yarn CLI, and you can use it in any Yarn project.
As we know, workspaces are nested within a larger project/REPO, and each workspaces can have its own dependencies with its own code and scripts. Workspaces can also be organized into subdirectories.
With Bolt, you can install all of these package dependencies at once (and you can do it really, really fast). Also, when you specify a dependency from one workspace to another, it will be linked to the source code. That way, when you test your code, all your changes will be tested together.
Source: blog. Zhangbing. Site / 2020/12/26 /…
Open Source Recommendation Series
- Open source recommended | 10 amazing CSS suite of restoring ancient ways
- Open source recommended | Vue. Js shuffling hot selected library
- Open source recommended | 19 front-end developer survival tool
- Open source recommended | 10 + a cool Vue. Js components, templates and demo sample
- Open source recommended | JSONsite: using JSON file creation SPA page
- Open source recommended | using GPU. Js JavaScript performance
- Open source recommended | how to convert the HTML form into elegant PDF, several schemes comparison
- Open source recommended | I can not live without the five Vue. Js library
- Open source recommended | 8 JavaScript library can better deal with the local store
- Open source recommended | Node development artifact: use the Llama Node error Logs real-time visualization
- Open source recommended | you should know about 23 very useful NodeJS library
- Open source recommended | 10 beautiful VSCode Light color (Light)
- Open source recommended | 2019 six JavaScript library user authentication
- Open source recommended | UI library which support dark pattern?
- Open source recommended | 5 you don’t know JavaScript string processing library
- Open source recommended | 9 CSS tools for Web developers
- Open source recommended | in addition to ali iconfont icon library, and the 10 open source SVG icon library
- 7 great JavaScript product step guides that you won’t need