Monorepo basics are required for this article

How Node.js project deployments differ from pure Web projects

In Monorepo, the output of a pure Web project is undeniably simple. Webpack packs/splits/categorizes all the resources nicely and outputs them to the dist/ Output folder. The subsequent product processing perspective can only be concentrated in the final Dist/OutPU folder.

Node.js projects, or more specifically Node.js server projects, are quite different. Regular projects do not include bundles. The end product is in the sense that it contains all the dependencies that are used in runtime, that is, the dependencies installed in node_modules are also part of the end product. This is easier to handle in the case of Multirepo, using the production only option of the package management tool to install dependencies, and then uploading the entire REPO package to a container in a real production environment such as Docker. In the case of Monorepo, packaging the entire REPO would obviously be problematic because it would be too large and his installed dependency would tend to be the entire Monorepo latitude. Therefore, the Monorepo toolchain with strong intelligence front-end is constantly trying to optimize the product output of Node.js project in the process of its own evolution.

Ancient – Copy Lock & Scandisk

# packages/xxxx/build.sh
#! /bin/bash

# remove node_modules soft link
rm -rf node_modules output static_resources

# install dependencies
Install using the yarn.lock file of the current projectcp -r .. /.. /yarn.lock yarn.lock# Delete package.json from the root directory and cancel the yarn workspace staterm -rf .. /.. /package.json# yarn handles a BUG in the local TGZ
yarn install --registry=http://our.own.registry
yarn install --registry=http://our.own.registry

Create an output directory
yarn run build
# give bootstrap.sh executable permission
chmod a+x bootstrap.sh

# copy file
mkdir output
cp -r $(ls | grep -v output) output

mkdir static_resources
Copy the code

At the beginning, the monorePO of the smart front end was maintained in pure YARN form. The above build script comes from packages/ XXXX /build.sh. In the root build script, the build script of the specified package is called based on environment variables. In simple terms, this script removes all monorepo-based effects of the project, then copies the top-level lock into the project and executes YARN, This results in a dependent node_modules that is only used by the project and ensures that the standalone version lock is consistent with the monorepo case. Here’s a quick explanation of three key commands.

rm -rf node_modules output static_resource

Delete node_modules first, because YARN has hoist logic in workspace state and may install some dependencies in root/node_modules. Therefore, in the current state (YARN has installed all dependencies of Monorepo in workspace mode), node_modules in the project is untrusted and needs to be deleted. Output and static_resource are output files that might be generated in the workspace state (some packages might have postinstall logic executed).

cp -r .. /.. /yarn.lock yarn.lock & rm -rf .. /.. /package.json

Copy the lock file from the root directory to the current package to ensure that the version number of resolve is the same as that installed in monorepo. (It has been proved that unnecessary information in yarn.lock does not affect the installation result.)

At the same time, delete package.json from monorepo root to prevent Yarn from thinking that it is still in monorepo. Otherwise, many default behaviors of Yarn install will be affected.

yarn install --registry=``http://bnpm.byted.org x2

Postinstall: localpackage localpackage localpackage localpackage localpackage localpackage localpackage /.. / XXX.tar. gz (see Selective Dependency Resolutions and YARN Add for details). To solve the problem of installing manually written local tar.gz, run YARN twice.

Modern – Yarn isolateWorkspace

# packages/xxxx/build.sh
#! /bin/bash

cd ../..
yarn deploy:node @xxx/xxx  # === yarn isolate-workspace -o output -w @xxx/xxx
cd output
yarn install --registry=http://bnpm.byted.org

# give bootstrap.sh executable permission
chmod a+x bootstrap.sh
cd. mkdir static_resourcesCopy the code

In December, 19, Junchen introduced LerNA to Monorepo and yarn-workspace (Yarn-workspace isolator) to complete deployment of Node project. The isolate-workspace converges the functions performed by the original script and ADAPTS the internal dependencies of Monorepo. Here’s a closer look at what happens inside the Isolate-Workspace.

Transfer target package + rewrite package.json

The first step is to move the target package to the production directory, which is like a normal single-package repO, and then start traversing the target’s dependencies based on the YARN Root workspace field to see if there are dependencies in monorepo. Json dependecy/devDependency: “@xxx/yyy”: “file: path/to/output/node_modules/@xxx/yyy” Yarn will treat this as a local dependency, and will treat it as a downloaded package when installing, and install the dependency of the local package.

Transfer the monorepo dependencies of the target package + override the package.json dependencies

In the same way as in the previous steps, all packages dependent on the workspace will be placed in node_modules in the artifacts folder, and if their dependencies have dependencies in monorepo, they will be treated in the same way.

Ref: github.com/choffmeiste…

But there are two small caveats

  1. Because the version value in package.json is changed, the original yarn.lock cannot be used any more. After the preceding steps are complete, yarn install is required again. The resolve version in the lock is invalid. If you install the resolve version again, it may be different from the lock version.
  2. If sibling package exists in devDependencies of the target package and is used later, it will cause an error because the ISOLate-workspace only handles dependencies in devDependencies.

Modern – Rush Deploy

# packages/xxxx/build.sh

#! /bin/bashnode .. /.. /common/scripts/install-run-rush.js deploy-node -target @xxx/xxx --output outputcd. /.. /output# give bootstrap.sh executable permission
chmod a+x bootstrap.sh
Copy the code

In early 21, MonorePO with a strong smart front end migrated from YARN + LERna to Rush + PNPM. The migration of the toolchain also resulted in the unavailability of the original isolate workspace, because PNPM organized node_modules differently and managed workspace/project differently. Fortunately, Rush itself provides a set of capabilities for generating Node.js deployment products, known as Rush Deploy, but since the end product of Rush Deploy contains the distribution of files in Monorepo, it is necessary to complete the deployment while meeting some cloud platform specifications. We still wanted to add some extra logic before and after rush deploy, so we added our own Rush Command Rush deploy-node.

Pre Rush Deploy (For Web)

While migrating Rush Deploy, we found some boilerplate code generated by convention in most backend Node.js projects. For example, admin_node is accompanied by an Admin_Web, and the artifacts in the Web are served statically by the Node project after a directory transfer. Therefore, we write the logic of the associated web/ Node relationship in a unified script, which is convenient for later maintenance.

Rush Deploy

Rush Deploy is a complete set of deployment tools that support multiple deploy scenarios for a single MonorePO. For Monorepo, we used the simplest form and relied heavily on deploy’s ability to separate the entire MonorePO and its already installed dependencies.

Rush Deploy prints the target package and its dependencies in Monorepo and installed dependencies to the artifacts folder, while retaining the overall directory structure of the original Monorepo file. In other words, it can be interpreted as deleting all unrelated files in Monorepo that are not dependent onor used based on the target package, and then printing the entire Monorepo directory.

An extra note here is that deploy has default filter logic. That is, the files that judge the target package based on.npmignore need to be moved to the artifacts folder, and the.gitignore file will take effect if the.npmignore file does not exist. Most of our Node scenarios are typescript-based server projects that need to be compiled into JS to run, and there is no.npmignore file in the project. So we set “includeNpmIgnoreFiles” to true in the deploy configuration file and include it automatically before rush deploy is executed! *. Js and! **/*.js.npmignore to ensure that ts complier output js will eventually be transferred to the production folder.

IO /pages/maint…

Post Rush Deploy (Facading)

In the case of internal deployment, we have many conventions that rely on the files in the root directory of the production file, such as xsettings.txt, bootstrap.sh, etc. After switching to rush deploy, the target package no longer exists in the root of the production directory, so we wrote the corresponding Facading logic for each convention to ensure that the contract files of the target package appear in the root of the production directory as before. This way, the Node project’s runtime configuration on a specific PaaS cloud platform or other platforms does not need to change.

  // facade xsettings.txt
  const sourceSettingsPyPath = path.resolve(
    monoRoot,
    mainProjectSubPath,
    'xsettings.txt',);const targetSettingsPyPath = path.resolve(distDirname, 'xsettings.txt');
  if (fs.existsSync(sourceSettingsPyPath)) {
    logger.info(
      '[START] copy xsettings.txt ' +
        `${sourceSettingsPyPath} -> ${targetSettingsPyPath}`,);await fs.copyFile(sourceSettingsPyPath, targetSettingsPyPath);
    logger.info('[DONE] copy xsettings.txt');
  }

  // facade bootstrap.sh
  if (fs.existsSync(path.join(mainProjectSubPath, 'bootstrap.sh'))) {
    const facadeBootstrapShellPath = path.resolve(distDirname, 'bootstrap.sh');
    logger.info(`[START] create bootstrap.sh ${facadeBootstrapShellPath}`);
    await fs.outputFile(
      facadeBootstrapShellPath,
      ` #! /bin/bash source${path.join(mainProjectSubPath, 'bootstrap.sh')}
  `,); logger.info('[DONE] create bootstrap.sh');
  }

  // facade bootstrap.js
  if (fs.existsSync(path.join(mainProjectSubPath, 'bootstrap.js'))) {
    const facadeBootstrapJsPath = path.resolve(distDirname, 'bootstrap.js');
    logger.info(`[START] create bootstrap.js ${facadeBootstrapJsPath}`);
    await fs.outputFile(
      facadeBootstrapJsPath,
      `
  require('./${path.join(mainProjectSubPath, 'bootstrap.js')}')
  `,); logger.info('[DONE] create bootstrap.js');
  }
Copy the code

After migrating with Rush Deploy, we resolved CI/CD efficiency issues caused by lock failure and multiple installations. Overall, the benefits of efficiency and stability are considerable.

Welcome to “Byte front-end ByteFE” resume delivery email “[email protected]”