One NPM package per day: egg-init

Egg-init serves as the scaffolding for egg.js, and today we’ll focus on an implementation inside.

The entrance

package.json

"bin": {
    "egg-init": "bin/egg-init.js"
  },
Copy the code

egg-init/bin/egg-init.js

#! /usr/bin/env node

'use strict';

const co = require('co');
const Command = require('.. ');

co(function* () {
  yield new Command().run(process.cwd(), process.argv.slice(2));
}).catch(err= > {
  // co returns a promise
  console.error(err.stack);
  // The shell process exits
  process.exit(1);
});
Copy the code

Knowledge:

#! /usr/bin/env nodeWhat’s the use?

For details, read What-exactly – does-usral-bin-env-node-do-at-the-beginning-of-node-files.

As you can easily understand, Shebang (also known as Hashbang) is a string of characters #! , which appears in the first two characters of the first line of the text file. In the case of Shebang in the file, the program loader of unix-like operating system will analyze the contents after Shebang, take these contents as the interpreter instruction, and call the instruction, and take the file path containing Shebang as the interpreter parameter [1][2].

The leading character can be followed by one or more whitespace characters, followed by the absolute path to the interpreter, which is used to invoke the interpreter. When a script is invoked directly, the caller uses the information provided by Shebang to invoke the corresponding interpreter, making the script file invoked just like a normal executable.

Next we can see the new Command() class. In combination with

const Command = require('.. ');
Copy the code

And package. Under the json

"main": "lib/init_command.js".Copy the code

Find the Command class from egg-init/lib/init_command.js.

egg-init/lib/init_command.js

Let’s take a look at the corresponding source code, the notes are very detailed.

'use strict';

const os = require('os');
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const urllib = require('urllib');
const updater = require('npm-updater');
const mkdirp = require('mkdirp');
const inquirer = require('inquirer');
const yargs = require('yargs');
const glob = require('globby');
const is = require('is-type-of');
const homedir = require('node-homedir');
const compressing = require('compressing');
const rimraf = require('mz-modules/rimraf');
const isTextOrBinary = require('istextorbinary');
const ProxyAgent = require('proxy-agent');

require('colors');

module.exports = class Command {
  constructor(options) {
    // Perform some initialization operations
    options = options || {};
    this.name = options.name || 'egg-init';
    // egg-init-config is the default boilerplate config
    this.configName = options.configName || 'egg-init-config';
    this.pkgInfo = options.pkgInfo || require('.. /package.json');
    this.needUpdate = options.needUpdate ! = =false;
    this.httpClient = urllib.create();

    this.inquirer = inquirer;
    // Special file name mapping, used when downloading resource copy
    this.fileMapping = {
      gitignore: '.gitignore'._gitignore: '.gitignore'.'_.gitignore': '.gitignore'.'_package.json': 'package.json'.'_.eslintrc': '.eslintrc'.'_.eslintignore': '.eslintignore'.'_.npmignore': '.npmignore'}; } *run(cwd, args) {
    // CWD: indicates the working directory where the Node process is currently running
    // args => process.argv.slice(2)
    // args: if you run egg-init --type simple test2 in shell, args is ['--type', 'simple', 'test2']
    const argv = this.argv = this.getParser().parse(args || []);
    this.cwd = cwd;
    // console.log('%j', argv);

    const proxyHost = process.env.http_proxy || process.env.HTTP_PROXY;
    if (proxyHost) {
      const proxyAgent = new ProxyAgent(proxyHost);
      this.httpClient.agent = proxyAgent;
      this.httpClient.httpsAgent = proxyAgent;
      this.log(`use http_proxy: ${proxyHost}`);
    }

    --registry= NPM
    this.registryUrl = this.getRegistryByType(argv.registry);
    this.log(`use registry: The ${this.registryUrl}`);

    if (this.needUpdate) {
      // Check whether the package needs to be updated
      // Take a look at the internal implementation https://github.com/node-modules/npm-updater
      yield updater({
        package: this.pkgInfo,
        registry: this.registryUrl,
        level: 'major'}); }// Get the project directory
    this.targetDir = yield this.getTargetDirectory();

    // use local template
    let templateDir = yield this.getTemplateDir();

    if(! templateDir) {// support --package=<npm name>
      // boilerplate package name
      Support for other Boilerplate PKgnames
      let pkgName = this.argv.package;
      if(! pkgName) {// list boilerplate
        const boilerplateMapping = yield this.fetchBoilerplateMapping();
        // ask for boilerplate
        let boilerplate;
        if (argv.type && boilerplateMapping.hasOwnProperty(argv.type)) {
          // Get the Boilerplate package name based on the type passed in by our shell
          For example, egg-init --type simple demo22
          / / the boilerplate is mapped to an egg - boilerplate - simple, https://github.com/eggjs/egg-boilerplate-simple
          boilerplate = boilerplateMapping[argv.type];
        } else {
          // If the type you entered is not found in the corresponding boilerplate list
          boilerplate = yield this.askForBoilerplateType(boilerplateMapping);
          if(! boilerplate)return;
        }
        // Print type, boilerplate name
        this.log(`use boilerplate: ${boilerplate.name}(${boilerplate.package}) `);
        / / such as egg - boilerplate - simple
        pkgName = boilerplate.package;
      }
      // download boilerplate
      templateDir = yield this.downloadBoilerplate(pkgName);
    }

    // copy template
    yield this.processFiles(this.targetDir, templateDir);
    // done
    this.printUsage(this.targetDir);
  }
  / * * * *@param {object} obj origin Object
   * @param {string} key group by key
   * @param {string} otherKey  group by other key
   * @return {object} result grouped object
   */
  groupBy(obj, key, otherKey) {
    const result = {};
    for (const i in obj) {
      let isMatch = false;
      for (const j in obj[i]) {
        // check if obj[i]'s property is 'key'
        if (j === key) {
          const mappingItem = obj[i][j];
          if (typeof result[mappingItem] === 'undefined') {
            result[mappingItem] = {};
          }
          result[mappingItem][i] = obj[i];
          isMatch = true;
          break; }}if(! isMatch) {// obj[i] doesn't have property 'key', then use 'otherKey' to group
        if (typeof result[otherKey] === 'undefined') { result[otherKey] = {}; } result[otherKey][i] = obj[i]; }}return result;
  }
  /**
   * show boilerplate list and let user choose one
   *
   * @param {Object} mapping - boilerplate config mapping, `{ simple: { "name": "simple", "package": "egg-boilerplate-simple", "description": "Simple egg app boilerplate" } }`
   * @return {Object} boilerplate config item
   */
  * askForBoilerplateType(mapping) {
    // group by category
    // group the mapping object by property 'category' or 'other' if item of mapping doesn't have 'category' property
    const groupMapping = this.groupBy(mapping, 'category'.'other');
    const groupNames = Object.keys(groupMapping);
    let group;
    // If there are more than one, let the user select the boilerplate
    if (groupNames.length > 1) {
      const answers = yield inquirer.prompt({
        name: 'group'.type: 'list'.message: 'Please select a boilerplate category'.choices: groupNames,
        pageSize: groupNames.length,
      });
      group = groupMapping[answers.group];
    } else {
      // By default, the first one is only one
      group = groupMapping[groupNames[0]];
    }

    // ask for boilerplate
    const choices = Object.keys(group).map(key= > {
      const item = group[key];
      return {
        name: `${key} - ${item.description}`.value: item,
      };
    });

    choices.unshift(new inquirer.Separator());
    const { boilerplateInfo } = yield inquirer.prompt({
      name: 'boilerplateInfo'.type: 'list'.message: 'Please select a boilerplate type',
      choices,
      pageSize: choices.length,
    });
    if(! boilerplateInfo.deprecate)return boilerplateInfo;

    // ask for deprecate
    const { shouldInstall } = yield inquirer.prompt({
      name: 'shouldInstall'.type: 'list'.message: 'It\'s deprecated'.choices: [{name: ` 1.${boilerplateInfo.deprecate}`.value: false}, {name: '2. I still want to continue installing'.value: true,}]});if (shouldInstall) {
      return boilerplateInfo;
    } else {
      console.log(`Exit due to: ${boilerplateInfo.deprecate}`);
      return; }}/**
   * ask user to provide variables which is defined at boilerplate
   * @param {String} targetDir - target dir
   * @param {String} templateDir - template dir
   * @return {Object} variable scope
   */
  * askForVariable(targetDir, templateDir) {
    let questions;
    try {
      questions = require(templateDir);
      / loading/here is: / var/folders/ws/zpkrs6dn7qd7_rqkzk8d2bqr0000gn/T/egg - init - boilerplate/package/index. The content of js
      / / such as
      / / {
      // name: {
      // desc: 'project name',
      / /},
      // description: {
      // desc: 'project description',
      / /},
      // author: {
      // desc: 'project author',
      / /},
      // keys: {
      // desc: 'cookie security keys',
      // default: Date.now() + '_' + random(100, 10000),
      / /},
      // };
      // support function
      if (is.function(questions)) {
        questions = questions(this);
      }
      // use target dir name as `name` default
      if(questions.name && ! questions.name.default) { questions.name.default = path.basename(targetDir).replace(/^egg-/.' '); }}catch (err) {
      if(err.code ! = ='MODULE_NOT_FOUND') {
        this.log(`load boilerplate config got trouble, skip and use defaults, ${err.message}`.yellow);
      }
      return {};
    }

    // Collect some configuration of the boilerplate input by the user
    this.log('collecting boilerplate config... ');
    const keys = Object.keys(questions);
    // Silent: Use the default value, and do not ask the user
    if (this.argv.silent) {
      const result = keys.reduce((result, key) = > {
        const defaultFn = questions[key].default;
        const filterFn = questions[key].filter;
        if (typeof defaultFn === 'function') {
          result[key] = defaultFn(result) || ' ';
        } else {
          result[key] = questions[key].default || ' ';
        }
        if (typeof filterFn === 'function') {
          result[key] = filterFn(result[key]) || ' ';
        }

        return result;
      }, {});
      this.log('use default due to --silent, %j', result);
      return result;
    } else {
      // Ask the user interactively
      return yield inquirer.prompt(keys.map(key= > {
        const question = questions[key];
        return {
          type: question.type || 'input'.name: key,
          message: question.description || question.desc,
          default: question.default,
          filter: question.filter,
          choices: question.choices, }; })); }}/**
   * copy boilerplate to target dir with template scope replace
   * @param {String} targetDir - target dir
   * @param {String} templateDir - template dir, must contain a folder which named `boilerplate`
   * @return {String[]} file names
   */
  * processFiles(targetDir, templateDir) {
    // targetDir is your project path
    // templateDir is where we downloaded the template resources
    // The boilerplate in templateDir is the actual template file we agreed on
    const src = path.join(templateDir, 'boilerplate');
    // locals: input or default configuration project information for users
    {name: '1', description: '2', author: '3', keys: '4'}
    const locals = yield this.askForVariable(targetDir, templateDir);
    // glob is a node package that implements a file lookup similar to the glob card used on Unix systems
    // Match the file under the target template according to the rule, return the array
    const files = glob.sync('* * / *', {
      cwd: src,
      dot: true.onlyFiles: false.followSymlinkedDirectories: false}); files.forEach(file= > {
      const { dir: dirname, base: basename } = path.parse(file);
      // Copy the source path
      const from = path.join(src, file);
      const name = path.join(dirname, this.fileMapping[basename] || basename);
      // Copy the destination path
      const to = path.join(targetDir, this.replaceTemplate(name, locals));

      const stats = fs.lstatSync(from);
      if (stats.isSymbolicLink()) {
        const target = fs.readlinkSync(from);
        fs.symlinkSync(target, to);
        this.log('%s link to %s', to, target);
      } else if (stats.isDirectory()) {
        mkdirp.sync(to);
      } else if (stats.isFile()) {
        const content = fs.readFileSync(from);
        this.log('write to %s', to);

        // check if content is a text file
        const result = isTextOrBinary.isTextSync(from, content)
          ? this.replaceTemplate(content.toString('utf8'), locals)
          : content;
        fs.writeFileSync(to, result);
      } else {
        this.log('ignore %s only support file, dir, symlink', file); }});return files;
  }

  /**
   * get argv parser
   * @return {Object} yargs instance
   */
  getParser() {
    // yargs: a library for parsing CLI parameters
    return yargs
      // Sets the CLI usage information that will be displayed when the --help command is called. $0 represents the name of the current script
      .usage('init egg project from boilerplate.\nUsage: $0 [dir] --type=simple')
      .options(this.getParserOptions())
      // Create an alias for the -h option --help
      .alias('h'.'help')
      // Display the version
      .version()
      // Bind the help command to option h.
      .help();
  }

  /** * parse the configuration options that you see when you type egg-init -h *@return {Object} opts* /
  getParserOptions() {
    return {
      type: {
        type: 'string'.description: 'boilerplate type',},dir: {
        type: 'string'.description: 'target directory',},force: {
        type: 'boolean'.description: 'force to override directory'.alias: 'f',},template: {
        type: 'string'.description: 'local path to boilerplate',},package: {
        type: 'string'.description: 'boilerplate package name',},registry: {
        type: 'string'.description: 'npm registry, support china/npm/custom, default to auto detect'.alias: 'r',},silent: {
        type: 'boolean'.description: 'don\'t ask, just use default value',}}; }/** * corresponds to registry * according to the alias name shortcut mapping@param {String} key - short name, support `china / npm / npmrc`, default to read from .npmrc
   * @return {String} registryUrl* /
  getRegistryByType(key) {
    switch (key) {
      case 'china':
        return 'https://registry.npm.taobao.org';
      case 'npm':
        return 'https://registry.npmjs.org';
      default: {
        if (/^https? : /.test(key)) {
          // Replace the extra '/' at the end
          // 'https://registry.nodejitsu.com/'.replace(/\/$/, '') => 'https://registry.nodejitsu.com'
          return key.replace(/ / / $/.' ');
        } else {
          // support .npmrc
          // Get the user directory
          // For example, NPM config is written to your.npmrc
          const home = homedir();
          let url = process.env.npm_registry || process.env.npm_config_registry || 'https://registry.npmjs.org';
          // Check whether the file path is CNPM or TNPM
          if (fs.existsSync(path.join(home, '.cnpmrc')) || fs.existsSync(path.join(home, '.tnpmrc'))) {
            url = 'https://registry.npm.taobao.org';
          }
          url = url.replace(/ / / $/.' ');
          returnurl; }}}}/**
   * ask for target directory, will check if dir is valid.
   * @return {String} Full path of target directory
   */
  * getTargetDirectory() {
    // Get the destination directory name
    const dir = this.argv._[0] | |this.argv.dir || ' ';
    // Splice the absolute path of the directory
    let targetDir = path.resolve(this.cwd, dir);
    // Determine whether to force an overwrite update
    const force = this.argv.force;

    // Verify whether the directory is valid
    const validate = dir= > {
      // If it does not exist, return true
      if(! fs.existsSync(dir)) { mkdirp.sync(dir);return true;
      }

      // If it is not a directory, it is a file
      if(! fs.statSync(dir).isDirectory()) {return `${dir} already exists as a file`.red;
      }

      // If it is an existing directory, get the file names in it
      const files = fs.readdirSync(dir).filter(name= > name[0]! = ='. ');
      if (files.length > 0) {
        // Determine whether to forcibly overwrite files in the target directory
        if (force) {
          this.log(`${dir} already exists and will be override due to --force`.red);
          return true;
        }
        return `${dir} already exists and not empty: The ${JSON.stringify(files)}`.red;
      }
      return true;
    };

    // if argv dir is invalid, then ask user
    const isValid = validate(targetDir);
    if(isValid ! = =true) {
      // If it is illegal, print the corresponding prompt
      this.log(isValid);
      // Ask the user to enter the correct project directory
      const answer = yield this.inquirer.prompt({
        name: 'dir'.message: 'Please enter target dir: '.default: dir || '. '.filter: dir= > path.resolve(this.cwd, dir),
        validate,
      });
      targetDir = answer.dir;
    }
    this.log(`target dir is ${targetDir}`);
    return targetDir;
  }

  /**
   * find template dir from support `--template=`
   * @return {undefined|String} template files dir
   */
  * getTemplateDir() {
    // Get the project template URL from the named line, that is, support customizing your own template
    let templateDir;
    // when use `egg-init --template=PATH`
    if (this.argv.template) {
      templateDir = path.resolve(this.cwd, this.argv.template);
      if(! fs.existsSync(templateDir)) {this.log(`${templateDir} is not exists`.red);
      } else if(! fs.existsSync(path.join(templateDir,'boilerplate'))) {
        // The template should contain a convention for the boilerplate directory
        / / such as https://github.com/eggjs/egg-boilerplate-plugin/tree/master/boilerplate
        this.log(`${templateDir} should contain boilerplate folder`.red);
      } else {
        this.log(`local template dir is ${templateDir.green}`);
        returntemplateDir; }}}/**
   * fetch boilerplate mapping from `egg-init-config`
   * @param {String} [pkgName] - config package name, default to `this.configName`
   * @return {Object} boilerplate config mapping, `{ simple: { "name": "simple", "package": "egg-boilerplate-simple", "description": "Simple egg app boilerplate" } }`
   */
  * fetchBoilerplateMapping(pkgName) {
    const pkgInfo = yield this.getPackageInfo(pkgName || this.configName, true);
    // Get the boilerplate configured under config
    // https://github.com/eggjs/egg-init-config/blob/a50d3ce45f6363467359a6842f63d13f9bf2afde/package.json#L8
    const mapping = pkgInfo.config.boilerplate;
    Object.keys(mapping).forEach(key= > {
      const item = mapping[key];
      item.name = item.name || key;
      // from indicates a dependency
      item.from = pkgInfo;
    });
    return mapping;
  }

  /** * print Usage guide */
  printUsage() {
    this.log(`usage:
      - cd The ${this.targetDir}
      - npm install
      - npm start / npm run dev / npm test
    `);
  }

  /**
   * replace content with template scope,
   * - `{{ test }}` will replace
   * - `\{{ test }}` will skip
   *
   * @param {String} content - template content
   * @param {Object} scope - variable scope
   * @return {String} new content
   */
  replaceTemplate(content, scope) {
    return content.toString().replace(/ (\ \)? {{ *(\w+) *}}/g.(block, skip, key) = > {
      if (skip) {
        return block.substring(skip.length);
      }
      return scope.hasOwnProperty(key) ? scope[key] : block;
    });
  }

  /**
   * download boilerplate by pkgName then extract it
   * @param {String} pkgName - boilerplate package name
   * @return {String} extract directory
   */
  * downloadBoilerplate(pkgName) {
    const result = yield this.getPackageInfo(pkgName, false);
    // Get the downloaded.tgz path
    / / https://registry.npmjs.org/egg-boilerplate-simple/-/egg-boilerplate-simple-3.3.1.tgz
    const tgzUrl = result.dist.tarball;

    this.log(`downloading ${tgzUrl}`);
    // Save the files in the default temporary folder of the OPERATING system
    / / like/var/folders/ws/zpkrs6dn7qd7_rqkzk8d2bqr0000gn/T/egg - init - boilerplate
    const saveDir = path.join(os.tmpdir(), 'egg-init-boilerplate');
    // Delete the previous one each time
    yield rimraf(saveDir);

    // Download resources
    const response = yield this.curl(tgzUrl, { streaming: true.followRedirect: true });
    // Select the gzip format and call compressFile,
    / / https://zhuanlan.zhihu.com/p/33783583 interested in reading
    yield compressing.tgz.uncompress(response.res, saveDir);

    this.log(`extract to ${saveDir}`);
    // Return to download the decompressed resource path: remember to splicebox /package/
    return path.join(saveDir, '/package');
  }

  /** * encapsulates the request resource method *@param {String} url - target url
   * @param {Object} [options] - request options
   * @return {Object} response data
   */
  * curl(url, options) {
    return yield this.httpClient.request(url, options);
  }

  /**
   * get package info from registry
   *
   * @param {String} pkgName - package name
   * @param {Boolean} [withFallback] - when http request fail, whethe to require local
   * @return {Object} pkgInfo* /
  * getPackageInfo(pkgName, withFallback) {
    this.log(`fetching npm info of ${pkgName}`);
    try {
      // Register is a query service
      / / https://registry.npm.taobao.org/egg-init-config/latest
      const result = yield this.curl(`The ${this.registryUrl}/${pkgName}/latest`, {
        dataType: 'json'.followRedirect: true.maxRedirects: 5.timeout: 5000});// Assert is thrown if the query fails
      assert(result.status === 200.`npm info ${pkgName} got error: ${result.status}.${result.data.reason}`);
      return result.data;
    } catch (err) {
      if (withFallback) {
        this.log(`use fallback from ${pkgName}`);
        return require(`${pkgName}/package.json`);
      } else {
        throwerr; }}}/** * log with prefix */
  log() {
    // The wrapped console.log method is prefixed with the [egg-init] prefix
    const args = Array.prototype.slice.call(arguments);
    args[0] = ` [The ${this.name}] `.blue + args[0];
    console.log.apply(console, args); }};Copy the code

Next, I’ll show you what the process goes through with a shell example.

egg-init --type simple22 demo22
Copy the code

new Command(): Creates Command instances and performs initialization operations, such as configName. The default is configName'egg-init-config'. mounthttpClientUsed to request download, mountinquirerFor shell interaction, inject some configuration information, such as pkgInfo, defaultpackage.jsonInformation, andfileMappingMapping of special file names.
Execute instancerun(process.cwd(), process.argv.slice(2))Methods.
- throughyargsParses the parameters passed in by the user shell and configures some shell examples to display version information, option information, usage information, etc
- To obtainregistrySupport shell--registry=npmor--registy=https://registry.npmjs.org/, as well as.npmrcWay. Registry is used for subsequent package queries
- Check whether packages need to be updated
- Get the project directory path based on shell parameters
  - Determines whether directory pathnames are valid
    - If it does not exist, create it
    - If it is not a directory but a file, an error message is displayed
    - If it is an existing directory, get all the file names in it, according to--forceParameter prompts whether to force overwrite
  - If not, let the user enter the project directory name by shell interaction
- Gets the project template path from the command linetemplateDir, that is, support to customize their own templatesegg-init --template=PATH.
- If the user is not definedtemplateDirAnd support from--package=<npm name>Get the template.
  - If the user also does not specify--pkgName, then read the egg’s own default,egg-init-config.
  - Get the Boilerplate package name based on the type passed in by our shell, for exampleegg-init --type simple demo22, then the boilerplate mapping is egg-boilerplate-simple.https://github.com/eggjs/egg-boilerplate-simple.
  - If the type you entered is not found in the corresponding Boilerplate list.
    - If the number of Boilerplate lists is greater than 1, let the user select them interactively.
    - If the number of Boilerplate lists is 1, the first one is default.
- After obtaining the Boilerplate package name, splice the path and download the corresponding resources.
- Save the files in the default temporary folder of the OPERATING system /var/folders/ws/zpkrs6dn7qd7_rqkzk8d2bqr0000gn/T/egg-init-boilerplateDownload and unzip to this path.
- Copy from the save path to the project target path.
- In the shell output prompt operation.

In the following figure, we can see the whole process of logging shell execution.

Egg boilerplate is managed by default through the config field under egg-init-config package.json.

The order between dependencies is indicated by from. Let’s take this as an example.

  "simple": {
        "package": "egg-boilerplate-simple"."description": "Simple egg app boilerplate"
      }
Copy the code

Egg-boilerplate-simple is the template we really pull.

After reading this, you should have an idea of what a CLI process is that you can apply to your own projects.

Well, that’s all for today. See you next time.

Ps: If you are also interested in Node, please follow my official account: XYZ Programming Diary.

The entrance

egg-init/bin/egg-init.js

egg-init/lib/init_command.js

Related Posts

Quickly configure a front end Macbook

Git reset differs from Git Revert

Interviewer: How do you correctly identify data types in Javascript?