preface

Scaffolding, whether last year invited online sharing or front-end morning reading class more or less have recommended several periods, but today this is a detailed sharing. If you’re going to build scaffolding for your project, don’t miss this one. This morning reading article is shared with permission from Ali @Zhang Guoyu.

The text begins here

preface

The front-end project of our team was developed based on an internal backend framework, which was customized based on Vue and ElementUI, and added some modules designed by our team to further simplify the development of backend pages.

This framework is divided into three modules: basic component module, user authority module and data chart module. The development of background business layer should be based on the basic component module at least, and user authority module or data chart module can be added according to specific needs. Although VUE provides some scaffolding tools vuE-CLI, our project is developed and packaged based on multi-page configuration, which is somewhat different from the project structure and configuration generated by VUe-CLI. Therefore, we still need to modify many places manually when creating the project. Even for convenience, Copy it directly from the previous project and make magic changes. On the surface, there are not many problems, but there are many:

  • Repetitive work, tedious and time-consuming

  • A copy template is prone to extraneous code

  • There are a lot of configuration needs in the project, easy to ignore some configuration points, and then buried pits

  • Human error is always possible, and it takes time to correct errors when building a new project

  • Internal frameworks are also constantly iterating, and manual projects often don’t know what the latest version of the framework is, and using older versions of the framework may reintroduce bugs

To address these issues, I developed a scaffolding tool that dynamically generates the project structure based on interactions, automatically adds dependencies and configurations, and removes unwanted files.

Let’s take a look at my entire development history.

The basic idea

Before we start coding, let’s get some ideas. In fact, before implementing my own scaffolding, I repeatedly organized and analyzed the vuE-CLI implementation, found a lot of interesting modules, and borrowed some of its good ideas.

Vue-cli is to independently publish the project template on Git as a resource, and then download the template during the running, and render it by the template engine, and finally generate the project. The main purpose of this separation of project templates and tools is that the project template is responsible for the structure and dependency configuration of the project, while the scaffolding is responsible for the construction process of the project. These two parts are not closely related, and the separation ensures that the two parts are maintained independently. If the structure, dependencies, or configuration of the project changes, simply update the project template.

Following the idea of VUe-CLI, I also independently published the project template to Git, downloaded it through scaffolding tools, obtained the information of the new project through interaction with scaffolding, rendered the project template with the input of interaction as meta information, and finally got the infrastructure of the project.

Engineering structure

The project is developed based on NodeJS 8.4 and ES6. The directory structure is as follows

/ bin # -- -- -- -- -- - / lib command file # -- -- -- -- -- - tool module package. The jsonCopy the code

The following sections of the code require some understanding of promises to help you understand them.

Use commander. Js to develop command-line tools

Nodejs has built-in support for command-line operations. The bin field in package.json of the Node project can define command names and associated execution files.

{" name ":" macaw - cli ", "version" : "1.0.0", "description" : "my cli", "bin" : {" macaw ":". / bin/macaw. Js "}}Copy the code

A nodeJS project configured in this way will automatically create symlinks associated with the executable file in the [prefix]/bin directory of the system when using the -g option for global installation. If installed locally, the symlink is generated in the./node_modules/.bin directory. The advantage of this is that the nodeJS file can be executed directly from the terminal as if it were a command. To obtain the prefix, run the NPM config get prefix command.

hello, commander.js

Create a macaw.js file in the bin directory to process the command line logic.

touch ./bin/macaw.jsCopy the code

Next up is the commander. Js module developed by a github god named TJ. Commander. Js can automatically parse commands and parameters, combine multiple options, handle short arguments, etc. It is powerful and easy to use. See your project’s README for details on how to use it.

Write command line entry logic in macaw.js

#! /usr/bin/env nodeconst program = require('commander') // NPM I commander -dprogram.version ('1.0.0'). Usage ('<command> [project name]').command('hello', 'hello').parse(process.argv)Copy the code

Next, create macaw-hello.js in the bin directory and place a print statement

touch ./bin/macaw-hello.jsecho "console.log('hello, commander')" > ./bin/macaw-hello.jsCopy the code

So, test it with the Node command

node ./bin/macaw.js helloCopy the code

Not surprisingly, you can see a sentence on the terminal: Hello, commander.

Commander supports git-style subcommand processing. Based on the subcommand, you can automatically lead to a command execution file named in a specific format. The file name format is [command]-[subcommand], for example:

  • macaw hello => macaw-hello

  • macaw init => macaw-init

Define the init subcommand

We need a command to create a new project, and we can define a subcommand called init, using some common nouns.

Make some changes to bin/macaw.js.

Const program = require('commander')program.version('1.0.0').usage('<command> [project name]').command('init', 'Create new project ').parse(process.argv)Copy the code

Create an execution file associated with the init command in the bin directory

touch ./bin/macaw-init.jsCopy the code

Add the following code

#! /usr/bin/env nodeconst program = require('commander')program.usage('<project-name>').parse(process.argv) Let projectName = program.args[0]if (! ProjectName) {// project-name mandatory // equivalent to the --help option of the command, display help information, This is a built-in command option of Commander program.help() return}go()functiongo () {// reserved, processing subcommands}Copy the code

Note the first line #! /usr/bin/env node = Shebang /usr/bin/env

Project-name is a mandatory parameter, but I want to automate it a little bit.

  • The current directory is empty. If the name of the current directory is the same as project-name, the project is directly created in the current directory. Otherwise, the project root directory is created in the current directory with project-name as the name

  • The current directory is not empty. If there is no directory with the same name as project-name, create a directory with project-name as the root directory of the project. Otherwise, a message is displayed indicating that the project already exists.

According to the above Settings, and then do some improvement to the execution file

#! /usr/bin/env nodeconst program = require('commander')const path = require('path')const fs = require('fs')const glob = Require ('glob') // NPM I glob-dprogram. usage('<project-name>')// let projectName = program.args[0]if (! ProjectName) {// project-name mandatory // equivalent to the --help option of the command, display help information, This is a built-in command option of COMMANDER program.help() return}const list = glob.sync('*') // Traversing the current directory let rootName = Path.basename (process.cwd())if (list.length) {// If the directory is not empty if (list.filter(name => {const fileName = path.resolve(process.cwd(), path.join('.', name))const isDir = fs.stat(fileName).isDirectory()return name.indexOf(projectName) ! == -1 && isDir }).length ! == 0) {console.log(' project ${projectName} already exists')return} rootName = projectName} elseif (rootName === projectName) {rootName } else {rootName = projectName}go()functiongo () {// reserved, Log (path.resolve(process.cwd(), path.join('.', rootName)))}Copy the code

Create an empty directory in any path and execute the initialization commands we define in this directory

node /[pathto]/macaw-cli/bin/macaw.js init hello-cliCopy the code

Normally, you can see the path to the printed item on the terminal.


Use the download-git-repo command to download the template

The tool to download the template uses another Node module, download-git-repo. Refer to the README of the project to simply package the tool.

Create a download.js file in the lib directory

const download = require('download-git-repo')module.exports = function (target) { target = path.join(target || '.', '.download-temp')returnnewPromise(resolve, reject) { Branch can not ignore the download url (' https://github.com:username/templates-repo.git#master 'target, {clone: True}, (err) => {if (err) {reject(err)} else {// The downloaded template is stored in a temporary path. After the download is complete, you can notify the temporary path for subsequent processing. Resolve (target)}})}}Copy the code

The Download-Git-repo module is essentially a method that follows node.js CPS and handles asynchronous results with callbacks. If you’re familiar with Node.js, you probably know that there’s a downside to this approach. I wrapped it and converted it to the more popular Promise style of handling async.

Modify macaw-init.js once again

const download = require('./lib/download')... Functiongo () {download(rootName).then(target =>console.log(target)).catch(err =>console.log(err))}Copy the code

Once the download is complete, transfer the project template files from the temporary download directory to the project directory, and a simple scaffolding is basically complete. We won’t go into details about how to implement the migration, but refer to the Node.js API. If your node.js version is under 8, use stream and pipe. If it’s 8 or 9, use the new API — copyFile() or copyFileSync().

But…

The world is not as simple as we think. We might want some files or code in the project template to be processed dynamically. Such as:

  • The name, version number, description, and so on of the new project can be entered through the scaffolding interaction and then inserted into the template

  • The project template is not used for all files, and the scaffolding provides options to remove unwanted files or directories.

For this type of situation, we also need to use other toolkits to do this.

Use inquirer. Js to handle command-line interactions

Inquirer. Js can be used to handle command line interaction functions. The usage is actually quite simple:

const inquirer = require('inquirer') // npm i inquirer -Dinquirer.prompt([ { name: 'projectName', message: 'please input projectName'}]). Then (answers => {console.log(' you input projectName: ${answers.projectname} ')}).Copy the code

Prompt () takes data from a question object, stores the user’s input in an answer object during the user’s interaction with the terminal, and returns a Promise to retrieve the answer object through then(). So easy!

Next, macaw-init.js will be improved.

/ /... const inquirer = require('inquirer')const list = glob.sync('*')let next = undefinedif (list.length) {if (list.filter(name => {const fileName = path.resolve(process.cwd(), path.join('.', name))const isDir = fs.stat(fileName).isDirectory()return name.indexOf(projectName) ! == -1 && isDir }).length ! == 0) {console.log(' project ${projectName} already exists')return} next = promise.resolve (projectName)} elseif (rootName === ProjectName) {next = inquirer. Prompt ([{name: 'buildInCurrent', message: 'the current directory is empty and the directory name is the same as the projectName, do you want to create a project directly under the current directory?' type: 'confirm',default: true } ]).then(answer => {returnPromise.resolve(answer.buildInCurrent ? '. ': projectName) })} else { next = Promise.resolve(projectName)}next && go()functiongo () { next.then(projectRoot => {if (projectRoot ! == '.') { fs.mkdirSync(projectRoot) }return download(projectRoot).then(target => {return { projectRoot, downloadTemp: target } }) })}Copy the code

If the current directory is empty and the directory name is the same as the project name, verify that the project is created directly in the current directory through terminal interaction, which makes scaffolding more human.

As mentioned earlier, the name, version number, description, and other information of the new project can be directly inserted into the project template through terminal interaction, so further perfecting the interaction process.

/ /... Const latestVersion = require('latest-version') // NPM I latest-version -d //... functiongo () { next.then(projectRoot => {if (projectRoot ! == '.') { fs.mkdirSync(projectRoot) }return download(projectRoot).then(target => {return { name: projectRoot, root: projectRoot, downloadTemp: target } }) }).then(context => {return inquirer.prompt([ { name: 'projectName', message: 'project name ',default: context.name}, {name: 'projectVersion', message: 'projectVersion number ',default: '1.0.0'}, {name: 'projectDescription', message: 'Project introduction ',default: `A project named ${context.name}` } ]).then(answers => {return latestVersion('macaw-ui').then(version => { answers.supportUiVersion = versionreturn { ... context, metadata: { ... answers } } }).catch(err => {returnPromise.reject(err) }) }) }).then(context => {console.log(context) }).catch(err => {console.error(err) })}Copy the code

When the download is complete, the user is prompted to enter new project information. Of course, the problem of interaction is not limited to this, you can add more interaction problems depending on your project situation. The power of inquirer. Js is that it supports a wide variety of interaction types, including confirm, list, password, and checkbox, in addition to simple input. For details, see the README of the project.

Then, how to insert this input into the template, came another simple but not simple toolkit called Metalsmith.

Use Metalsmith to process templates

Quote from the official website:

An extremely simple, pluggable static site generator.

It’s a static website generator that can be used for batching template scenarios, like Wintersmith, Assemble, Hexo. One of its biggest features IS the EVERYTHING IS PLUGIN, so metalsmith IS essentially a glue framework that makes production work by gluing together various plugins.

Add variable placeholders to the project template

For the template engine I chose Handlebars. Of course, there are other options available, such as EJS, JADE, swiG.

Use handlebars syntax to make some changes to the template, such as modifying package.json in the template

{"name": "{{projectName}}","version": "{{projectVersion}}","description": "{{projectDescription}}","author": "Forcs Zhang","private": true,"scripts": {"dev": "node build/dev-server.js","start": "node build/dev-server.js","build": "node build/build.js","unit": "cross-env BABEL_ENV=test karma start test/unit/karma.conf.js --single-run","test": "npm run unit","lint": "Eslint -- ext. js,.vue SRC test/unit/specs"},"dependencies": {"element-ui": "^2.0.7","macaw-ui": "{{supportUiVersion}}", "vue" : "^ 2.5.2", "vue - the router" : "^ 2.3.1"},... }Copy the code

The name, version, and description fields of package.json are replaced with placeholders for handlebar syntax, and similar changes are made elsewhere in the template.

Scaffolding to achieve template interpolation function

Create generator.js in the lib directory and wrap metalsmith.

touch ./lib/generator.js// npm i handlebars metalsmith -Dconst Metalsmith = require('metalsmith')const Handlebars = require('handlebars')const rm = require('rimraf').syncmodule.exports = function (metadata = {}, src, dest = '.') {if (! SRC) {returnpromise.reject (newError(' invalid source: ${src}`)) }returnnewPromise((resolve, reject) => { Metalsmith(process.cwd()) .metadata(metadata) .clean(false) .source(src) .destination(dest) .use((files, metalsmith, done) => {const meta = metalsmith.metadata()Object.keys(files).forEach(fileName => {const t = files[fileName].contents.toString() files[fileName].contents = new Buffer(Handlebars.compile(t)(meta)) }) done() }).build(err => { rm(src) err ? reject(err) : resolve() }) })}Copy the code

Add build logic to macaw-init.js’s go().

/ /... const generator = require('.. /lib/generator')functiongo () { next.then(projectRoot => {// ... }). Then (context => {console.log(' created successfully :)'). Then (context => {console.log(' created successfully :)')}). {console.error(' failed to create: ${err.message} ')})}Copy the code

At this point, an interactive, dynamically interpolating scaffold for templates is almost complete.

Consolidate. Js is found in vue-CLI. If you are interested, you can check it out.

Beautify our scaffolding

Make scaffolding more human with some kits. Here are two toolkits found in vue-CLI:

  • Ora – Displays spinner

  • Chalk – Adds color to a boring terminal interface

These two tool kits are not complicated to use and will make the scaffold look more lofty

Optimize load wait interaction with ORA

Ora can be used in load wait scenarios, such as when a project template is being downloaded from scaffolding, or if there is an obvious wait to interpolate the template to generate the project.

Take downloading as an example to make some improvements to download.js:

npm i ora -D const download = require('download-git-repo')const ora = require('ora')module.exports = function (target) {  target = path.join(target || '.', '.download-temp')returnnewPromise(resolve, Reject) {const url = 'https://github.com:username/templates-repo.git#master' const spinner = ora (` are downloaded project template, the source address: ${url}`) spinner.start() download(url, target, { clone: true }, (err) => {if (err) { spinner.fail() // wrong :( reject(err) } else { spinner.succeed() // ok :) resolve(target) } }) }}Copy the code
Use Chalk to optimize the display effect of terminal information

Chalk can set colors for terminal text.

/ /... const chalk = require('chalk')const logSymbols = require('log-symbols')// ... functiongo () {// ... next.then(/* ... / * * /)... */. Then (context => {// success is displayed in green, Give positive feedback console.log(logsymbols.success, Log ()console.log(' chalk. Green '(' CD' + context.root + '\ NNPM install\ NNPM run dev')) Error (logsymbols. error, chalk. Red (' failed to create: ${error. Message} '))})})}Copy the code

Remove unwanted files from the template based on the input

Sometimes, not all files in a project template are needed. To ensure that the newly generated project is as clean as possible, we may need to verify the resulting project structure against scaffolding inputs and remove unwanted files or directories. For example, vuE-CLI will ask us whether we need to add the test module when creating the project. If not, the final generated project code does not contain the test-related code. How does this function work?

Ideas of implementation

I’ve followed git’s lead and created an ignore file that lists the files that need to be ignored in this ignore file with template syntax. The scaffolding builds the project by rendering the ignore file based on the input, removing unwanted template files based on the ignore file, and then rendering the actual project template.


Implementation scheme

With that in mind, I defined our own ignore file called templates.ignore.

Then add the file name that you want to ignore to the ignore file.

{{#unless supportMacawAdmin}}# The login page and password change page are not needed SRC /entry/login.js SRC /entry/password.js{{/unless}}# Ignore text itself templates. Ignore is not needed in the final generated projectCopy the code

Then add the temp. ignore processing logic to lib/generator.js

/ /... const minimatch = require('minimatch') // https://github.com/isaacs/minimatchmodule.exports = function (metadata = {}, src, dest = '.') {if (! SRC) {returnpromise.reject (newError(' invalid source: ${src}`)) }returnnewPromise((resolve, reject) => {const metalsmith = Metalsmith(process.cwd()) .metadata(metadata) .clean(false) .source(src) .destination(dest)// Check whether templates are in the downloaded project templates. Ignoreconst ignoreFile = path.join(SRC, 'templates.ignore')if (fs.existssync (ignoreFile)) {// Define a plugin for removing ignored files from templates Done) => {const meta = metalsmith.metadata() Const ignores = handlebars.compile (fs.readfilesync (ignoreFile).tostring ())(meta).split('\n').filter(item =>!! Item.length) object.keys (files). ForEach (fileName => {// hide the ignores (minimatch(fileName, ignorePattern)) {delete files[fileName] } }) }) done() }) } metalsmith.use((files, metalsmith, done) => {const meta = metalsmith.metadata()Object.keys(files).forEach(fileName => {const t = files[fileName].contents.toString() files[fileName].contents = new Buffer(Handlebars.compile(t)(meta)) }) done() }).build(err => { rm(src) err ? reject(err) : resolve() }) })}Copy the code

Based on the idea of plug-ins, Metalsmith is easy to extend and easy to implement. See the comments in the code for details.

conclusion

After the vue-CLI arrangement, with the help of many Node modules, the whole scaffolding implementation is not complicated.

  • Separate project templates from scaffolding tools for better maintenance of templates and scaffolding tools.

  • The command line is processed using commander.js

  • Handle the download via download-git-repo

  • Handle terminal interactions through Inquirer. Js

  • Interactive input items are inserted into the project template through Metalsmith and the template engine

  • Git uses templates. Ignore to dynamically remove unnecessary files and directories

The above is the main experience of my development of scaffolding, there are many inadequacies in the middle, and then slowly improve it in the future.

Finally, vuE-CLI can do a lot of things, see the project README and source code. For scaffolding development, you don’t have to completely build a wheel, but look at YEOMAN, another powerful module that allows you to quickly implement your own scaffolding tools.

About the author: @ Zhang Guoyu original: http://zhangguoyu.org/2017/12/10/developing-a-cli-on-nodejs/

Finally, a recommendation for you

Build scaffolding from scratch

# 672 teaches you how to build a front-end scaffolding tool from scratch

Daily sentence

The more busy and tired people are, the less willing they are to learn time management and life planning, and the less willing they are to settle down to study