Build scaffolding from scratch

Within the group already has a very perfect and smooth development, release process, usually only need to move silently belongs to own that piece of brick, but every time a kind of new technology in the community, want to try I always lack of a “hands”, can quickly to new technologies to integrated into your own scaffold, or workflow, based on this purpose, thought began to do

Scaffold for the benefits of the team is self-evident, can through the command line mode to quickly generate a seed file, development as well as output after building code, usually we only need to develop, with complex compilation process, rather than building services dealing process, in addition, can also be the node we need modules are installed to the scaffold, Instead of installing a huge node_module, we’ll just develop it, keep the directory clean, and even scaffolding can be combined with subsequent continuous integration to provide more power

Building scaffolding from scratch requires some front-end engineering knowledge. I recommend the WebPack guide, which covers a lot of front-end engineering things to do. In fact, I went up step by step from here, and finally developed the scaffolding qD-CLI. Scaffolding development is essentially webpack. Webpack is used to build workflows, and can eventually be packaged as a command line tool using Commander. Commander is described in detail in this article, not to be retold: Node.js based scaffolding tool development experience

This article from the following three aspects to do the introduction, build: how to step by step development of QD-CLI (including my understanding of front-end engineering), QD-CLI installation, use, features, some pits encountered in the process of build

Set up

Scaffolding technical scheme selection

Start with something simple and work your way up, so for now we are looking at only supporting mobile projects and the VUE stack

Technical solution: Workflow preparation without suspense to choose Webpack, now the most popular front-end packaging tool, Webpack first to solve the front-end modular problem, out of the box, native support ES Module, here choose the latest Webpack4, on the other hand, workflow integration into cli using Commander

How is the development environment set up

The following three aspects are mainly considered:

• Local server

In the development environment, we need to have a server to start and refresh our application automatically. Sometimes we even want to set up a proxy to facilitate front-end and back-end tuning. We can use Webpack-dev-server, which is easy to configure

// webpack.config.js
module.exports = {
  // ...
+ devServer: {
+   ...
+   contentBase: cwd('dist'), + proxy: { ... +}}}Copy the code

• Support for thermal overloading

After changing the code without manually refreshing the browser can preview the effect, fast and convenient, even if js hot reload a little pit, sometimes need to manually refresh, but the overall benefits still outweigh the disadvantages

const webpack = require('webpack');

module.exports = {
  devServer: {... + hot:true.contentBase: cwd('dist'),
    proxy: {... }},plugins: [+new webpack.NamedModulesPlugin(),
+   new webpack.HotModuleReplacementPlugin()
  ]
}
Copy the code

• Provide sourcemap

Webpack-packed code is bad for us to locate the error location, Soucemap can help us accurately locate the source code error location

const webpack = require('webpack');

module.exports = {
+ devtool: 'inline-source-map'
  devServer: {
    hot: true,
    contentBase: cwd('dist'),
    proxy: { ... }
  },
  plugins: [
    new webpack.NamedModulesPlugin(),
    new webpack.HotModuleReplacementPlugin()
  ]
}
Copy the code

More Sourcemap options

A production environment with the simplest source-Map is fine, because a more complex source-map is usually bulky

Generated environment package is too big how to do, how to deal with the cache problem 😰

The build environment needs to optimize the size of the code as much as possible, and WebPack provides a complete solution with little configuration

• The code segment

Code splitting is A very necessary operation. In multi-page applications, pages A,B and C may depend on A large number of third-party libraries at the same time. Extracting the common libraries is convenient for the browser to cache and can effectively reduce the volume of pages A,B and C

One-page application code should also do break up, will be the third party libraries extracted, on the one hand, we usually need to constantly iterative part are generally business code, a third-party library code is there will be no change, the extract also conducive to the browser cache, js is single-threaded, on the other hand, means that the volume of the package is too big to download slow, lead to js thread is hung

module.exports = {
  ...
  optimization: {
    splitChunks: {
      cacheGroups: {
        // Extract the third-party library in node_modules
        vendors: {
          test: /[\\/]node_modules[\\/]/.name: "vendors".chunks: "all"
        },
        commons: {
            name: "commons".chunks: "initial".minChunks: 2
        }
      }
    }
  }
}
Copy the code

• Shake the tree (tree shaking)

Tree shaking uses the static features of export and import to remove unnecessary code, such as in code:

import { forEach } from 'lodash-es'
Copy the code

At the end of the packaging process, WebPack only packages forEach methods in Lodash-es. Other useless code is not packaged. The configuration of tree shaking in WebPack is very simple as follows:

module.exports = {
  mode: 'production'
}
Copy the code

In the Babel configuration you need:

module.exports = {
  presets: [['env'.// Start tree shaking
      {
        modules: false}].'stage-2']... }Copy the code

Add-on: The concept of tree shaking basically refers to the analogy of our code as a tree, shaking down useless code (yellow leaves), stepping on a hole here, and filling up later

• Lazy loading

In order to improve the first screen time, a lot of code can be lazy loaded, which is very convenient in webPack packaged code

1 / / method
import('./someLazyloadCode').then(_= >{... })// Method 2, which is called magic annotation, can be used to name the resulting file lazyload to help us analyze the packaged code
import(/* webpackChunkName: "lazyload" */ './someLazyloadCode').then(_= >{... })Copy the code

Lazy Loading in vue using Webpack’s Code Splitting is also convenient

Note that using lazy loading requires adding a Promise shim, because even on mobile, some older browsers still don’t support Promises. You can use ES6-Promise or Promise-polyfill

In an interview with Webpack author Tobias, when asked if he could recommend any webPack best practices? The authors answer: Use load on demand. Very simple, very effective.

• A hash

Browsers have caches. How do I get the browser to reload resources after code changes?

Traditional practice is behind all resources links and timestamp, but the disadvantages of doing so is just update a file, other did not change the file will be reloaded by timestamp updates, does not favor the browser cache, now the industry is more mature approach to the filename and hash stamp, hash stamp is the one-to-one mapping file content, code changes, The hash stamp will also change, so will the hash stamp of a file whose content has not changed

module.exports = {
  output: {
    filename: isDev ? '[name].js' : '[name].[chunkhash:4].js'. },plugins: [
    new Webpack.NamedModulesPlugin(),
  ]
}
Copy the code

Qd-cli legacy issue, CSS hash is the same as JS, is not conducive to browser cache

• The image processing

Sprite image on mobile terminal will have a decimal point in width and height, so it is not easy to deal with. (If you have a good plan, welcome to provide). Images that are too small can be converted to base64 format for inline files. In addition, image-webpack-loader can be used to compress images as follows:

module.exports = {
  module: {
    rule:
    {
      test: /\.(png|svga? |jpg|gif)$/.use: [{loader: 'url-loader'.options: {
            limit: 8192.fallback: 'file-loader'} } ].concat(isDev ? [] : [{loader: 'image-webpack-loader'.options: {
            pngquant: {
              speed: 4.quality: '75-90'
            },
            optipng: {
              optimizationLevel: 7
            },
            mozjpeg: {
              quality: 70.progressive: true
            },
            gifsicle: {
              interlaced: false}}}])}}}Copy the code

• The CSS code is pulled away

CSS extraction can reduce the size of the page entry and also facilitate CSS caching. Use the official recommended mini-CSS-extract-Plugin

const MiniCssExtractPlugin = require("mini-css-extract-plugin");

module.exports = {
  module: {
    rules: [{test: /\.scss$/.use: [
          isDev ? 'vue-style-loader' : MiniCssExtractPlugin.loader,
          'css-loader',
          {
            loader: 'postcss-loader'.options: {
              config: {
                path: ownDir('lib/config/postcss.config.js')}}},'sass-loader'}]}}Copy the code

• Resource prefetching and resource preloading

Webpack4.6 + supports prefetch and preload, which are not explained here due to unsuccessful attempts. Please refer to code-splitting for details

Improve the packaging efficiency of Webpack

Happypack is used to start multiple processes to accelerate the packaging of Webpack. The code is as follows:

const os = require('os')
const HappyPack = require('happypack')
const happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length })

module.exports = {
  plugins: [
    new HappyPack({
      id: 'eslint'.verbose: false.loaders: [...]. .threadPool: happyThreadPool
    })
  ]
}

Copy the code

There are many articles in the community that suggest using DDL packaging to speed up webpack packaging, you can see: Solve the problem of Webpack packaging performance completely, because the concept is not very well understood, I will not do integration

Code directory structure planning (how to support multi-page applications)

To further scaffold your project, have a directory structure in place so that you have a direction to work with

+ vue-project + src-index. js index.art // Each xxx.art corresponding to the SRC directory xxx.js, to develop multi-page applications only need to add these two files mock. Mock service config file config.js // Must: config fileCopy the code

I use art-template as the template tool. I use art-template purely because I am familiar with it. It is also ok to use other templates. The plugins attribute has the corresponding HTMl-webpack-plugin, and the code is as follows:

const glob = require('globa')

const entry = {}
const htmlPlugins = []

glob.sync(cwd('./src/*.@(js|jsx)')).forEach((filePath) => {
  const name = path.basename(filePath, path.extname(filePath))
  const artPath = cwd(`${name}.art`)
  if (fs.existsSync(artPath)) {
    htmlPlugins.push(new HtmlWebpackPlugin({
      filename: `${name}.html`, template: artPath })) } entry[name] = filePath }) module.exports = { entry, plugins: [...] .concat(htmlPlugins) }Copy the code

Mobile terminal adaptation scheme

At present, only the mobile terminal project is considered. Speaking of the mobile terminal, the first thing to consider is the adaptation scheme. Here, VW layout scheme recommended by The God of Desert is selected

Technical selection – VUE, ES6

Because I am familiar with VUE, VUE is used here. In fact, to support React, you only need to make a little change to react stack. Use VUE – Loader, refer to the document, support PUG syntax, stylus, SCSS, the document is very detailed, there are too many configuration items not attached here. Interested can directly look at the source: QD – CLI

Es6 supports async,await, and decorator syntaxes, both of which are useful and have been described in many articles in the community

module.exports = {
  presets: [['env'.// Start tree shaking
      {
        modules: false}].'stage-2'].plugins: [
    'transform-runtime'.// async await
    'transform-decorators-legacy' / / a decorator]}Copy the code

Code specification

Using the looser Standard specification, here is the CONFIGURATION file for ESLint

{
  extends: [
    'standard'.'plugin:vue/essential'].rules: {
    'no-unused-vars': 1.// When importing unused modules, a warning will be sent instead of interrupting compilation. I am particularly annoyed with no-unused-vars, especially for debug --!
    'no-new': 0             // New is allowed
  },
  // EsLint will report an error with async await if lazy loading is not added
  parserOptions: {
    parser: 'babel-eslint'.ecmaVersion: 2017.sourceType: 'module'
  },
  plugins: [
    'vue']}Copy the code

Mock data support

Mock data is very meaningful. After the interface is set with the backend, the front end can generate fake data to write display logic through the mock server. Here, it uses its own wheel easy-config-Mock, which is easy to inherit from the existing scaffolding, supports the automatic restart of the Mock service, supports the mock JS library mock data format. Support for writing data return logic using custom middleware

const EasyConfigMock = require('easy-config-mock');

new EasyConfigMock({
  path: cwd('mock.config.js')})Copy the code

Mock.config.js is the following demo:

// mock.config.js
module.exports = {
  // The common option is not required, you can do without it, the built-in configuration is as follows, of course you can change it
  common: {
    // The mock service's default port is automatically replaced if the port is occupied
    port: 8018.// If you want to see the loading effect of Ajax, this configuration item can set the return delay of the interface
    timeout: 500.// If you want to see the effect of interface request failure, set the rate to 0. The rate ranges from 0 to 1, representing the probability of success
    rate: 1.// The default is true to enable mock services automatically, but you can also disable mock services by setting it to false
    mock: true
  },
  // Common API...
  '/pkApi/getList': {
    code: 0.'data|5': [{
      'uid|1000-99999': 999.'name': '@cname'}].result: true
  },
  // Middleware API (standard Express middleware), where you can write interface return logic
  ['/pkApi/getOther'] (req, res, next) {
    const id = req.query.id
    req.myData = {   / / important! Mount the returned data on req.myData
      0: {
        code: 0.'test|1-100': 100
      },
      1: {
        code: 1.'number|+1': 202
      },
      2: {code: 2.'name': '@cname'
      }
    }[id]
    next()  // Finally, don't forget to call next manually, otherwise the interface will pause processing!}}Copy the code

Here’s how it works: Build a mock service from scratch

Project set Support

The structure of the project set can be as follows:

+ vue-projects - project1 - project2 + project3 + src index.js ... Index.art config.js // project configuration mock.config.js // Project mock service readme.md // Project description document... - web_modules // Public module of the project set config.js // Project set configuration readme. md // Project set description documentCopy the code

Each small project has its own config.js configuration file and readme.md documentation. Each project set also has its own config.js configuration file and readme.md documentation. The configuration in the small project configuration file can override the configuration of the project set. There is also the webpack_modules directory, which stores public modules that each project can use. The advantage of this is that projects of the same type can be thrown together and modules with the same dependencies can be thrown into web_modules. When the web_modules file changes and needs to be distributed, Subsequent continuous integration can be unified processing, one-click all release

The code to generate the final configuration file is as follows:

const R = require('ramda')

const cwd = file= > path.resolve(file || ' ')
const generateConfig = path= > {
  const cfg = require(cwd(path))
  if (typeof cfg === 'function') {
    return cfg({})
  } else {
    return cfg
  }
}

module.exports = {
  getConfig: R.memoize(_= > {
    let config = {}
    // If it is a project set, the project set will also have a config.js
    if (fs.existsSync('.. /config.js')) {
      config = R.merge(config, generateConfig('.. /config'))
    }
    config = R.merge(config, generateConfig('config.js'))
    return config
  })
}
Copy the code

Configuration Item Support

Currently, only the following configuration items are supported

// config.js
module.exports = {
  // The standard Webpack4 configuration can override the default configuration
  webpack: {},

  // The default boot port is 8018, which can be switched
  port: 8017.// The default design width is 750, which can be changed here
  viewportWidth: 750.viewportHeight: 1334.// The production environment sourcemap is fixed using 'source-map'. The development environment can be set using devtool
  devtool: 'inline-source-map'.// webpack-dev-server proxy Settings
  proxy: {},

  // EsLint's rule, because of my own habit, sets 'no-unused-vars' to 1, which can be modified by default
  rules: {},

  // PostCSS plug-in, if customized, also need to install the corresponding node module
  postcssPlugin: {},

  //.eslintrc configuration item that can be overridden
  eslintConfig: {},

  Transform-runtime and transform-decorators-legacy are already available by default
  babelPlugins: [],

  // Babel preset, env and stage-2 are already available by default
  babelPresets: []
}

Copy the code

Cli support

This is enough. Next, we need to integrate the workflow built with WebPack into CLI. The advantages of this are that first, we can develop and build through the command line. Because of qd – cli built-in vue, vuex, vue – the router, axios, json, ramda, jquery modules, such as no secondary installation, reduce the size of the project, brief description and integration into the cli is done some attention

• Use COMMANDER to build cli, you can directly read qD-CLI source code, the main code in the bin and lib/command directory, you can also refer to node.js based scaffolding tool development experience

• The webpack configuration item resolve.modules stands for when require a file to retrieve from these directories. The qD-CLI configuration item is as follows

const cwd = p= > path.resolve(__dirname, p)
const ownDir = p= > path.join(__dirname, p)

module.exports = {
  resolve: {
    modules: [cwd(), cwd('node_modules'), ownDir('node_modules'), cwd('.. /web_modules')]}Copy the code

Such as: If require(‘jquery’) is not found in the current project directory, it will go to node_modules in the current directory, If you haven’t found it yet, go to node_modules in the scaffolding directory and web_modules in the previous directory (project set support). Since jquery is installed in the scaffolding, the project itself doesn’t need to be installed

• Configuration items for WebPackresolveLoaderOption, the configuration is as follows:

  resolveLoader: {
    modules: [cwd('node_modules'), ownDir('node_modules')]},Copy the code

Webpack can’t find the loader. Add node_modules from the scaffolding directory to the loader path list

• The scaffold package.json is requiredbinfield

Specifies the location of the executable corresponding to the qd command

"bin": {
  "qd": "./bin/cli.js"   // Indicates the cli execution file
}
Copy the code

• ./bin/cli.jsTop line

#! /usr/bin/env node
Copy the code

What program is used to launch the script, we use Node

Writing a seed file

qd-vue-seed

Publish to the NPM community

See how to publish a custom Node.js module to NPM.

Since the NPM community does not register the name qD-CLI (there are already warehouses with similar names), I changed it to qD-clis 😂

---

Qd – CLI installation and use

The installation

npm i qd-clis -g
or
yarn global add qd-clis
Copy the code

For Windows, use administrator permission. For MAC, add sudo before the command

If you don’t want to install globally, pull into a local directory and view the source code, you can :(again as an administrator)

git clone [email protected]:nwa2018/qd-cli.git
cd qd-cli
npm i / yarn
npm link
Copy the code

use

After the installation, enter qd command to see the introduction of all commands, as shown in the following figure

As shown above, QD-CLI has the most basic generation seed project, development and build three functions

features

• Qd – cli with built-in vue, vuex, vue – the router, axios, json, ramda, jquery, without secondary installation
• Support ES6 syntax, support async,await, support decorator
• Eslint uses the standard specification
• Support puG syntax, stylus, SCSS
• The production environment supports automatic image compression
• Support single page application, multi-page application, support project set structure
• Supports a small number of configuration items
• Support for Mock services
• Production environment supports compression, code splitting, lazy loading, hashing…

---

Some potholes I’ve stepped on

Combined with vue-loader, mini-CSS-extract-Plugin can not extract CSS, CSS was inexplicably deleted

The tree-shaking section of the Webpack Guide is recommended for package.json

  "sideEffects": [
    "*.css"
  ]
Copy the code

The solution is to remove this option to avoid CSS files being accidentally deleted, which can be done with vue-loader

Windows failed to startwebpackwithwebpack-dev-serverThe command

I use ShellJS to start the action of packing and starting the server. The code is as follows

// build.js...
shell.exec(`${ownDir('node_modules/webpack/bin/webpack.js')} --config ${ownDir('lib/webpack/webpack.prod.js')} --progress --report`)

// dev.js...
shell.exec(`${ownDir('node_modules/webpack-dev-server/bin/webpack-dev-server.js')} --config ${ownDir('webpack/webpack.dev.js')} --color`)
Copy the code

This is fine on MAC, Windows opens webpack.dev.js and webpack.prod.js directly on my sublime. The window system does not know which program to start the file

// build.js...
shell.exec(`node ${ownDir('node_modules/webpack/bin/webpack.js')} --config ${ownDir('lib/webpack/webpack.prod.js')} --progress --report`)

// dev.js...
shell.exec(`node ${ownDir('node_modules/webpack-dev-server/bin/webpack-dev-server.js')} --config ${ownDir('webpack/webpack.dev.js')} --color`)
Copy the code

Eslint cannot parse import() and async await correctly

Parsing error with import() #7764 with ‘Parsing error: Unexpected token function’ using async/await + ecmaVersion 2017 #8366

At first I thought it was Babel and spent a lot of time trying to locate it. Add the following configuration and install babel-eslint to.eslintrc

  parserOptions: {
    parser: 'babel-eslint'.ecmaVersion: 2017.sourceType: 'module'
  }
Copy the code

I can’t find babel-core. Babelrc

To add the following configuration to.babelrc, I changed it to babel.js and threw it in the webpack/config/ directory along with postCSS and esLint configurations. In fact, babel.js is what we normally write

{
  // Enter the Babel configuration path
  filename: ownDir('lib/webpack/config/babel.js'),}Copy the code

---

Refer to the link

• Vip Mobile Cli Tutorial
• vue-cli
• webpack-guide
• How to publish a custom Node.js module to NPM
• Node.js based scaffolding tool development experience
• How to use VW for mobile adaptation in a Vue project

---

Github address, such a long article are finished, through the passing handsome boy beauty, a praise 😂😂

In this paper, to the end.