ES2017+ eliminates the need for complex build tool technology selection.

Also no longer need to gulp, grunt, yeoman, metalsmith and fis3.

All of these building tools can be crossed off in your mind forever.

In 100 lines of code, you will see through the essence of the build tool.

100 lines of code, and you’ll have a modern, standardized, test-driven, highly extensible front-end build tool.

Before we read, a little cliffhanger:

  • What are chain operations, middleware mechanisms?
  • How to read and build a file tree?
  • How to achieve batch template rendering, code translation?
  • How to realize data sharing among middleware.

I believe that after learning this class, you will find ———— these professional terms, the principle behind it is… That’s easy!

Build tool experience: popup + Uglify + template engine + Babel transcoding…

To experience its power immediately, type NPX mofast Example from the command line to create a mofast-example folder.

After entering the file, run node compile to experience the function.

By the way, the NPX mofast Example command line itself is implemented using this tutorial’s build tool. Isn’t it incredible?

This course code has been published on NPM and can be installed directly

NPM mofast – D can be used in any project I mofast, alternative gulp of grunt/yeoman metalsmith/fis3 to install and use.

The github address for this course is github.com/wanthering…. At the end of the course, you can submit PR and maintain the library together to make it more and more extensible!

Step 1: Build the Github/NPM standard development stack

Please set up the following environment:

  • Jest test environment
  • The ESLint format standardized environment
  • Babel ES2017 code environment

Or use NPX lunz mofast

And then all the way back to the car.

The constructed file system is as follows

├─..Bass Exercise.Bass Exercise.bass Exercise.bass Exercise.bass Exercise.bass Exercise.bass Exercise.bass Exercise.Bass Exercise.Bass Exercise.Bass Exercise.Bass Exercise.Bass Exercise.Bass Exercise.Bass Exercise.Package Bass Exercise.src │ └ ─ ─ index. Js ├ ─ ─ the test │ └ ─ ─ but the spec. Js └ ─ ─ yarn. The lockCopy the code

Step 2: Set up the sandbox environment

Build tools, all need to operate on file systems.

During testing, it is common to pollute the local file system, resulting in the accidental loss and modification of some important files.

So we tend to create a sandbox environment for testing.

In the package.json sibling directory, enter the command

 mkdir __mocks__ && touch __mocks__/fs.js
 
 yarn add memfs -D
 yarn add fs-extraCopy the code

After creating the __mocks__/fs.js file, write:

const { fs } = require('memfs')
module.exports = fsCopy the code

Then write in the first line of the test file index.spec.js:

jest.mock('fs')
import fs from 'fs-extra'Copy the code

To explain: the files in __mocks__ will be automatically loaded into the test’s mock environment, and with jest. Mock (‘fs’), the fs operation will be overwritten and the entire test will be run in the sandbox environment.

Step 3: Basic configuration of a class

src/index.js

import { EventEmitter } from 'events'

class Mofast extends EventEmitter {
  constructor () {
    super()
    this.files = {}
    this.meta = {}
  }

  source (patterns, { baseDir = '.', dotFiles = true } = {}) {
    // TODO: parse the source files
  }

  async dest (dest, { baseDir = '.', clean = false } = {}) {
    // TODO: conduct to dest
  }
}

const mofast = () => new Mofast()

export default mofastCopy the code

EventEmitter is used as a parent class because emit events are needed to monitor the motion of the file stream.

Use this.files to save the file chain.

Use this.meta to save data.

So inside there are the source methods and dest methods. The usage method is as follows:

test/index.spec.js

import fs from 'fs-extra' import mofast from '.. Mock ('fs')/SRC 'import path from "path" jest. Mock ('fs') const templateDir = path.join(__dirname, 'fixture/templates') fs.ensureDirSync(templateDir) fs.writeFileSync(path.join(templateDir, 'add.js'), `const add = (a, b) => a + b`) test('main', async ()=>{ await mofast() .source('**', {baseDir: templateDir}) .dest('./output', {baseDir: __dirname}) const fileOutput = fs.readFileSync(path.resolve(__dirname, 'output/tmp.js'), 'utf-8') expect(fileOutput).toBe(`const add = (a, b) => a + b`) })Copy the code

Now, with the goal of passing the test, we complete the preliminary compilation of the Mofast class.

Step 4: Gulp-like, chain file flow operation.

The source function:

Mount the patterns, baseDir, and dotFiles arguments to this and return this to facilitate chain operations.

Dest function:

Dest is an asynchronous function.

It does two things:

  1. Read all files from the source folder and assign them to the this.files object.
  2. Write the files in the this.files object to the location of the destination folder.

These two operations can be separated into two asynchronous functions: process() and writeFileTree().

The process function

  1. Using the fast-glob package, read the status stats of all files under the target folder, return an array composed of file status stats
  2. Fs.readfile () is used to read the content in the absolute path.
  3. Mount Content, STATS, and Path to this.files.

Note that because it is batch processing, you need to execute promise.all () simultaneously.

Suppose the /fixture/template/add.js file is const add = (a, b) => a + b

This. File object after processing:

{ 'add.js': { content: 'const add = (a, b) => a + b', stats: {... }, path: '/fixture/template/add.js' } }Copy the code

WriteFileTree function

After iterating through this.file and using fs.ensuredir to ensure the folder exists, write this.file[filename].content to the absolute path.

import { EventEmitter } from 'events' import glob from 'fast-glob' import path from 'path' import fs from 'fs-extra' Class Mofast extends EventEmitter {constructor () {super() this.files = {} this.meta = {}} /** * Attach parameters to this * @param * @param baseDir source file root * @param dotFiles identifies hidden files * @returns this returns this, Source (Patterns, {baseDir = '.', dotFiles = true } = {}) { // this.sourcePatterns = patterns this.baseDir = baseDir this.dotFiles = dotFiles return this } /** * The contents, state, and absolute path of files in baseDir, */ async process () {const allStats = await glob(this.sourcepatterns, {CWD: this.basedir, dot: this.dotFiles, stats: true }) this.files = {} await Promise.all( allStats.map(stats => { const absolutePath = path.resolve(this.baseDir, stats.path) return fs.readFile(absolutePath).then(contents => { this.files[stats.path] = { contents, stats, path: AbsolutePath}})}) return this} @param destPath {/ async writeFileTree(destPath){ await Promise.all( Object.keys(this.files).map(filename => { const { contents } = this.files[filename] const target = path.join(destPath, filename) this.emit('write', filename, target) return fs.ensureDir(path.dirname(target)) .then(() => fs.writeFile(target, Contents))}))} /** ** @param dest target folder * @param baseDir target file root directory * @param clean Whether to empty target folder */ async dest (dest, { baseDir = '.', clean = false } = {}) { const destPath = path.resolve(baseDir, dest) await this.process() if(clean){ await fs.remove(destPath) } await this.writeFileTree(destPath) return this } } const mofast = () => new Mofast() export default mofastCopy the code

Run yarn test to test the yarn test.

Step 5: Middleware mechanisms

Let’s say the class we’re writing is a gun.

Middleware, then, is bullet after bullet.

You need to push the bullets into the gun one by one and then fire them all at once.

Write a test case that changes const add = (a, b) => a + b in the add.js file to var add = (a, b) => a + b

test/index.spec.js

test('middleware', async () => {
  const stream = mofast()
    .source('**', { baseDir: templateDir })
    .use(({ files }) => {
      const contents = files['add.js'].contents.toString()
      files['add.js'].contents = Buffer.from(contents.replace(`const`, `var`))
    })

  await stream.process()
  expect(stream.fileContents('add.js')).toMatch(`var add = (a, b) => a + b`)
})Copy the code

Ok, now let’s implement middleware

Initialize the Constructor array in Constructor

src/index.js > constructor

  constructor () {
    super()
    this.files = {}
    this.middlewares = []
  }Copy the code

Create a use function to push middleware into an array, like bullets into a magazine.

src/index.js > constructor

  use(middleware){
    this.middlewares.push(middleware)
    return this
  }Copy the code

In the process asynchronous function, the middleware is executed immediately after the file is processed. Note that the middleware argument should be this, so that the parameters such as this.files, this.baseDir, etc. mounted on the main class can be retrieved.

src/index.js > process

async process () {
    const allStats = await glob(this.sourcePatterns, {
      cwd: this.baseDir,
      dot: this.dotFiles,
      stats: true
    })

    this.files = {}
    await Promise.all(
      allStats.map(stats => {
        const absolutePath = path.resolve(this.baseDir, stats.path)
        return fs.readFile(absolutePath).then(contents => {
          this.files[stats.path] = { contents, stats, path: absolutePath }
        })
      })
    )


    for(let middleware of this.middlewares){
      await middleware(this)
    }
    return this
  }Copy the code

Finally, we wrote a new method called fileContents that reads the contents of a file object for testing

  fileContents(relativePath){
    return this.files[relativePath].contents.toString()
  }Copy the code

Run yarn test. The test succeeds.

Step 6: Template engine, Babel translation

Now that you have the middleware mechanism.

We can encapsulate some common middleware, such as ejS/Handlebars template engines

My name is <%= name %> my name is {{name}}

Input jack} {name: ‘

My name is Jack

And Babel translation:

Const add = (a, b) => a + b

Var add = function(a, b){return a + b}

Ok, let’s write the test case:

Fs.writefilesync (path.join(templateDir, 'ejstmp.txt'), `my name is <%= name %>`) fs.writeFileSync(path.join(templateDir, 'hbtmp.hbs'), `my name is {{name}}`) test('ejs engine', async () => { await mofast() .source('**', { baseDir: templateDir }) .engine('ejs', { name: 'jack' }, '*.txt') .dest('./output', { baseDir: __dirname }) const fileOutput = fs.readFileSync(path.resolve(__dirname, 'output/ejstmp.txt'), 'utf-8') expect(fileOutput).toBe(`my name is jack`) }) test('handlebars engine', async () => { await mofast() .source('**', { baseDir: templateDir }) .engine('handlebars', { name: 'jack' }, '*.hbs') .dest('./output', { baseDir: __dirname }) const fileOutput = fs.readFileSync(path.resolve(__dirname, 'output/hbtmp.hbs'), 'utf-8') expect(fileOutput).toBe(`my name is jack`) }) test('babel', async () => { await mofast() .source('**', { baseDir: templateDir }) .babel() .dest('./output', { baseDir: __dirname }) const fileOutput = fs.readFileSync(path.resolve(__dirname, 'output/add.js'), 'utf-8') expect(fileOutput).toBe(`var add = function (a, b) { return a + b; } `)})Copy the code

Engine () takes three arguments

  • Type: specifies the template type
  • Locals: provides input parameters
  • Patterns: Specifies the matching format

Babel () takes one argument

  • Patterns: Specifies the matching format

Engine () implementation principle:

Make sure type is one of EJS and Handlebars via NodeJS Assert

Jstransformer + JstransFormer – EJS and JstransFormer – HandleBars

Determine the type of locals and, if it is a function, pass in the execution context so that files and meta equivalents can be accessed. If it is an object, merge the meta values into it.

Using minimatch, match the filename to see if it matches the given pattern, and if so, process it. If pattern is not entered, all files are processed.

Create a middleware, traverse files in the middleware, take out the contents of a single file for processing, and update to the original location.

Push the middleware into the array

Implementation principle of Babel ()

Make sure type is one of EJS and Handlebars via NodeJS Assert

Conversion code conversion is done through the Buble package (a simplified version of Bable).

Using minimatch, match the filename to see if it matches the given pattern, and if so, process it. If pattern is not entered, all JS and JSX files are processed.

Create a middleware, iterate through files in the middleware, extract the contents of a single file and convert it into ES5 code, then update it to the original location.

Next, install the dependencies

yarn add jstransformer jstransformer-ejs jstransformer-handlebars minimatch buble Copy the code

And introduce it in the head

src/index.js

import assert from 'assert'
import transformer from 'jstransformer'
import minimatch from 'minimatch'
import {transform as babelTransform} from 'buble'Copy the code

Add the engine and bable methods

engine (type, locals, pattern) { const supportedEngines = ['handlebars', 'ejs'] assert(typeof (type) === 'string' && supportedEngines.includes(type), `engine must be value of ${supportedEngines.join(',')}`) const Transform = transformer(require(`jstransformer-${type}`))  const middleware = context => { const files = context.files let templateData if (typeof locals === 'function') { templateData = locals(context) } else if (typeof locals === 'object') { templateData = { ... locals, ... context.meta } } for (let filename in files) { if (pattern && ! minimatch(filename, pattern)) continue const content = files[filename].contents.toString() files[filename].contents = Buffer.from(Transform.render(content, templateData).body) } } this.middlewares.push(middleware) return this } babel (pattern) { pattern = pattern || '*.js?(x)' const middleware = (context) => { const files = context.files for (let filename in files) { if (pattern && ! minimatch(filename, pattern)) continue const content = files[filename].contents.toString() files[filename].contents = Buffer.from(babelTransform(content).code) } } this.middlewares.push(middleware) return this }Copy the code

Step 7: Filter files

Write test cases

test/index.spec.js

test('filter', async () => { const stream = mofast() stream.source('**', { baseDir: templateDir }) .filter(filepath => { return filepath ! == 'hbtmp.hbs' }) await stream.process() expect(stream.fileList).toContain('add.js') expect(stream.fileList).not.toContain('hbtmp.hbs') })Copy the code

– Added a fileList method that retrieves an array of names from this.files.

Again, the filter() method is created by injecting middleware.

src/index.js

filter (fn) { const middleware = ({files}) => { for (let filenames in files) { if (! fn(filenames, files[filenames])) { delete files[filenames] } } } this.middlewares.push(middleware) return this } get fileList () { return Object.keys(this.files).sort() }Copy the code

Run the YARN Test and pass the test

Step 8: Package and deliver

At this point, basically all the functionality of a small build tool has been implemented.

Enter yarn Lint unified file format.

Then enter the YARN Build package file. Dist /index.js is the file used by NPM

Add a main field to package.json pointing to dist/index.js

Add a files field to indicate that the NPM package contains only the dist folder

  "main": "dist/index.js",
  "files": ["dist"],Copy the code

Then use the

npm publishCopy the code

The package can be published on the NPM.

Conclusion:

Ok, to answer the first question:

What is chain operation?

Answer: Return this

What are middleware mechanisms

A: That is, one asynchronous function after another is pushed onto the stack and executed in a final loop.

How to read and build a file tree.

A: File tree is an object whose key is the relative path of the file and value is the content of the file.

All batch fs.readFile fetch the contents of the file and mount them to this.files.

Build the file tree, which is this.files using promise. all batch fs.writeFile to target folder.

How to implement template rendering and code translation?

Ejs.render () or able. Transform () and put it back in place.

How to realize data sharing among middleware?

A: Contructor creates this.meta={}.

The principle behind the front-end build tool is simpler than you might think.