Write a lightweight JS packaging tool

Throw out problem

Now we have the following four files 👇 :

// index.js
let text = require('./text.js').text
let aa = require('./a.js').aa

let str = `${text} I'm ${aa} years old.`
console.log(str)

// text.js
let text = 'hello, water! '
exports.text = text

// a.js
let bb = require('./b.js').bb
exports.aa = 2 * bb

// b.js
exports.bb = 14
Copy the code

When we run it with Nodeindex.jsYou get something like thishello, water! I'm 28 years old.. But when we introduce that directly in HTMLindex.jsFile, the console will report an error like this:That’s pretty obvious, the browser is already telling us that there’s no definitionrequireThis function, so it can’t be executed. In Node, there isrequireThis built-in function is node’s modularity mechanism. So the question is, how do I get that code to run in the browser? As a project gets bigger and bigger, file splitting is essential, and how do we integrate this code?

👉 the answer is packaging tools, of course, there are a lot of ready-made packaging tools, most of our front-end is known as webpack, but this thing is cumbersome, understanding and familiar with the threshold, and easy to forget. By chance, I recently saw a really good packaging idea (really good 👍), so hereby precipitation shares ✍, I hope it can help you.

So, let’s first clarify the goal of this article here: is to write a lightweight JS packaging tool, the above multiple JS files into a final packagebundle.jsThe file makes thisbundle.jsFile can run in the browser, and does not depend on other packages, also does not involve any arcane AST, of course, also not to write a short version of the webpack, here only packaged js file, so it is not difficult to understand, please rest assured to eat 🍖.

thinking

Think about 1

What is the crudest way to package multiple files into a single file? Is directly copied, kao under the same file is ok, obviously this doesn’t work, after all, the require and exports is what things I don’t know… So, we can do this by using every single js file as a function, so that require and exports can be passed in via arguments. Take a look at the following example code to understand 👇 :

// bundle.js
function (require.exports) { // index.js
  let text = require('./text.js').text
  let aa = require('./a.js').aa

  let str = `${text} I'm ${aa} years old.`
  console.log(str)
}

function (require.exports) { // text.js
  let text = 'hello, water! '
  exports.text = text
}

function (require.exports) { // a.js
  let bb = require('./b.js').bb
  exports.aa = 2 * bb
}

function (require.exports) { // b.js
  exports.bb = 14
}
Copy the code

We don’t know what require and exports are yet, but at least now every JS file is a function, so we can see the implementation possibilities 😬.

Thinking about 2

Don’t know you found no, thinking about the code above there is a problem 🤔, function name, because we pack will eventually generate a js file, if the source file is much, also is the function is much, then how should we distinguish between the function, we can’t name it 🚫 one by one, if many modules, name and conflict are problems, So we need to effectively distinguish these modules, simple point is to give an ID, specific how to give how corresponding, see the following code to understand 👇 :

// bundle.js
let modules = {
  0: function(require.exports) { // index.js
    let text = require('./text.js').text
    let aa = require('./a.js').aa

    let str = `${text} I'm ${aa} years old.`
    console.log(str)
  },
  1: function(require.exports) { // text.js
    let text = 'hello, water! '
    exports.text = text
  },
  2: function(require.exports) { // a.js
    let bb = require('./b.js').bb
    exports.aa = 2 * bb
  },
  3: function(require.exports) { // b.js
    exports.bb = 14}}Copy the code

Is it suddenly refreshing a lot 😁…

Thinking about 3

Now that we have the mapping for each function, you can see that Modules [0] is our entry function, and our final expectation is to execute that entry function, so we’ll start with a simple implementation function (see the comments) 👇 :

// bundle.js
function handle(id) {
  // Select a function based on its id (file or module)
  let fn = modules[id]
  
  // Exports are called require and exports
  let exports = {} // Is an empty object to store the values exported by each module
  function require(path) { // path is the file path
    // Regardless of what is inside, but you need to know its function
    // Find the file in the path + execute the file (i.e. function) + return the result of the function
  }
  
  // Execute this function
  fn(require.exports)
  
  // Exports the exports object, which contains the exported value
  return exports
}
handle(0) // The file id is passed in, which starts with the entry file
Copy the code

Where the code above might confuse you is the declarations of require and exports. I also read some articles, speak is not very easy to understand, is indeed it itself is not good description, so here, I will in my way to you explain it, of course, I do not know I speak good understand 😂 :

• First we know that a function is a module (a file)
• Since modules are modules that reference each other (that is, they have dependencies), modules naturally have two things: the dependencies they need and the values they return
• requireThe dependencies that the module needs,exportsThe payload is the module’s return value (think about it) 🤔
• requireThe essence is a function, the function is based on the path to find the corresponding dependency (function) and execute
• exportsIs an empty object, but the module exports things to mount toexportsOn this object, just like we normally export and writeexports.xxx = 10And then you can get itxxxThe value of the

Take our a. JS file above for example, the general process looks like the following:

// a.js
let bb = require('./b.js').bb Exports = {bb: 14} // go to require and execute b. Exports = {bb: 14}
exports.aa = 2 * bb; // Exports is an empty object of exports where aa is mounted
Copy the code

If you don’t understand, just read it again 😂…

Thinking about 4

Now we can already find the corresponding module by id, but the input parameter of require is path. Normally, when we write require, we need to find the corresponding module according to the path. Now, it seems a bit twisted for us to find the corresponding module by path. So we need to make some small changes to the modules structure. Let’s take a look at the result 👇 :

// bundle.js
let modules = { // The first element of the array is the same function, and the second element is a mapping between path and ID, so that we can find the corresponding module through path to ID
  0: [function(require.exports) { // index.js
      let text = require('./text.js').text
      let aa = require('./a.js').aa

      let str = `${text} I'm ${aa} years old.`
      console.log(str)
    }, {
      './text.js': 1.'./a.js': 2}].1: [function(require.exports) { // text.js
      let text = 'hello, water! '
      exports.text = text
    }, {}],
  2: [function(require.exports) { // a.js
      let bb = require('./b.js').bb
      exports.aa = 2 * bb
    }, {
      './b.js': 3,}],3: [function(require.exports) { // b.js
      exports.bb = 14}}, {}]Copy the code

The above code should be fairly straightforward, so our Handle should also do something like this:

// bundle.js
function handle(id) {
  let [fn, mapping] = modules[id] // This is a deconstructed assignment, which takes the first and second items of the array
  let exports = {}

  function require(path) { // require finds the corresponding module and executes to get the return value
    // Mapping [path] is the ID of the corresponding module
    return handle(mapping[path]) // Is the return exports object, so we can get the exported value
  }

  fn(require.exports) // Take the corresponding dependency and execute it

  return exports
}
handle(0)
Copy the code

In fact, the above code and the original compared to more from the path to the ID to find the module process, frommappingThat’s what this variable is. And now it’s time to see the magic 😏, we take this code to the browser to execute, surprised to find it successfully print 👏 :Oh, It ‘s opponents! 💯 but our business is not over, at present ourmodulesThe modules are written to death, so what we really need to do next is to convert the source file automatically to the above code, so that it can run in the browser. So now our goal is to transform, and this diagram should be a little easier to understand:

Start writing

So, how do you generate the bundle.js file? Here you can think about it for a few seconds and then go down 🤔… Of course, I have to rely on a little bit of node knowledge, if you don’t know node, I wrote a good article (Portal: node knowledge for understanding front-end tools), welcome to like it. However, do not look at the fact that there is no relationship, do not affect understanding 😬. 👌, the idea behind generating bundle.js is pretty much the same. Basically, you read the import file, do dependency analysis, and then try to piece together a bunch of strings that look like bundle.js, and then write them to bundle.js. Now let’s get started (the file we wrote is called wp.js).

Read the file

This step is very simple. We simply use node’s fs module to read and write the entry file. The API returns a string of the file contents, as shown in 👇 :

// wp.js
const fs = require('fs') // This is node's API for reading files
let fileStr = fs.readFileSync('./src/index.js'.'utf8') // the js file reads as a string
typeof fileStr // string
// "let text = require('./text.js').text\nlet aa = require('./a.js').aa\n\nlet str = `${text} I'm ${aa} years old.`\nconsole.log(str)\n"
Copy the code

Analysis rely on

Because the entry file depends on several other files, we need to know what file we depend on so we can continue parsing. So how do you analyze the dependency of the entry file? Similarly, here we can also think for a few seconds 🤔… The proper way to write dependency analysis is to use an off-the-shelf tool (such as Babel) to parse the string and generate an AST syntax tree, which is then retrieved from the AST, but that would easily dissuade a bunch of people, so it’s not done here. Instead, we take the opportunistic approach of using regular expressions to match fields like require(‘./ XXX /xxx.js’) and then strip paths from them, like ‘./ XXX /xxx.js’. In fact, this test is the re, can be written in various ways, here is just one of them, see the following code:

// wp.js
function getDependencies(str) { // The STR is the string read from the entry file
  let rs = str.match(/require\('(.+)'\)/g) // [ "require('./text.js')", "require('./a.js')" ]
  rs = rs ? rs.map(r= > r.slice(9, -2)) : [] // [ "./text.js", "./a.js" ]
  return rs
}
Copy the code

Process files into objects

Now that we know this dependency, we don’t seem to have a clue what to do. Indeed, let’s go back and think about it, and ultimately we’re going to produce something like this:thismodulesInside id, name of the file (that is, path), the file content, file corresponding to the dependence and mapping, etc., that if we make every file (module), an object with object way of describing a file, so that we follow-up to deal with each file (module), will be more convenient. Like… Right… That words do not say, directly on the code 👇 :

// wp.js
let ID = 0; / / on the ID
function createAsset(filename) { // filename looks like this: './ SRC /index.js'
  let fileStr = fs.readFileSync(filename, 'utf8') // the js file reads as a string
  return { // Describe a file as an object
    id: ID++,
    filename, // './src/index.js'
    dependencies: getDependencies(fileStr), // [ "./text.js", "./a.js" ]
    code: `function(require, exports) {
      ${fileStr}} `}}Copy the code

Consolidate all files

We now have a code that generates each file objectcreateAssetMethod to place all files in an array as objects. Why so, patience to look down to know! Now we want to generate something like 👇 :Zha generated? This is not difficult, so we directly on the code, full of comments 👇, please rest assured to eat 🍗 :

// wp.js
const path = require('path') // The module in node that handles paths
function createAssetArr(filename) {
  let entryModule = createAsset(filename)
  let moduleArr = [entryModule] // This is used to store all modules, i.e. all files

  for(let m of moduleArr) { // moduleArr currently has only one entry module, but it will continue to append modules to moduleArr when resolving dependencies, so it will continue to loop backwards instead of loop once
    let dirname = path.dirname(m.filename)
    m.mapping = {} // This is the dependency mapping
    m.dependencies.forEach(relativePath= > {
      let absolutePath = path.join(dirname, relativePath)
      let childAsset = createAsset(absolutePath) // Use absolute path, relative path is easy to find, we should have experienced this during development
      m.mapping[relativePath] = childAsset.id // Store dependency mappings
      moduleArr.push(childAsset) // Add more modules to moduleArr to continue the loop})}return moduleArr // Returns an array of all modules
}

let moduleArr = createAssetArr('./src/index.js')
console.log(moduleArr) // This is what the image above printed
Copy the code

The above code does not understand words, more than glance at two eyes, is the integration of resources into an array, is not difficult to understand 😂.

Output package file

In fact, the previous steps were all files, and now we are just one step away from generating bundle.js. Hang in there 👊. If you understand the previous steps, you can simply concatenate a single string from the bundle.js file, like 👇 :

// wp.js
function createBundleJs(moduleArr) {
  let moduleStr = ' '
  moduleArr.forEach((m, i) = > { // Concatenate the contents of modules
    moduleStr += `${ m.id }: [${ m.code }.The ${JSON.stringify(m.mapping) }], `
  })
  let output = `let modules = { ${moduleStr} }
  function handle(id) {
    let [fn, mapping] = modules[id]
    let exports = {}
    function require(path) {
      return handle(mapping[path])
    }
    fn(require, exports)
    return exports
  }
  handle(0)`
  fs.writeFileSync('./bundle.js', output) // write to the bundle.js file in the current path
}
Copy the code

And finally, we usenode wp.jsExecute the code above to see what is generatedbundle.jsFile:Well, I think we can 👏… Although a little ugly, but this is not important, after all, this is packaged after the file, not for you to see, is for the browser to see, and you can also hide the alarm 😁, simple to the next space, as a compressed file.

Alternatively, you can concatenate strings as functions are executed to make them more modular, like this:

// wp.js
(function (modules) { // Here's another way to write bundle.js, which is to wrap the immediate function around modules, so that it doesn't affect the whole world
  function handle(id) {
    let [fn, mapping] = modules[id]
    let exports = {}
    function require(path) {
      return handle(mapping[path])
    }
    fn(require.exports)
    return exports
  }
  handle(0) ({})// This is modules, passed in as an argument
  0: [function(require.exports) {
    // index.js
  }, {
    './text.js': 1.'./a.js': 2}].1: [function(require.exports) {
    // text.js}, {},2: [function(require.exports) {
    // a.js
  }, {
    './b.js': 3,}],3: [function(require.exports) {
    // b.js}, {}})Copy the code

conclusion

Here to briefly summarize the train of thought, this paper is make each js file is actually a module, which is an object, the object mounted above a few files relevant properties for our subsequent operations, and then we got an array of all modules, can be pieced together from joining together into the output file, probably is such a process. Of course, this is an immature example, and there are definitely problems, such as reparsing dependencies, that you can try to optimize with caching. At this point, we have finally completed a lightweight JS packaging tool, sparrow is small, but the idea in place, I believe that after you look at the other webpack source tutorial and such things should be able to slightly ease some familiar road (should… 😂…) For example, if you change the require name to __webpack_require__, you can see what webpack looks like, hahaha 😁. The last of the last, feel the cover good-looking words point a praise, see you later 👋. Ps: project address portal