preface

Before, my friend wanted to introduce webAPCK related content, so recently, I spent more than two months to prepare, and finally completed the WebAPCK series, which includes the following parts:

  • Webapck Series 1: Hand write a JavaScript wrapper
  • Webpack Series 2: Best configuration refers to north
  • Webpack Series 3: Optimize 90% packing speed
  • Webpack Series 4: Optimize package volume
  • Webapck Series 5: Optimize first screen loading time and page smoothness
  • Webapck Series six: Build package analysis
  • Webapck Series 7: Detailed configuration
  • Webapck series 8: Write a WebAPck plug-in (simulate HtmlWebpackPlugin implementation)
  • Webapck Series 9: WebAPCK4 core source code interpretation
  • Webapck Series 10: WebAPCK5 Outlook

All the content will be released in succession, if you have any content you want to know or have any questions, follow the public number [front bottle Gentleman] reply [123] add friends, I will answer your questions.

As front-end developers, we spend a lot of time working with packaging tools like WebPack and Gulp, packaging advanced JavaScript projects into more complex and difficult to read packages that run in the browser, so understanding JavaScript packaging is essential. It helps you debug projects better, locate problems faster, and better understand and use packaging tools like WebPack. In this chapter you will learn more about what a JavaScript wrapper is and what its packaging mechanism is. What problem was solved? If you understand this, the rest of the WebPack optimization will be simple.

What is a module

A module can have many definitions, but IN my opinion: a module is a set of code associated with a particular function. It encapsulates implementation details, exposes a public API, and combines with other modules to build larger applications.

Modularity is the encapsulation of one or more implementations into a module in order to achieve a higher level of abstraction. We don’t have to worry about the dependencies within the module, we just call the API it exposes.

For example, in a project:

<html>
  <script src="/src/man.js"></script>
  <script src="/src/person.js"></script>
</html>
Copy the code

Where person.js depends on man.js, you will get an error if you reference them in reverse order. In large projects, this dependency is especially important and extremely difficult to maintain, in addition to the following problems:

  • Everything is loaded into the global context, leading to name collisions and overrides
  • A lot of manual work on the part of the developer to figure out the dependencies and inclusion order

So modules are especially important.

Since the front and back ends of JavaScript reside on opposite sides of HTTP, they play different roles and focus differently. Browser-side JavaScript needs to go through the process of distributing from one server to multiple clients, while server-side JS is the same code that needs to be executed multiple times. The bottleneck of the former is broadband, while the bottleneck of the latter is memory resources such as CPU. The former requires code to be loaded over the network, while the latter requires it to be loaded from disk, and the loading speed is not on the same order of magnitude. Therefore, the definition of modules on the front and back ends is not consistent. The definition of modules on the server side is:

  • CJS (CommonJS) : Intended for server-side JavaScript synchronization definitions, Node’s module system is actually based on CJS;

However, CommonJS is imported synchronously, because it is used on the server side and the files are all local. Even if the main thread is stuck synchronously, it will not affect the user experience. On the browser side, if you have to spend a lot of time waiting for the script to load while the UI is loading, it will cause a lot of problems for the user experience. Because of the network, the CommonJS specification for the backend JavaScript is not entirely suitable for the front-end application scenario. Here is the specification for the JavaScript front end.

  • AMD (Asynchronous Module Definition) : Defined as an asynchronous model for modules in browsers, RequireJS is AMD’s most popular implementation;
  • UMD (Generic Module Definition) : It’s essentially a piece of JavaScript code that sits at the top of the library and lets any loader, any environment, load them;
  • ES2015 (ES6) : Defines the semantics of asynchronous import and export modules, which are compiled intorequire/exportsThis is the most common module definition we use today;

What is a packer

Packagers are tools that front-end developers use to package JavaScript modules into an optimized JavaScript file that can be run in a browser, such as Webapck, rollup, gulp, etc.

For example, if you introduce multiple JavaScript files into an HTML file:

<html>
  <script src="/src/entry.js"></script>
  <script src="/src/message.js"></script>
  <script src="/src/hello.js"></script>
  <script src="/src/name.js"></script>
</html>
Copy the code

The following dependencies exist among the four imported files:

1. The modular

When the HTML is introduced, we need to pay attention to the order in which these four files are introduced (if the order is wrong, the project will report an error), and if we extend this to a functional, usable Web project, we may need to introduce dozens of files, and the dependencies are even more complex.

So, we need to modularize each dependency and let the packager help us manage them so that each dependency is referred to in the right place at the right time.

2. The bundle

In addition, when the browser opens the page, each JS file requires a separate HTTP request, which is four round trips, to properly launch your project.

We know that browsers load modules slowly, and even though HTTP/2 supports loading many small files efficiently, the performance is not as efficient as loading one (even without any optimization).

Therefore, it is best to merge all four files into one:

<html>
  <script src="/dist/bundle.js"></script>
</html>
Copy the code

This only requires one HTTP request.

So, modularity and bundling are the two main features that a packer needs to implement.

How to pack

How to package into a file? It usually has an entry file that starts with the entry file, takes all the dependencies, and packages them into a file called bundle.js. For example, we can merge the remaining three JavaScript files with/SRC /entry.js as the entry file.

Of course merging cannot be as simple as putting all the contents of four files into one bundle.js. So let’s think about, how does it actually implement an nan?

1. Parse the entry file to obtain all dependencies

First of all, the only thing we can determine is the address of the entry file

  • Gets the contents of its file
  • Gets the relative address of its dependent module

Since the dependency module is introduced through the relative path (import ‘./ message-js’), we need to save the path of the entry file, combine the relative address of the dependency module, we can determine the absolute address of the dependency module, and read its contents.

How do I represent a module in a dependency so that I can refer to it in a dependency graph

So we can express the module as:

  • Code: The parsed content of the file. Note that the parsed code can run in the current browser or environment as well as in older browsers or environments.
  • Dependencies: Array of dependencies (relative) paths for all modules.
  • Filename: indicates the absolute file path whenimportThe dependent module is a relative path, and the path of the dependent module is obtained by combining the current absolute path.

The filename (absolute path) can be used as the unique identifier of each module, and the content of the file can be directly obtained in the form of key: value.

/ / module
'src/entry': {
  code: ' '.// The content of the parsed file
  dependencies: ["./message.js"]./ / dependencies
}
Copy the code

2. Recursively parse all dependencies to generate a dependency graph

Now that we have determined the representation of the modules, how can we correlate all the modules together to produce a dependency graph that directly retries the dependencies of all modules, the code of the dependencies, the source of the dependencies, and the dependencies of the dependencies?

How do I maintain relationships between dependent files

For each module, the only name you can represent is filename. When you recurse to parse the entry file, you can get the dependencies array dependencies for each file, so you need to define one:

// Association
let mapping = {}
Copy the code

Used to map the relative import path to the absolute import path when the code is running.

So our module can be defined as [filename: {}] :

/ / module
'src/entry': {
  code: ' '.// The content of the parsed file
  dependencies: ["./message.js"]./ / dependencies
  mapping:{
    "./message.js": "src/message.js"}}Copy the code

Then the dependency graph is:

// graph Dependency graph
let graph = {
  / / entry module
  "src/entry.js": {
    code: ' '.dependencies: ["./src/message.js"].mapping: {"./message.js": "src/message.js"}},/ / the message module
  "src/message.js": {
    code: ' '.dependencies: [].mapping: {},}}Copy the code

When the project is running, the code content of the entry file is successfully obtained through the entry file, and the code is run. When the import dependent module is encountered, the module content can be read successfully by mapping it to the absolute path.

And the absolute path filename of each module is unique. When we connect the module to the dependency graph graph, we only need to judge whether graph[filename] exists. If it exists, there is no need to add it twice, and the repeated packaging of modules is removed.

3. Use the dependency graph to return a JavaScript file that can be run in the browser

The most popular form of code that can be executed immediately today is IIFE (Execute immediately functions), which also addresses the problem of global variable contamination.

IIFE

IIFE is an anonymous function that is called directly in the declared city. Since JavaScript variables are scoped only within a function, you don’t have to worry about contaminating global variables.

(function(man){
  function log(name) {
    console.log(`hello ${name}`);
  }
  log(man.name)
})({name: 'bottle'});
// hello bottle
Copy the code

4. Output to dist/bundle.js

Fs. writeFile Write dist/bundle.js.

So far, the packaging process and implementation scheme have been determined, let’s practice it again!

Create a MiniPack project

Create a minipack folder with NPM init and create the following files:

- src
- - entry.js Js / / entry
- - message.js / / dependencies
- - hello.js / / dependencies
- - name.js / / dependencies
- index.js / / packaging js
- minipack.config.js // Minipack packs configuration files
- package.json 
- .gitignore
Copy the code

One entry. Js:

import message from './message.js'
import {name} from './name.js'

message()
console.log('----name-----: ', name)
Copy the code

Message. Js:

import {hello} from './hello.js'
import {name} from './name.js'

export default function message() {
  console.log(`${hello} ${name}! `)}Copy the code

Hello. Js:

export const hello = 'hello'
Copy the code

Name. Js:

export const name = 'bottle'
Copy the code

Minipack. Config. Js:

const path = require('path')
module.exports = {
    entry: 'src/entry.js'.output: {
        filename: "bundle.js".path: path.resolve(__dirname, './dist'),}}Copy the code

And install files

npm install @babel/core @babel/parser @babel/preset-env @babel/traverse --save-dev
Copy the code

At this point, the entire project is created. Next comes the packing:

  • Parse the entry file and iterate over all dependencies
  • Recursively resolve all dependencies, generating a dependency graph
  • Using a dependency graph, return a JavaScript file that can be run in a browser
  • Output to the/dist/bundle.js

Parse the entry file and iterate over all dependencies

1. @babel/ Parser parses the entry file to obtain the AST

In the./index.js file, we create a wrapper that first parses the entry file, which we parse using the @babel/parser parser parser:

Step 1: Read the contents of the entry file

// Obtain the configuration file
const config = require('./minipack.config');
/ / the entry
const entry = config.entry;
const content = fs.readFileSync(entry, 'utf-8');
Copy the code

Step 2: Use@babel/parser(JavaScript parser) Parses the code and generates an AST (abstract syntax tree)

const babelParser = require('@babel/parser')
const ast = babelParser.parse(content, {
  sourceType: "module"
})
Copy the code

Where, sourceType indicates the schema that the code should parse. It can be one of “script”, “module”, or “unambiguous”, where “unambiguous” tells @babel/ Parser to guess, or “module” if using ES6 import or export, Otherwise, “script”. Here ES6 import or export is used, so “module”.

Since the AST tree is more complex, here we can look through astexplorer.net/ :

We’ve got all the ast in the entry file, what do we do next?

  • Parse the AST, parse the entry file content (a version of JavaScript that is backward compatible with current and older browsers or environments)
  • Gets all of its dependent modulesdependencies

2. Obtain the content of the entry file

Now that we know the entry file’s ast, we can parse the entry file’s contents using @babel/core’s transformFromAst method:

const {transformFromAst} = require('@babel/core');
const {code} = transformFromAst(ast, null, {
  presets: ['@babel/preset-env'],})Copy the code

3. Get all of its dependent modules

We need to obtain all dependent modules through ast, that is, we need to obtain all node.source. Value in AST, that is, the relative path of import module, through which we can find the dependent modules.

Step 1: Define a dependency array to hold all the dependencies resolved in the AST

const dependencies = []
Copy the code

Step 2: Use@babel/traverseIn conjunction with the Babel parser, it can be used to iterate over and update each child node

Traverse functions are a method of traversing the AST, provided by Babel-Traverse, whose traversing pattern is the classic visitor pattern, which defines a series of visitors, When the TYPE === visitor name of the AST is encountered, the function of that visitor is entered. The AST node of type ImportDeclaration is our import XXX from XXXX, and we push the address into dependencies.

const traverse = require('@babel/traverse').default
traverse(ast, {
  // Walk through all import modules and place relative paths in Dependencies
  ImportDeclaration: ({node}) = > {
    dependencies.push(node.source.value)
  }
})
Copy the code

3. Valid return

{
  dependencies,
  code,
}
Copy the code

Complete code:

/** * Resolve the file content and its dependencies, * expect to return: * dependencies: file dependencies * code: file parsing content * @param {string} filename File path */
function createAsset(filename) {
  // Read the contents of the file
  const content = fs.readFileSync(filename, 'utf-8')
  // Use @babel/parser (JavaScript parser) to parse code and generate an AST (abstract syntax tree)
  const ast = babelParser.parse(content, {
    sourceType: "module"
  })

  // Fetch all import modules from ast and place them in Dependencies
  const dependencies = []
  traverse(ast, {
    // Walk through all import modules and place relative paths in Dependencies
    ImportDeclaration: ({
      node
    }) => {
      dependencies.push(node.source.value)
    }
  })
  // Get the contents of the file
  const {
    code
  } = transformFromAst(ast, null, {
    presets: ['@babel/preset-env'],})// Return the result
  return {
    dependencies,
    code,
  }
}
Copy the code

Resolve all dependencies recursively to generate a dependency graph

Step 1: Obtain the entry file:

const mainAssert = createAsset(entry)
Copy the code

Step 2: Create a dependency graph:

Since each module is of the form key: value, define the dependency graph:

// entry: indicates the absolute address of the entry file
const graph = {
  [entry]: mainAssert
}
Copy the code

Step 3: recursively search for all dependency modules and add them to the dependency graph:

Define a recursive search function:

/** * iterates recursively to get all dependencies * @param {*} assert entry file */
function recursionDep(filename, assert) {
  // Trace all dependent files (module unique identifier)
  assert.mapping = {}
  // Obtain the current absolute path because the import paths of all dependent modules are relative
  const dirname = path.dirname(filename)
  assert.dependencies.forEach(relativePath= > {
    // Get the absolute path for createAsset to read the file
    const absolutePath = path.join(dirname, relativePath)
    // Associated with the current assert
    assert.mapping[relativePath] = absolutePath
    // Add the dependency files to the dependency graph because they are not included in the dependency graph
    if(! queue[absolutePath]) {// Get the dependent module content
      const child = createAsset(absolutePath)
      // Put the dependencies in queue so that recursionDep can be called again to resolve the dependencies of the dependent resource.
      // Until all dependencies are resolved, this creates a dependency graph from the entry file
      queue[absolutePath] = child
      if(child.dependencies.length > 0) {
        // Continue recursion
        recursionDep(absolutePath, child)
      }
    }
  })
}
Copy the code

Recursion from the entry file:

// Iterate through the queue, fetching each asset and all dependent modules and adding them to the queue until all dependent modules are traversed
for (let filename in queue) {
  let assert = queue[filename]
  recursionDep(filename, assert)
}
Copy the code

Use a dependency graph to return a JavaScript file that can be run in a browser

Step 1: Create an instant-execute function to run directly in the browser

const result = ` (function() { })() `
Copy the code

Step two: Pass the dependency graph as a parameter to the immediate-execute function

Define the passing parameter modules:

let modules = ' '
Copy the code

Iterating through the graph, adding each mod to modules with a key: value,

Note: Because the dependency graph is passed into the immediate execution function above, then written todist/bundle.jsRun, so,codeNeed to put infunction(require, module, exports){${mod.code}}If the browser does not support commonJS (there are no modules, exports, require, global), then we need to implement them. And inject it into the wrapper function.

for (let filename in graph) {
  let mod = graph[filename]
  modules += ` '${filename}': [
    function(require, module, exports) {
      ${mod.code}
    },
    The ${JSON.stringify(mod.mapping)}, `]
}
Copy the code

Step 3: Pass arguments to the immediate-execute function and execute the entry file immediately:

First implement a require function. Require (‘${entry}’) executes the entry file. Entry is the absolute path to the entry file and also the module unique identifier

const result = `
  (function(modules) {
    require('${entry}') ({})${modules}})
`
Copy the code

Note: Modules is a set of keys: values,, so we put it in {}

Step 4: Rewrite the browserrequireMethod when the code runsrequire('./message.js')Converted torequire(src/message.js)

const result = `
  (function(modules) {
    function require(moduleId) {
      const [fn, mapping] = modules[moduleId]
      function localRequire(name) {
        return require(mapping[name])
      }
      const module = {exports: {}}
      fn(localRequire, module, module.exports)
      return module.exports
    }
    require('${entry}') ({})${modules}})
`
Copy the code

Note:

  • moduleIdFor the incomingfilenameIs the unique identifier of the module
  • Through the deconstructionconst [fn, mapping] = modules[id]To get our function wrapper (function(require, module, exports) {${mod.code}}) andmappingsobject
  • Because in generalrequireAre allrequireRelative path, not absolute path, so rewritefn 的 requireMethods,requireRelative paths convert torequireThe absolute path, i.elocalRequirefunction
  • willmodule.exportsThe incoming tofnWhen the content of a dependent module needs to be output to another module for use, whenrequireWhen a dependent module, you can directly pass throughmodule.exportsReturn the result

Output to dist/bundle.js

/ / packaging
const result = bundle(graph)
/ / writer. / dist/bundle. Js
fs.writeFile(`${output.path}/${output.filename}`, result, (err) => {
  if (err) throw err;
  console.log('File has been saved');
})
Copy the code

Nine, summary and source code

Originally I wanted to write it simply, but as a result, there are so many fixes and modifications 🤦♀️🤦♀️🤦 30000, but it is always good to understand it thoroughly.

Source address: github.com/sisterAn/mi…

Minipack is referenced to solve the problem that modules are repeatedly packaged, and webPack is also referenced to define modules with filename as a unique identifier.

For more on this series,Go to the Github blog home page

Walk last

  1. ❤️ Have fun, keep learning, and always keep coding. 👨 💻

  2. If you have any questions or more unique opinions, please comment or directly contact The Bottle gentleman (the public number replies to 123)! 👀 👇

  3. 👇 welcome to pay attention to: front bottle gentleman, updated daily! 👇