preface

As a front-end engineer, you walk into the door of the company every morning, savor the smile of the girl at the front desk, take off your earphones, make a cup of tea, open Terminal and enter the corresponding project directory. Then NPM run start/dev or YARN start/dev begins the day’s work.

When you need to transform time just use dayJS or Momentjs, when you need to encapsulate HTTP requests you can use fetch or Axios, and when you need to do data processing you might use Lodash or underscore.

I don’t know if you’re aware that these toolkits are a huge productivity boost for us today, but where did it all start?

This starts with Modular design:

Modular design

When I was first introduced to the front end, I used to hear the term Modular design, and was often asked in interviews, “Talk about Modular front end”. Perhaps many of you could name a few familiar terms, and even their differences:

  • IIFE [Immediately Invoked Function Expression]
  • Common.js
  • AMD
  • CMD
  • ES6 Module

But as you read the source code of a project, from the first to commit to research, so you can reap maybe not, know what is the difference between them, and, more importantly, to know that before this history, what is the reason, caused the difference on the old specification to generate new specification, and on the basis of these, Maybe you’ll get a taste of what these changes mean, and maybe even be one of the rules makers at some point in the future.

So let’s go back a decade and see how modular design was implemented:

IIFE

IIFE is short for Immediately Invoked Function Expression and as a basic knowledge, many of you probably already know what IIFE is (if you already know IIFE, You can skip this section and read what comes later.) But we’ll still explain how it came about, because we’ll come back to it later:

In the beginning, our concept of module differentiation probably started with file differentiation. In a simple project, the programming convention is to use an HTML file plus several JavaScript files to distinguish different modules, like this:

We can illustrate this with a simple project to look at the contents of each file:

demo.html

This file simply introduces several other JavaScript files:

<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="Width = device - width, initial - scale = 1.0" />
  <meta http-equiv="X-UA-Compatible" content="ie=edge" />
  <title>demo</title>
</head>
<script src="main.js"></script>
<script src="header.js"></script>
<script src="footer.js"></script>

<body></body>

</html>
Copy the code

The other three JavaScript files

We define different variables in different JS files, corresponding to filename respectively:

var header = 'This is a top message.' //header.js
var main_message = 'This is a content message.'   //main.js
var main_error = 'This is an error message.'   //main.js
var footer = 'This is a bottom message.' //footer.js
Copy the code

Declaring variables in different files like this doesn’t really separate them from each other.

They are all bound to the Global window/Global object. Try printing to verify this:

This is a nightmare, and you may not realize what serious consequences this can lead to. Let’s try to assign to the header variable in footer.js. Let’s add this line at the end:

header = 'nothing'
Copy the code

If you print it, you’ll notice that window.header has been changed:

Imagine being on a team where you can never predict when or where you will accidentally change a previously defined variable.

Okay, now we know that we can’t separate these variables just by using different files, because they’re all tied to the same window variable.

But more importantly, how to fix it? We all know that in JavaScript, functions have their own scope, that is, if we can wrap these variables in a function, they will not be declared directly on the global variable window:

So now the contents of main.js will be changed to look like this:

function mainWarraper() {
  var main_message = 'This is a content message.' //main.js
  var main_error = 'This is an error message.' //main.js
  console.log('error:', main_error)
}

mainWarraper()
Copy the code

To make sure that what we defined in the function mainWarraper is executed, we have to execute mainWarraper() itself here. Now we can’t find main_message or main_error in the window. Because they are hidden in mainWarraper, but mainWarraper still contaminates our window:

This scheme is not perfect, how can it be improved?

The answer is IIFE. We can solve this problem by defining an anonymous function that executes immediately:

(function() {
  var main_message = 'This is a content message.' //main.js
  var main_error = 'This is an error message.' //main.js
  console.log('error:', main_error)
})()
Copy the code

Because it is an anonymous function that is released soon after execution, this mechanism does not pollute global objects.

It may seem cumbersome, but it does solve our need to separate variables, doesn’t it? Today, however, few people approach modular programming in this way.

What happened next?

CommonJS

In the winter of 2009, Kevin Dangoor, an engineer on the Mozilla team, started working on a project called ServerJS. He describes it this way:

What I’m describing here is not a technical problem. It’s a matter of people getting together and making a decision to step forward and start building up something bigger and cooler together.”

“What I’m describing here is not a technical problem. It’s a matter of everyone pulling together, making the decision to move forward and start building something bigger and cooler together.”

The project was renamed to CommonJS today in August 2009 to show the wider applicability of the API. I don’t think he expected that the rule would change the whole front end.

CommonJS is described in Wikipedia as follows:

CommonJS is a project with the goal to establish conventions on module ecosystem for JavaScript outside of the web browser. The primary reason of its creation was a major lack of commonly accepted form of JavaScript scripts module units which could be reusable in environments different from that provided by a conventional web browser e.g. web server or native desktop applications which run JavaScript scripts.

CommonJS is a contract project designed to build an ecosystem of modules for JavaScript beyond Web browsers. The main reason for its creation was the lack of a commonly accepted form of JavaScript script module unit that would allow JavaScript to be reused in environments other than those provided by traditional Web browsers, such as Web servers or native desktop applications running JavaScript scripts.

You already know the background of CommonJS, but it is a generic specification with many different implementations:

Image from Wiki

But we’ll focus on the implementation of Node.js.

Node.js Modules

We won’t explain the basic usage of the NODE.js Modules API here, because you can read the official documentation for all of this, and we’ll discuss why it’s designed this way, as well as some of the more difficult points to understand.

In The Node.js module system, each file is treated as a separate module. In a Node.js module, local variables are private, and this private implementation is achieved by wrapping The Node.js module in a function called The Module Wrapper. Let’s see what it looks like in the official example:

(function(exports, require, module, __filename, __dirname) {
// Module code actually lives in here
// Actually, the code inside the module is placed here
});
Copy the code

Yes, the code inside the module is actually contained in one of these functions before it is actually executed.

If you actually read about IIFE in the previous section, you’ll see that the core idea is the same: Node.js implements module privatization through a function. But what’s the difference?

There are five parameters, but let’s put them aside and try to think in terms of a module: What do you want to be able to do as a module?

  1. The ability to expose part of your method or variableThis is the purpose of my existence, because it is necessary for those who want to use me. [Exports: exported objects , Module: reference to a module ]
  2. The ability to introduce other modules: Sometimes I need help to implement some features and just focus on what I want to do (core logic). [Require: reference method ]
  3. Tell people where I am physically: Convenient for others to find me, and to update or modify me. [__filename: indicates the absolute filename.__dirname: indicates the directory path ]

An implementation of require in node.js Modules

Why do we need to know the implementation of the require method? By understanding this process, we can better understand the following questions:

  1. What exactly do we do when we introduce a module?
  2. exports 和 module.exportsWhat are the connections and differences?
  3. What’s the downside of this approach?

In the documentation, there is a simple implementation of require:

function require(/ *... * /) {
  const module = { exports: {}};((module, exports) = > {
    // Module code here. In this example, define a function.
    Here is the module code. In this case, we define a function
    function someFunc() {}
    exports = someFunc;
    // At this point, exports is no longer a shortcut to module.exports, and
    // this module will still export an empty default object.
    // When the code runs here, exports is no longer a reference to module.exports, and the current
    // Module will still export an empty object (just like the default object declared above)
    module.exports = someFunc;
    // At this point, the module will now export someFunc, instead of the
    // default object.
    // When the code runs up to this point, the current Module exports someFunc instead of the default object}) (module.module.exports);
  return module.exports;
}
Copy the code

Back to the question I just posed:

1. requireWhat kind of thing did you do?

Require copies the referenced module into the current module

2. exports 和 module.exportsThe connection and difference between?

Exports: module. Exports: module. Exports: module. Exports: module.

Exports is a reference to module.exports. As a reference, if we change its value, we are actually changing the value of its corresponding reference object.

Such as:

exports.a = 1
/ / is equivalent to
module.exports = {
    a: 1
}
Copy the code

But if we change the address referenced by exports, there is no effect on the original reference. Instead, we break the link between the reference and the original address:

exports = {
    a: 1
}

/ / equivalent to

let other = {a: 1} // To be more intuitive, we declare a variable like this
exports = other
Copy the code

Exports changed from pointing module. Exports to other.

3. The disadvantages

The CommonJS standard is intended to allow JavaScript to be modularized in multiple environments, but implementations in Node.js rely on Node.js environment variables: Module, exports, require, global, browsers didn’t work, so Browserify was implemented, but that’s not the topic of this article, you can read ruan yifen’s article.

So having said modularity on the server side, what happened to modularity on the browser side?

RequireJS & AMD (Asynchronous Module Definition)

Imagine if we were in a browser environment and managed our modules (such as Browserify) in a similar way to Node.js Modules.

Since we’ve seen the implementation of require(), you can see that this is a copy process, assigning the required content to a module object’s property and returning the exports property of that object.

What’s the problem with that? Methods and properties in the referenced module cannot be used until we have finished copying. On the server side this might not be a problem (because the server files are stored locally and cached), but in the browser environment it would block, preventing us from progressing further, and possibly executing an undefined method that would cause an error.

As opposed to server-side modularity, standards for modularity in the browser environment must meet a new requirement: asynchronous module management

RequireJS emerged in this context, and let’s briefly take a look at its core:

  • Introduce other modules:require()
  • Define a new module:define()

Examples of use in official documentation:

requirejs.config({
    // The module ID in js/lib is loaded by default
    baseUrl: 'js/lib'.// Remove module ID modules starting with "app" will be loaded from js/app.
    // The configuration for Paths is associated with baseURL, and since paths may be a directory,
    // Do not use the.js extension
    paths: {
        app: '.. /app'}});// Start the main logic
requirejs(['jquery'.'canvas'.'app/sub'].function   ($, canvas, sub) {
    //jQuery, Canvas and app/sub modules are loaded and ready to use here.
});
Copy the code

Examples of definitions in official documentation:

// Simple object definition
define({
    color: "black".size: "unisize"
});

// You can define this when you need some logic to do the preparatory work:
define(function () {
    // Some preparatory work can be done here
    return {
        color: "black".size: "unisize"}});// Rely on modules to define your own modules
define(["./cart"."./inventory"].function(cart, inventory) {
        Define your own module by returning an object
        return {
            color: "blue".size: "large".addToCart: function() {
                inventory.decrement(this);
                cart.add(this); }}});Copy the code

advantage

RequireJS is implemented based on the AMD specification, so what advantage does it have over node.js’s Module?

  • Returning module values as functions, especially constructors, makes API design easier. Node supports this through module.exports, but it is clearer to use “return function (){}”. This means that we don’t have to implement “module.exports” by dealing with “module”, which is a cleaner code expression.
  • Dynamic code loading (done on AMD systems by require ([], function () {})) is a basic requirement. CJS talks about it, has some suggestions, but doesn’t quite cover it. Node does not support this requirement and instead relies on the synchronization behavior of require (“), which is inconvenient for Web environments.
  • The Loader plug-in is very useful and helps avoid the common nested curly brace indentation in callback-based programming.
  • Selectively mapping a module to be loaded from another location conveniently provides mock objects for testing.
  • Each module can have at most one IO operation, and it should be concise. Web browsers cannot tolerate finding modules from more than one IO. This is the opposite of multipath look-ups in Node today, and avoids using the “main” property of package.json. Using only module names, simply mapping module names to a location based on the project location, does not require reasonable default rules for detailed configuration, but allows simple configuration if necessary.
  • Best of all, if there is an “opt-in” that can be called so that old JS code can be added to the new system.

If a JS module system fails to provide the above functionality, it will be at a significant disadvantage to AMD and its associated apis in terms of callback requirements, loader plug-ins, and path-based module ids.

The new problem

Using RequireJS to declare a module, you must specify all dependencies that will be passed to the factory as parameters. For dependent modules, early execution (or deferred execution is an option in RequireJS 2.0) is called dependency preloading.

What’s the problem with that?

This makes the development process more difficult, whether it’s reading previous code or writing new content, and it also happens that the content in another module introduced is executed conditionally.

SeaJS & CMD (Common Module Definition)

For those parts of the AMD specification that can be optimized, the CMD specification appears, and SeaJS as one of its concrete implementations is very similar to AMD:

// An example of AMD, of course, this is an extreme case
define(["header"."main"."footer"].function(header, main, footer) { 
    if (xxx) {
      header.setHeader('new-title')}if (xxx) {
      main.setMain('new-content')}if (xxx) {
      footer.setFooter('new-footer')}});// The equivalent of CMD
define(function(require, exports, module) {
    if (xxx) {
      var header = require('./header')
      header.setHeader('new-title')}if (xxx) {
      var main = require('./main')
      main.setMain('new-content')}if (xxx) {
      var footer = require('./footer')
      footer.setFooter('new-footer')}});Copy the code

We can clearly see that in the CMD specification, an external module is introduced only when it is used. This answers the question left in the previous section and is the biggest difference from the AMD specification: CMD advocates relying on nearby + delayed execution

There are still problems

As we can see, defining a module according to the CMD specification’s dependencies nearest rules can lead to heavy loading logic for modules, sometimes you don’t know exactly which modules the current module depends on or the dependencies are not intuitive.

And for AMD and CMD, are only applicable to the browser side of the specification, and node.js Module only applicable to the server side, have their own limitations.

ECMAScript6 Module

ECMAScript6 standard adds the JavaScript language level module architecture definition, as a common browser and server module solution it can replace the AMD, CMD,CommonJS we mentioned earlier. (There is also a UMD (Universal Module Definition) specification that also applies to the front and back ends, but it is not covered in this article. If you are interested, check out the UMD documentation.)

Module about ES6 believe that we will use every day in the work, for the use of questions can see ES6 Module introduction, Ruan Yifeng, of course, you can also view the official document TC39

Why add a modular architecture definition to the standard? To quote from the documentation:

“The goal for ECMAScript 6 modules was to create a format that both users of CommonJS and of AMD are happy with”

“The goal of ECMAScript 6 Modules is to create a format that satisfies both CommonJS and AMD users.”

By what means does it do this?

  • Like CommonJS, it has a compact syntax, loop dependency and support for individual exports.
  • As with AMD, asynchronous loading and configurable module loading are supported directly.

In addition, it has more advantages:

  • The syntax is more compact than CommonJS.
  • Structures can be statically analyzed (for static inspection, optimization, etc.).
  • Support for loop dependencies is better than CommonJS.

Notice that the words cyclic dependency and static analysis appear in this description, which we’ll discuss in more detail later. First let’s take a look at what the ES6 Modules specification is defined in the official TC39 documentation.

Dive into the ES6 Module specification

In section 15.2.1.15, Module Record Fields and Abstract Methods of Module Records that define Abstract Module Records

Module Record Fields Module Record Fields

Field Name(Field Name) Value Type A fancy (Meaning)
[[Realm]] domain Realm Record | undefined The Realm within which this module was created. undefined if not yet assigned.

Will create the current module, or undefined if the module is not declared.
[[Environment]] Environment Lexical Environment | undefined The Lexical Environment containing the top level bindings for this module. This field is set when the module is instantiated.

The lexical environment contains the top-level bindings for the current module. This field is set when the module is instantiated.
[[Namespace]] Indicates the Namespace Object | undefined The Module Namespace Object if one has been created for this module. Otherwise undefined.

Namespace object for the module, if one has been created for this module. Otherwise, undefined.
[[Evaluated]] End of execution Boolean Initially false, true if evaluation of this module has started. Remains true when evaluation completes, even if it is an abrupt completion

The initial value is false and becomes true when the module starts executing and continues until the end of execution, even if it terminates suddenly. (Sudden terminations can occur for a variety of reasons. If you are interested in the reason, read this answer.)

Abstract Methods of Module Records Abstract Methods of Module Records

Method Method The Purpose to
GetExportedNames(exportStarSet) Return a list of all names that are either directly or indirectly exported from this module.

Returns a list of all names exported directly or indirectly from this module.
ResolveExport(exportName, resolveSet, exportStarSet)

Return the binding of a name exported by this modules. Bindings are represented by a Record of the form {[[module]]: Module Record, [[bindingName]]: String}.

Returns a binding of the name exported by this module. {[[module]]: Module Record, [[bindingName]]: String}
ModuleDeclarationInstantiation(a)

Transitively resolve all module dependencies and create a module Environment Record for the module.

Transitively resolve all module dependencies and create an environment record for the module
ModuleEvaluation(a)

Do nothing if this module has already been evaluated. Otherwise, transitively evaluate all module dependences of this module and then evaluate this module.

If this module has already been executed, nothing is done. Otherwise, pass all module dependencies that execute the module, and then execute the module.

ModuleDeclarationInstantiation must be completed prior to invoking this method.

ModuleDeclarationInstantiation should be done before this method is called

That is, a very basic module should contain at least these fields and methods. After reading it over and over again, you will find that it is only a basic module that should contain methods of certain functions or define the format of the module. However, when we implement it, it is just like what the original text says:

An implementation may parse a sourceText as a Module, analyze it for Early Error conditions, and instantiate it prior to the execution of the TopLevelModuleEvaluationJob for that sourceText.

Implementation can is: will sourceText resolved as a module, the analysis of early fault conditions, and before the execution TopLevelModuleEvaluationJob to instantiate.

An implementation may also resolve, pre-parse and pre-analyze, and pre-instantiate module dependencies of sourceText. However, the reporting of any errors detected by these actions must be deferred until the TopLevelModuleEvaluationJob is actually executed.

Implementations can also be: parse, pre-parse, and pre-parse, and pre-instantiate sourceText’s module dependencies. However, these operations must be detected any error, delay to the actual execution TopLevelModuleEvaluationJob then reported.

From all this we can only draw one conclusion. When it comes to concrete implementation, only the first step is fixed, namely:

Parsing: As described in the ParseModule section, the module’s source code is first checked for syntax errors. For example, early-errors, if parsing fails, have the body report one or more parsing errors and/or early errors. If parsing succeeds and no earlier errors are found, execution continues with the body as the generated parse tree, finally returning a Source Text Module Records

So what happens next? We can read the specific implementation of the source code to analyze.

Transforms ES6 Module implementation from babel-helper-module-transforms

Babel, as the official compiler of ES6, plays a huge role in front-end development today. It can help us translate ES6 syntax code written by developers into ES5 code and then deliver it to the JS engine. This behavior allows us to use the convenience of ES6 without any compilers.

Transforms The ES6 module transforms using a concrete implementation of babel-helper-module-transforms from Babel

Instead of going through the source code line by line, I’ll look at the logic in terms of structure and invocation

Let’s start by listing all the methods that appear in this file (omitting the body and parameters).

/** * Perform all of the generic ES6 module rewriting needed to handle initial * module processing. This function will rewrite the majority of the given * program to reference the modules described by the returned metadata, * and returns a list of statements for use when initializing the module. * Perform all common ES6 module rewrites required to handle initialization * module processing. This function overrides most of the modules that are described by the metadata returned by the given program references, and returns a list of statements used when initializing the module. * /
export function rewriteModuleStatementsAndPrepareHeader() {... }/** * Flag a set of statements as hoisted above all else so that module init * statements all run before user code. * Marks one set of statements above all others so that the module initializes * statements to run all before user code. * /
export function ensureStatementsHoisted() {... }/** * Given an expression for a standard import object, like "require('foo')", * wrap it in a call to the interop helpers based on the type. * Wrap it in a call to the Interop helper based on type. * /
export function wrapInterop() {... }/** * Create the runtime initialization statements for a given requested source. * These will initialize all of the Runtime import/export Logic that * can't be handled statically by the statements created by * Create a runtime initialization statement for the given request source. * the import/export will initialize all runtime logic * cannot be handled by the create statement static * buildExportInitializationStatements (). * /
export function buildNamespaceInitStatements() {... }/** * Build an "__esModule" header statement setting the property on a given object. * Build an "__esModule" header statement setting the property on a given object
function buildESModuleHeader() {... }/** * Create a re-export initialization loop for a specific imported namespace. * Create a re-export initialization loop for a specific imported namespace. * /
function buildNamespaceReexport() {... }/** * Build a statement declaring a variable that contains all of the exported * variable names in an object so they can Easily referenced From an * export * from statement to check for conflicts. * Construct a statement that declares variables containing all exported variable names in the object, So that they can be easily referenced from the export * FROM statement to check for conflicts. * /
function buildExportNameListDeclaration() {... }/** * Create a set of statements that will initialize all of the statically-known * export names with their expected Values. * Creates a set of statements that will initialize all statically known export names with expected values */
function buildExportInitializationStatements() {... }/** * Given a set of export names, Create a set of nested assignments to * initialize them all to a given expression. Create a set of nested assignments and initialize them all to the given expression. * /
function buildInitStatement() {... }Copy the code

Then let’s look at their call relationships:

We call B from A in the form A -> B

  1. BuildNamespaceInitStatements: request for a given source to create a run-time initialization statement. These initialize all runtime import/export logic

  2. RewriteModuleStatementsAndPrepareHeader all general ES6 rewrite module to reference returns the metadata description of the module. – > buildExportInitializationStatements create all known the name of the static exports – > buildInitStatement given a set of export names, Create a set of nested assignments and initialize them all to the given expression.

So to summarize, plus the first step we already know, the next step is actually divided into two parts:

  1. Parsing: The source code of the module is checked for syntax errors first. For example, early-errors, if parsing fails, have the body report one or more parsing errors and/or early errors. If parsing succeeds and no earlier errors are found, execution continues with the body as the generated parse tree, finally returning a Source Text Module Records
  2. Initialize all runtime import/export logic
  3. The module described by referring to the returned metadata and initializing all static exports with a set of export names as the specified expression.

At compile time, we can see exactly what happens to the ES6 Module code:

ES6 Module source -> Babel translation -> a section of executable code

That is, until the end of compilation, the code inside our module is simply converted into a static piece of code, which will only be executed at runtime.

This makes static analysis possible.

The last

In this paper, we started from the development history of JavaScript Module and talked about the compilation of ES6 code, which is closely related to us today. We are very grateful to the predecessors for these roads, so that ordinary people like me can also enter the world of programming. I have to sigh that the deeper a problem is, it is not easy to find.

Thanks to those of you who are patient enough to read this, because this article also took 4 days to research, often lamenting how little valuable information there is.

We’ll talk more about static analysis and circular references in the next article

I’m Dendoink, the original writer of Odd Dance Weekly, nuggets [co-editor/booklet writer].

For technologists, skill is the ability of individual soldiers to fight, while skill is the method of using ability. In one’s ease, in one’s power is art. In front end entertainment, I want to be an excellent people’s artist.

Scan the public account [Front-end Bully] and I’ll be waiting for you here: