Babel’s amazing adventures

I was looking for articles on Babel’s principles and code parsing, but I couldn’t find any, so I decided to check the source code for myself.

Because the code structure is complicated, it is difficult to see directly, so I choose to start with the first commit, switch to the core commit, and see how the code changes.

git clone [email protected]:babel/babel.git
Copy the code

The starting point

git reset --hard aedcd4e12fd2dca3b415b0019fb96962a8bad27d
Copy the code

The initial submission was just a file acorn.js, and the Babel project was originally supposed to fork from Acorn.

Acorn. js is a JavaScript code parser. The entry point is the parse method, which is executed alternately through the core methods readToken and parseStatement.

The readToken method routes to the readWord, readNumber and other branches through various branches. After processing, the current TOK information is recorded through finishToken. ParseStatement is processed through various conditional branches. It will route to parseFunciton, parseNode and other branches, and record the current Node information through finishNode.

As a whole, the Parse method only needs to be traversed once to complete parsing and generate the AST syntax tree.

The first code submission is interesting and worth a look.

Performance comparison test

git reset --hard a1d958751911faa06a18d0f99d5ca98d053ce655
Copy the code

Acorn introduced Esprima and UglifyJS to compare the parsing speed.

The end of the acorn

git reset --hard daedc6fcb323cd6e205a87d1d8e5b78d1fbca090
Copy the code

This is the last commit of acorn code in the project. Babel was originally supposed to be the Acorn project from this commit fork.

[Sebastian McKenzie] (2014-09-28)

git reset --hard c97696c224d718d96848df9e1577f337b45464be
Copy the code

Babel project was originally named 6to5. This time, the message was ‘first commit’ and submitted on September 28, 2014. The code fork in Acorn was completely discarded after some research.

The overall structure of the project is relatively simple. The core code is in /lib/6to5. Esprima is used to generate AST tree, estraverse to traverse syntax tree for node modification, and escodeGen is used to generate Javascript code.

// package.json "escodegen": "https://github.com/Constellation/escodegen/tarball/master", "esprima": "Https://github.com/esnext/esprima/tarball/harmony-esnext", "estraverse" : "^ 1.5.1",Copy the code

A little demo

Here is a simple example of converting an arrow function to a function declaration.

var esprima    = require("esprima");
var estraverse = require("estraverse");
var escodegen = require("escodegen");

var raw = " \ const greet = (name) => { \
  return 'hello' + name; \
}; \
";

var ast = esprima.parse(raw);

estraverse.traverse(ast, {
  enter: function (node, parent) {
    if (node.type === 'VariableDeclaration') {
      node.kind = 'var'
    }
    if (node.type === 'ArrowFunctionExpression') {
      node.type = 'FunctionExpression'}}});var code = escodegen.generate(ast);
console.log(code);

/ / the result
var greet = function (name) {
    return 'hello' + name;
};
Copy the code

Source fragment parsing

Body. Type! == “BlockStatement” handles cases where arrow functions do not use {} such as () => return 1. If this expression is included, bind(this) will be template-bound.

exports.ArrowFunctionExpression = function (node) {
  var body = node.body;
  if(body.type ! = ="BlockStatement") {
    body = {
      type: "BlockStatement".body: [{
        type: "ReturnStatement".argument: body
      }]
    };
  }

  node.expression = false;
  node.body = body;
  node.type = "FunctionExpression";

  if (traverse.hasType(node, "ThisExpression")) {
    return util.template("function-bind-this", {
      FUNCTION: node
    });
  } else {
    returnnode; }};Copy the code

other

• MIT License
• Usually the front-end project has not used Makefile, at this time the project Makefile content is still very simple, suitable for learning
• Continuous integration Travis
• Editorconfig is handy for formatting with the editor plug-in
• Istanbul mocha does some code testing

A: we are fixtures every test/fixtures folder (suiteName), and a: taskName (per suite folder), and we are going to compare the actual.js code to the expected.

Looking at the UT code is also a quick way to see what the project implements

v1.7.7

This is the earliest remaining tag on Github

V1.10.7 (one month later, 2014-10-28)

/ / package. Json "es6 - shim" : "0.18.0", "es6 - symbol" : "while", "acorn - JSX" : "Https://github.com/sebmck/acorn-jsx/archive/master.tar.gz", "acorn - recast" : "0.8.0-4", "acorn - ast - types" : "0.5.3-1"Copy the code

The overall change is minor, parsing has been changed to Acorn-Recast, but some packages cannot be found when installing. It’s worth noting that you’ve already seen some simple descriptions of polyfills in the README

The birth of Babel

Track subsequent changes

You can view subsequent major changes in Changelog. md. Add tags in v2.6.0 Changelog. md.

[New Feature]
[Bug Fix]
[Spec Compliancy]
[Breaking Change]
[Documentation]
[Internal]
[Polish]
Copy the code

V4.0.0 was renamed Babel

After V4.0.1, changelog. md tracks changes after V4, and changes before V4 are recorded in Changelog-6to5.md

Some changes prior to V7.0.0

• 6.1.0 Add Babel – doctor CLI.
• 6.0.0 Split up internals into packages. The code structure is greatly adjusted, basically consistent with the current code
• 5.8.21 Add support for Flow export types.
• 5.7.0
  • Add .babelignore file to be consistent with other tools.
  • Allow .babelrc configs to be specified via package.json.
• 5.4.0 Added env option. Especially handy when using the .babelrc
• 5.1.0 Add trailing function Comma proposal
• 5.0.0 added a large version with more features
  • Allow ES7 transformer to be enabled via optional instead of only via stage
  • Add support for .babelrc on absolute paths.
  • Plugin API
• Babel Now Compiles itself! (hahahaha)
• 4.5.0 Add .babelrc support.
• 4.0.06to5 is now known as Babel
• 3.6.5 the Addvalidation.react transformer.
• 3.6.4 Add support for flow type casts and module types
• 3.4.0 Add commonStandard module formatter.

summary

Overall, the original Acorn AST syntax tree code; 6to5 first commit; V4.0.0 Babel molding code; V6.0.0 structure major adjustment; These four are important points to read about the Babel project.

Babel code comb

• @babel/core
  • @babel/parser Generates the AST syntax tree
  • The @babel/traverse traverse maintains the state of the entire tree and is responsible for replacing, deleting, and adding nodes
  • @babel/ Generator converts the AST syntax tree into Javascript code
• @babel/ CLI command line tool. The command line runs @babel/core
• The @babel/types check modifies the AST node

babel-polyfill

• use
  • Includes Regenerator and Core-JS to simulate a complete ES2015+ environment
  • It is implemented by rewriting global Prototype, handling the new API in the compiled code, and inserting some helper functions in the code
  • Fixed a problem with Babel not converting new apis
• The problem
  • Polluting the overall environment
  • Duplicate code in different code files, resulting in larger compiled code

babel-runtime

• Babel-runtime provides utility functions for compiling modules
• When the plug-in babel-plugin-transform-Runtime is enabled, Babel uses the utility functions under babel-Runtime
• The babel-Runtime plug-in converts the code of these utility functions into require statements pointing to a reference to Babel-Runtime. Require (‘ babel-Runtime ‘) manually whenever you want to translate an API
• According to the need to load

For example,

require Promise from 'babel-runtime/core-js/promise
Copy the code

babel-plugin-transform-runtime

• Use babel-Runtime to solve manual require problems.
• If the AST that is analyzed references spacers in babel-rumtime (by mapping), insert the required spacers at the top of the current module.
• Transform-runtime does not contaminate native objects, methods, or affect other polyfills