Each step in this process involves creating or manipulating an abstract syntax tree, also known as an AST.

Babel using a ESTree based and modified the AST, its kernel documentation can be in [here] (https://github. com/Babel/Babel/blob/master/doc/AST/spec. Md) found. .

function square(n) {
  return n * n;
}
Copy the code

The AST Explorer gives you a better sense of the AST nodes. Here is a link to an example of the code above.

The program can be represented as a tree like this:

- FunctionDeclaration:
  - id:
    - Identifier:
      - name: square
  - params [1]
    - Identifier
      - name: n
  - body:
    - BlockStatement
      - body [1]
        - ReturnStatement
          - argument
            - BinaryExpression
              - operator: *
              - left
                - Identifier
                  - name: n
              - right
                - Identifier
                  - name: n
Copy the code

Or a JavaScript Object like this:

{
  type: "FunctionDeclaration".id: {
    type: "Identifier".name: "square"
  },
  params: [{
    type: "Identifier".name: "n"}].body: {
    type: "BlockStatement".body: [{
      type: "ReturnStatement".argument: {
        type: "BinaryExpression".operator: "*".left: {
          type: "Identifier".name: "n"
        },
        right: {
          type: "Identifier".name: "n"}}}]}}Copy the code

You’ll notice that each layer of the AST has the same structure:

{
  type: "FunctionDeclaration".id: {... },params: [...]. .body: {...}
}
Copy the code

{
  type: "Identifier".name:... }Copy the code

{
  type: "BinaryExpression".operator:... .left: {... },right: {...}
}
Copy the code

Note: Some attributes have been removed for simplification purposes

Each of these layers is also called a Node. An AST can consist of a single node or hundreds or thousands of nodes. Together, they describe program syntax for static analysis.

Each node has the following Interface:

interface Node {
  type: string;
}
Copy the code

The type field is a string representing the type of the node (for example, “FunctionDeclaration”, “Identifier”, or “BinaryExpression”). Each type of node defines additional attributes that further describe the node type.

Babel also generates additional attributes for each node that describe its position in the original code.

{
  type:... .start: 0.end: 38.loc: {
    start: {
      line: 1.column: 0
    },
    end: {
      line: 3.column: 1}},... }Copy the code

Each node has attributes like start, end, and LOC.

The parsing step receives the code and outputs the AST. This step is broken down into two stages: Lexical Analysis ** and Syntactic Analysis. .

The parsing phase converts a token flow into the AST form. This phase uses the information in the tokens to transform them into an AST representation structure that makes subsequent operations easier.

The code generation step converts the final AST (after a series of transformations) into string code and creates source maps. .

Code generation is simple: Depth-first traverses the AST and builds a string that represents the transformed code.

When we talk about “entering” a node, we really mean that we are accessing them, and we use the term because of the concept of visitor patterns. .

Visitor is a cross-language pattern for AST traversal. They are simply an object that defines a method for obtaining a specific node in a tree structure. That’s a little abstract so let’s look at an example.

const MyVisitor = {
  Identifier() {
    console.log("Called!"); }};// You can also create a visitor object and add methods to it later.
let visitor = {};
visitor.MemberExpression = function() {};
visitor.FunctionDeclaration = function() {}
Copy the code

Note: Identifier() {… } is Identifier: {enter() {… }}. .

This is a simple visitor that, when used for traversal, calls the Identifier() method whenever an Identifier is encountered in the tree.

So the Identifier() method is called four times in the code below (there are four identifiers, including Square). .

function square(n) {
  return n * n;
}
Copy the code

path.traverse(MyVisitor);
Called!
Called!
Called!
Called!
Copy the code

These calls all occur on entry to the node, although we can sometimes call the visitor method on exit. .

Suppose we have a tree structure:

- FunctionDeclaration
  - Identifier (id)
  - Identifier (params[0])
  - BlockStatement (body)
    - ReturnStatement (body)
      - BinaryExpression (argument)
        - Identifier (left)
        - Identifier (right)
Copy the code

As we go down each branch of the tree we eventually get to the end, so we need to go up to get to the next node. As we walk down the tree we enter each node, and as we walk up we exit each node.

Let’s walk through this process using the tree above as an example.

• Enter theFunctionDeclaration
  • Enter theIdentifier (id)
  • Come to an end
  • exitIdentifier (id)
  • Enter theIdentifier (params[0])
  • Come to an end
  • exitIdentifier (params[0])
  • Enter theBlockStatement (body)
  • Enter theReturnStatement (body)
    • Enter theBinaryExpression (argument)
    • Enter theIdentifier (left)
      • Come to an end
    • exitIdentifier (left)
    • Enter theIdentifier (right)
      • Come to an end
    • exitIdentifier (right)
    • exitBinaryExpression (argument)
  • exitReturnStatement (body)
  • exitBlockStatement (body)
• exitFunctionDeclaration

So you actually have two chances to visit a node when creating visitors.

const MyVisitor = {
  Identifier: {
    enter() {
      console.log("Entered!");
    },
    exit() {
      console.log("Exited!"); }}};Copy the code

If necessary, you can use the method name | divided into Idenfifier | MemberExpression form of string, use the same function in a variety of access nodes. .

An example in the flow-Comments plug-in is as follows:

const MyVisitor = {
  "ExportNamedDeclaration|Flow"(path) {}
};
Copy the code

You can also use aliases (as defined by babel-types) in visitors.

For example,

Function is an alias for FunctionDeclaration, FunctionExpression, ArrowFunctionExpression, ObjectMethod and ClassMethod.

const MyVisitor = {
  Function(path) {}
};
Copy the code

When you have a visitor to the Identifier() member method, you are actually accessing the path rather than the node. In this way, you are dealing with the responsive representation of the node rather than the node itself.

const MyVisitor = {
  Identifier(path) {
    console.log("Visiting: "+ path.node.name); }};Copy the code

a + b + c;
Copy the code

path.traverse(MyVisitor);
Visiting: a
Visiting: b
Visiting: c
Copy the code

Next, let’s introduce the concept of scope. JavaScript supports lexical scope, where blocks of code create new scopes in a tree nested structure.

// global scope

function scopeOne() {
  // scope 1

  function scopeTwo() {
    // scope 2}}Copy the code

In JavaScript, whenever you create a reference, whether through variables, functions, types, params, module imports, labels, etc., it belongs to the current scope.

var global = "I am in the global scope";

function scopeOne() {
  var one = "I am in the scope created by `scopeOne()`";

  function scopeTwo() {
    var two = "I am in the scope created by `scopeTwo()`"; }}Copy the code

Deeper inner scope code can use references from outer scopes.

function scopeOne() {
  var one = "I am in the scope created by `scopeOne()`";

  function scopeTwo() {
    one = "I am updating the reference in `scopeOne` inside `scopeTwo`"; }}Copy the code

An inner scope can also create a reference with the same name as an outer scope.

function scopeOne() {
  var one = "I am in the scope created by `scopeOne()`";

  function scopeTwo() {
    var one = "I am creating a new `one` but leaving reference in `scopeOne()` alone."; }}Copy the code

When writing a transformation, you must be careful about scoping. We need to make sure that changing parts of the code doesn’t break existing code.

When we add a new reference, we need to make sure that the name of the newly added reference does not conflict with all existing references. Or we just want to find all references that use a variable, we just want to find those references within a given Scope.

The scope can be expressed as follows:

{
  path: path,
  block: path.node,
  parentBlock: path.parent,
  parent: parentScope,
  bindings: [...]. }Copy the code

When you create a new scope, you need to give its path and parent scope, and then it collects all references (” bindings “) in that scope during traversal.

Once the references are collected, you can use various methods on Scopes, which we’ll learn about later.

Babel is actually a collection of modules. In this section we explore some of the major modules, explaining what they do and how to use them.

Note: This section is not a substitute for the detailed API documentation, which will be available shortly.

Babylon is the parser for Babel. Originally came out of Acorn project Fork. Acorn was fast, easy to use, and designed a plug-in based architecture for non-standard features (and those that will be standard features in the future).

First, let’s install it.

$ npm install --save babylon
Copy the code

Start by parsing a code string:

import * as babylon from "babylon";

const code = `function square(n) { return n * n; } `;

babylon.parse(code);
// Node {
// type: "File",
// start: 0,
// end: 38,
// loc: SourceLocation {... },
// program: Node {... },
// comments: [],
// tokens: [...]
// }
Copy the code

We can also pass options to the parse() method as follows:

babylon.parse(code, {
  sourceType: "module".// default: "script"
  plugins: ["jsx"] // default: []
});
Copy the code

SourceType can be “module” or “script”, which indicates which mode Babylon should be parsed in. “Module” will parse in strict mode and allow module definition, “script” will not.

Note: sourceType defaults to “script” and generates an error when import or export is found. ScourceType: “module” is used to avoid these errors.

Because Babylon uses a plug-in based architecture, there is a plugins option to switch on and off the built-in plug-ins. Note that Babylon has not yet made this API available to external plug-ins, and it is not ruled out that it will be available in the future.

For a complete list of plug-ins, see the Babylon README file. .

The Babel Traverse module maintains the state of the entire tree and is responsible for replacing, removing, and adding nodes.

Run the following command to install:

$ npm install --save babel-traverse
Copy the code

We can use Babylon with us to iterate over and update nodes:

import * as babylon from "babylon";
import traverse from "babel-traverse";

const code = `function square(n) { return n * n; } `;

const ast = babylon.parse(code);

traverse(ast, {
  enter(path) {
    if (
      path.node.type === "Identifier" &&
      path.node.name === "n"
    ) {
      path.node.name = "x"; }}});Copy the code

The Babel Types module is a Lodash library for AST nodes that contains methods for constructing, validating, and transforming AST nodes. This tool library contains thoughtful tool methods that are useful for writing logic that processes AST.

To install it, run the following command:

$ npm install --save babel-types
Copy the code

Then use it as follows:

import traverse from "babel-traverse";
import * as t from "babel-types";

traverse(ast, {
  enter(path) {
    if (t.isIdentifier(path.node, { name: "n" })) {
      path.node.name = "x"; }}});Copy the code

You’ll notice that the BinaryExpression definition above has a Builder field. .

builder: ["operator"."left"."right"]
Copy the code

This is because each node type has a constructor method builder, which is used like this:

t.binaryExpression("*", t.identifier("a"), t.identifier("b"));
Copy the code

You can create an AST like this:

{
  type: "BinaryExpression".operator: "*".left: {
    type: "Identifier".name: "a"
  },
  right: {
    type: "Identifier".name: "b"}}Copy the code

When printed it looks like this:

a * b
Copy the code

The constructor also validates the nodes it creates and throws a descriptive error in case of misuse, which leads to the next method type.

[WIP]

The Babel Generator module is a code Generator for Babel that reads the AST and translates it into code and sourcemaps (sourcemaps).

Run the following command to install it:

$ npm install --save babel-generator
Copy the code

Then use it as follows:

import * as babylon from "babylon";
import generate from "babel-generator";

const code = `function square(n) { return n * n; } `;

const ast = babylon.parse(code);

generate(ast, {}, code);
/ / {
// code: "..." .
// map: "..."
// }
Copy the code

You can also pass options to the generate() method. .

generate(ast, {
  retainLines: false.compact: "auto".concise: false.quotes: "double".// ...
}, code);
Copy the code

Babel-template is another module that is small but very useful. It lets you write string code with placeholders instead of manual encoding, especially when generating large AST. In computer science, this ability is called quasiquotes.

$ npm install --save babel-template
Copy the code

import template from "babel-template";
import generate from "babel-generator";
import * as t from "babel-types";

const buildRequire = template(` var IMPORT_NAME = require(SOURCE); `);

const ast = buildRequire({
  IMPORT_NAME: t.identifier("myModule"),
  SOURCE: t.stringLiteral("my-module")});console.log(generate(ast).code);
Copy the code

var myModule = require("my-module");
Copy the code

To get the property values of an AST node, we typically access the node and use the path.node.property method.

// the BinaryExpression AST node has properties: `left`, `right`, `operator`
BinaryExpression(path) {
  path.node.left;
  path.node.right;
  path.node.operator;
}
Copy the code

If you want to access the path inside the property, use the get method of the path object, passing the property as a string.

BinaryExpression(path) {
  path.get('left');
}
Program(path) {
  path.get('body.0');
}
Copy the code

A path has the same method to check the type of node:

BinaryExpression(path) {
  if (path.get('left').isIdentifier({ name: "n"{}))// ...}}Copy the code

Equivalent to:

BinaryExpression(path) {
  if (t.isIdentifier(path.node.left, { name: "n"{}))// ...}}Copy the code

Sometimes you need to traverse a path up the syntax tree until a condition is met.

For each parent path, callback is called with its NodePath as an argument, and when callback returns true, its NodePath is returned. .

path.findParent((path) = > path.isObjectExpression());
Copy the code

If you also need to traverse the current node:

path.find((path) = > path.isObjectExpression());
Copy the code

Find the nearest parent function or program:

path.getFunctionParent();
Copy the code

Walk up the syntax tree until you find the parent path in the list

path.getStatementParent();
Copy the code

If your plugin needs to stop running in some way, the easiest thing to do is to write it back as soon as possible.

BinaryExpression(path) {
  if(path.node.operator ! = ='* *') return;
}
Copy the code

If you are subtraversing a top-level path, you can use two provided API methods:

path.skip() skips traversing the children of the current path. path.stop() stops traversal entirely.

outerPath.traverse({
  Function(innerPath) {
    innerPath.skip(); // if checking the children is irrelevant
  },
  ReferencedIdentifier(innerPath, state) {
    state.iife = true;
    innerPath.stop(); // if you want to save some state and then stop traversal, or deopt}});Copy the code

BinaryExpression(path) {
  path.replaceWith(
    t.binaryExpression("* *", path.node.left, t.numberLiteral(2))); }Copy the code

  function square(n) {
- return n * n;
+ return n ** 2;
  }
Copy the code

Copy the code

FunctionDeclaration(path) { path.replaceWithSourceString(function add(a, b) { return a + b; }); }

```diff
- function square(n) {
-   return n * n;
+ function add(a, b) {
+   return a + b;
  }
Copy the code

** Note: </> This API is not recommended unless you are dealing with dynamic source strings, otherwise it is more efficient to parse the code outside the visitor.

If you want to insert an array like body
in the AST node property. It is similar to insertBefore/insertAfter, but you must specify the listKey (usually the body).

Copy the code

ClassMethod(path) { path.get(‘body’).unshiftContainer(‘body’, t.expressionStatement(t.stringLiteral(‘before’))); path.get(‘body’).pushContainer(‘body’, t.expressionStatement(t.stringLiteral(‘after’))); }

```diff
 class A {
  constructor() {+"before"
    var a = 'middle';
+   "after"}}Copy the code

Simply call replaceWith using parentPath: ‘path.parentPath

BinaryExpression(path) {
  path.parentPath.replaceWith(
    t.expressionStatement(t.stringLiteral("Anyway the wind blows, doesn't really matter to me, to me."))
  );
}
`Copy the code

  function square(n) {
- return n * n;
+ "Anyway the wind blows, doesn't really matter to me, to me.";
  }
Copy the code