Author: BoBoooooo

preface

You can’t talk about Babel without AST. The AST is an important thing to know, but because it involves compiling code, it is mostly handled by the framework itself, so it is often overlooked by developers themselves. I hope that through this article, I can bring you into AST and give full play to your imagination.

AST overview

You’ve heard AST all the time, but what exactly is AST?

AST is an Abstract Syntax Tree, which transforms the code we write into a Tree structure that can be recognized by the machine. It itself is made up of a bunch of nodes, each representing a structure in the source code. Different structures are distinguished by types. Common types include: Identifier, Expression, VariableDeclaration, FunctionDeclaration, etc.

AST structure

With the development of JavaScript, in order to unify the ECMAScript standard syntax expression. The ESTree Spec is a syntactic expression standard followed by the community.

ESTree provides common node types such as Identifier and Literal.

The node type

type instructions
File Files (top level node contains Program)
Program The entire program node (including the body attribute representing the program body)
Directive Directives (e.g. “use strict”)
Comment Code comments
Statement Statements (statements that can be executed independently)
Literal Literals (primitive data types, complex data types equivalent types)
Identifier Identifiers (variable names, attribute names, function names, parameter names, etc.)
Declaration Declarations (variable declarations, function declarations, Import, Export declarations, etc.)
Specifier Keywords (ImportSpecifier, ImportDefaultSpecifier, ImportNamespaceSpecifier, ExportSpecifier)
Expression expression

Public attribute

type instructions
type Type of an AST node
start Records the initial subscript of the node code string
end Records the end index of the node code string
loc Contains line and column attributes, which record the start and end column numbers respectively
leadingComments Opening note
innerComments Middle comment
trailingComments Closing note
extra Additional information

AST sample

Some students may ask, do you need to remember so many types? No, we can query the AST structure with the following two tools.

  • AST Explorer (Common)

  • AST visualization

With an example, I’ll give you a quick look at the AST structure.

function test(args) {
  const a = 1;
  console.log(args);
}
Copy the code

The above code declares a function called test with a parameter args.

In the function body:

  • Declared aconstType variableaAnd has a value of1
  • A console.log statement was executed

Paste the above code into the AST Explorer and the result should look like this:

Let’s move on to the internal structure, using const a = 1 as an example:

Variable declarations in the AST correspond to nodes of type VariableDeclaration. This node contains the kind and declarations mandatory attributes, which represent the declared variable type and variable content, respectively.

The declarations are an array if you are careful. Why is that? Variable declarations support const a=1,b=2, so we need to support multiple VariableDeclarators.

A node with Type VariableDeclarator represents a declaration like a=1, which contains the ID and init attributes.

Id is the Identifier where the name value corresponds to the variable name.

Init is the initial value that contains the type and value properties. Represents the initial value type and initial value, respectively. Here type is NumberLiteral, indicating that the initial value type is number.

Babel overview

Babel is a JavaScript compiler that is often used to perform AST related operations during actual development.

Babel workflow

Babel AST

The AST generated after Babel parses the code is based on ESTree, with minor modifications.

The official text reads as follows:

The Babel parser generates AST according to Babel AST format. It is based on ESTree spec with the following deviations:

  • Literal token is replaced with StringLiteral, NumericLiteral, BigIntLiteral, BooleanLiteral, NullLiteral, RegExpLiteral
  • Property token is replaced with ObjectProperty and ObjectMethod
  • MethodDefinition is replaced with ClassMethod
  • Program and BlockStatement contain additional directives field with Directive and DirectiveLiteral
  • ClassMethod, ObjectProperty, and ObjectMethod value property’s properties in FunctionExpression is coerced/brought into the main method node.
  • ChainExpression is replaced with OptionalMemberExpression and OptionalCallExpression
  • ImportExpression is replaced with a CallExpression whose callee is an Import node.

Babel core packages

kit instructions
@babel/core Core package for Babel transcoding, including the entire Babel workflow (integrated with @babel/types)
@babel/parser Parser that parses code into an AST
@babel/traverse Tools to traverse/modify the AST
@babel/generator Generator that restores the AST to code
@babel/types Contains methods to manually build the AST and check the AST node type
@babel/template String snippets can be converted into AST nodes 
npm i @babel/parser @babel/traverse @babel/types @babel/generator @babel/template -D
Copy the code

Babel plug-in

Babel plug-ins fall into two broad categories: syntax plug-ins and transformation plug-ins. The syntax plugin works on @babel/ Parser and parses code into abstract syntax trees (AST) (the official syntax plugin starts with babel-plugin-syntax). The transformation plugin works on @babel/core to transform the AST form. Most of the time we are writing transformation plug-ins.

Babel works with plug-ins. Plug-ins are like instructions that tell Babel what needs to be done. If there is no plug-in, Babel prints the code as it is.

The Babel plug-in is essentially writing visitors to traverse nodes on the AST. When a node of that type is encountered, the visitor does the processing, transforming the original code into the final code.

export default function (babel) {
  // @babel/types, used to generate AST nodes
  const { types: t } = babel;

  return {
    name: "ast-transform".// not required
    visitor: {
      Identifier(path) {
        path.node.name = path.node.name.split("").reverse().join(""); ,}}}; }Copy the code

This is the transform template code in the AST Explorer. All this code does is reverse all the node names of type identifiers entered in the code.

Writing a Babel plug-in is easy. All we need to do is pass back a visitor object that defines the function named Node Type. This function takes path and state.

Where path provides access to/manipulation of AST nodes. The path itself represents the object connected between the two nodes. For example, path.node can access the current node, and path.parent can access the parent node. Path.remove () removes the current node. See the following figure for specific apis. Other visible Handlebook.

Babel Types

The Babel Types module is a LoDash-style library for AST nodes that contains methods for constructing, validating, and transforming AST nodes.

Type judgment

Babel Types provides a method for determining node Types, and each type of node has its own method. See more of the Babel-types API.

import * as types from "@babel/types";

// Whether it is an identifier type node
if (types.isIdentifier(node)) {
  // ...
}

// Whether it is a numeric literal node
if (types.isNumberLiteral(node)) {
  // ...
}

// Whether it is an expression statement node
if (types.isExpressionStatement(node)) {
  // ...
}
Copy the code

Create a node

Babel Types also provides ways to create nodes of various Types, as shown in the example below.

Note: AST nodes generated by Babel Types need to be converted using @babel/ Generator to get the corresponding code.

import * as types from "@babel/types";
import generator from "@babel/generator";

const log = (node: types.Node) = > {
  console.log(generator(node).code);
};

log(types.stringLiteral("Hello World")); // output: Hello World
Copy the code

Basic data types

types.stringLiteral("Hello World"); // string
types.numericLiteral(100); // number
types.booleanLiteral(true); // boolean
types.nullLiteral(); // null
types.identifier(); // undefined
types.regExpLiteral("\\.js? $"."g"); / / regular
Copy the code
"Hello World"
100
true
null
undefined/\.js? $/gCopy the code

Complex data types

  • An array of
types.arrayExpression([
  types.stringLiteral("Hello World"),
  types.numericLiteral(100),
  types.booleanLiteral(true),
  types.regExpLiteral("\.js? $"."g"),]);Copy the code
["Hello World".100.true./.js? $/g];
Copy the code
  • object
types.objectExpression([
  types.objectProperty(
    types.identifier("key"),
    types.stringLiteral("HelloWorld")
  ),
  types.objectProperty(
    // The string type is key
    types.stringLiteral("str"),
    types.arrayExpression([])
  ),
  types.objectProperty(
    types.memberExpression(
      types.identifier("obj"),
      types.identifier("propName")
    ),
    types.booleanLiteral(false),
    // Calculate the value key
    true),]);Copy the code
{
  key: "HelloWorld"."str": [],
  [obj.propName]: false
}
Copy the code

JSX node

Creating a JSX AST node is slightly different from creating a data type node, and a diagram is organized here.

  • JSXElement

    types.jsxElement(
  types.jsxOpeningElement(types.jsxIdentifier("Button"), []),
  types.jsxClosingElement(types.jsxIdentifier("Button")),
  [types.jsxExpressionContainer(types.identifier("props.name")));Copy the code
    <Button>{props.name}</Button>
Copy the code

  • JSXFragment

    types.jsxFragment(types.jsxOpeningFragment(), types.jsxClosingFragment(), [
  types.jsxElement(
    types.jsxOpeningElement(types.jsxIdentifier("Button"), []),
    types.jsxClosingElement(types.jsxIdentifier("Button")),
    [types.jsxExpressionContainer(types.identifier("props.name"))]
  ),
  types.jsxElement(
    types.jsxOpeningElement(types.jsxIdentifier("Button"), []),
    types.jsxClosingElement(types.jsxIdentifier("Button")),
    [types.jsxExpressionContainer(types.identifier("props.age"))));Copy the code
    <>
  <Button>{props.name}</Button>
  <Button>{props.age}</Button>
</>
Copy the code

The statement

  • variableDeclaration

    types.variableDeclaration("const", [
  types.variableDeclarator(types.identifier("a"), types.numericLiteral(1)));Copy the code
    const a = 1;
Copy the code

  • functionDeclaration

    types.functionDeclaration(
  types.identifier("test"),
  [types.identifier("params")],
  types.blockStatement([
    types.variableDeclaration("const", [
      types.variableDeclarator(
        types.identifier("a"),
        types.numericLiteral(1)
      ),
    ]),
    types.expressionStatement(
      types.callExpression(types.identifier("console.log"), [
        types.identifier("params"(), [), [), [), [), [Copy the code
    function test(params) {
  const a = 1;
  console.log(params);
}
Copy the code

React functional components

Comprehensive above content, a little actual combat ~

We need to generate button.js code with Babel Types. Don’t know where to start at first glance?

// button.js
import React from "react";
import { Button } from "antd";

export default (props) => {
  const handleClick = (ev) = > {
    console.log(ev);
  };
  return <Button onClick={handleClick}>{props.name}</Button>;
};
Copy the code

Tip: First use the AST Explorer website to observe the AST tree structure. The code is then written layer by layer through Babel Types. Twice the work!

types.program([
  types.importDeclaration(
    [types.importDefaultSpecifier(types.identifier("React"))],
    types.stringLiteral("react")
  ),
  types.importDeclaration(
    [
      types.importSpecifier(
        types.identifier("Button"),
        types.identifier("Button")
      ),
    ],
    types.stringLiteral("antd")
  ),
  types.exportDefaultDeclaration(
    types.arrowFunctionExpression(
      [types.identifier("props")],
      types.blockStatement([
        types.variableDeclaration("const", [
          types.variableDeclarator(
            types.identifier("handleClick"),
            types.arrowFunctionExpression(
              [types.identifier("ev")],
              types.blockStatement([
                types.expressionStatement(
                  types.callExpression(types.identifier("console.log"), [
                    types.identifier("ev"),
                  ])
                ),
              ])
            )
          ),
        ]),
        types.returnStatement(
          types.jsxElement(
            types.jsxOpeningElement(types.jsxIdentifier("Button"), [
              types.jsxAttribute(
                types.jsxIdentifier("onClick"),
                types.jSXExpressionContainer(types.identifier("handleClick"))
              ),
            ]),
            types.jsxClosingElement(types.jsxIdentifier("Button")),
            [types.jsxExpressionContainer(types.identifier("props.name")),false(), [) (), [) (), [)Copy the code

Application scenarios

AST itself is widely used, such as the Babel plug-in (ES6 to ES5), build-time compression, CSS preprocessor compilation, WebPack plug-ins, and so on.

As you can see, when it comes to compilation, or the handling of the code itself, it’s all about the AST. Here are some common applications to see how they are handled.

Code conversion

// ES6 => ES5 let go to var
export default function (babel) {
  const { types: t } = babel;

  return {
    name: "let-to-var".visitor: {
      VariableDeclaration(path) {
        if (path.node.kind === "let") {
          path.node.kind = "var"; ,}}}}; }Copy the code

babel-plugin-import

Under the CommonJS specification, when we need to introduce ANTD on demand, we usually use this plug-in.

This plug-in does the following:

// Through the ES specification, the Button component is introduced by name
import { Button } from "antd";
ReactDOM.render(<Button>xxxx</Button>);

// The Babel compile phase is converted to require implementation on demand
var _button = require("antd/lib/button");
ReactDOM.render(<_button>xxxx</_button>);
Copy the code

For a brief analysis, the core processing: replace the import statement with the corresponding require statement.

export default function (babel) {
  const { types: t } = babel;

  return {
    name: "import-to-require".visitor: {
      ImportDeclaration(path) {
        if (path.node.source.value === "antd") {
          // var _button = require("antd/lib/button");
          const _botton = t.variableDeclaration("var", [
            t.variableDeclarator(
              t.identifier("_button"),
              t.callExpression(t.identifier("require"), [
                t.stringLiteral("antd/lib/button"))));// Replace the current import statementpath.replaceWith(_botton); ,}}}}; }Copy the code

TIPS: EsM specification files are currently included in the ANTD package and can be imported on demand depending on the WebPack native TreeShaking implementation.

LowCode Visual coding

LowCode is still a hot area on the front end. The current mainstream approach is roughly the following two.

  • Schema driven

    The current mainstream approach is to describe the configuration of a form or table as a Schema, and the visual designer is based on the Schema driver, combined with the drag-and-drop ability, to build quickly.

  • AST drive

    Online compilation and coding via CloudIDE, CodeSandbox and other browsers. Plus visual designer, the final realization of visual coding.

The general process is shown in the figure above. Since code modification is involved and AST operations are indispensable, Babel’s capabilities can be used again.

Suppose the initial code for the designer looks like this:

import React from "react";

export default() = > {return <Container></Container>;
};
Copy the code

At this point, we dragged a Button into the designer. According to the flow in the figure above, the core AST modification process is as follows:

  • Add import statementimport { Button } from "antd";
  • will<Button></Button>Insert into<Container></Container>

Without further ado, go directly to the code:

import traverse from "@babel/traverse";
import generator from "@babel/generator";
import * as parser from "@babel/parser";
import * as t from "@babel/types";

/ / the source code
const code = ` import React from "react"; export default () => { return 
      ; }; `;

const ast = parser.parse(code, {
  sourceType: "module".plugins: ["jsx"]}); traverse(ast, {// 1. Add an import statement at the top of the program
  Program(path) {
    path.node.body.unshift(
      t.importDeclaration(
        // importSpecifier indicates the named import. The corresponding anonymous import is ImportDefaultSpecifier
        Import {Button as Button} from 'antd'
        Import {Button} from 'antd'
        [t.importSpecifier(t.identifier("Button"), t.identifier("Button"))],
        t.stringLiteral("antd"))); },// Access the JSX node and insert Button
  JSXElement(path) {
    if (path.node.openingElement.name.name === "Container") {
      path.node.children.push(
        t.jsxElement(
          t.jsxOpeningElement(t.jsxIdentifier("Button"), []),
          t.jsxClosingElement(t.jsxIdentifier("Button")),
          [t.jsxText("Button")].false)); }}});const newCode = generator(ast).code;
console.log(newCode);
Copy the code

The results are as follows:

import { Button } from "antd";
import React from "react";
export default() = > {return (
    <Container>
      <Button>button</Button>
    </Container>
  );
};
Copy the code

ESLint

Is custom ESlint-rules, which essentially access AST nodes, similar to the way the Babel plugin is written?

module.exports.rules = {
  "var-length": (context) = > ({
    VariableDeclarator: (node) = > {
      if (node.id.name.length <= 2) {
        context.report(node, "Variable name needs to be longer than 2"); }}})};Copy the code

Code2Code

For example, the process of Vue To React is similar To that of ES6 => ES5. The Vue AST => is converted To React AST => The React code is output using the Vue template-Compiler.

Vue-to-react for those interested

Other multi-terminal frameworks: one piece of code => multi-terminal, the general idea is the same.

conclusion

In the actual development, the situation encountered is often more complex, we suggest that more documents, more observation, heart to feel ~

Refer to the article

  1. babel-handlebook
  2. @babel/types
  3. First visit AST by making babel-Plugin
  4. @babel/types Deep application

This article is published by NetEase Cloud Music Technology team. Any unauthorized reprinting of this article is prohibited. We recruit technical positions all year round. If you are ready to change your job and you like cloud music, join us!