How to understand AST and implement handwritten Babel plug-in

The program appears to be the characters in the text file. The computer first conducts lexical and grammatical analysis on it, and then reexpresses the program in a low-level language that the computer can understand, among which AST is a key point.

The level is limited, briefly speaking the AST understanding, and through handwritten Babel plug-in to deepen understanding.

How does JS compile and execute

• Lexical analysisTo generate the original codeTokens
• Syntax analysisThat will beTokensgenerateAbstract Syntax Tree (AST)
• Precompile, when the JavaScript engine parses the script, it processes all declared variables and functions at precompile time! And is pre-declared variables, and then pre-defined functions!
• Explain execution. During execution, the JavaScript engine is strictly scoped, and the scope of JavaScript variables and ordinary functions is determined at definition time, not at execution time.

The last two are not introduced, first of all, a brief understanding of lexical analysis and grammatical analysis:

Lexical analysis

Word analysis: Convert the original code into an array of smallest Tokens. The professional word for the smallest array of Tokens is Tokens.

For example: var a = ‘hello’

After lexical analysis, Tokens are expressed as follows:

[{type: "Keyword".value: "var"}, {type: "Identifier".value: "a"}, {type: "Punctuator".value: "="}, {type: "String".value: "'hello'",},];Copy the code

It can be generated online via esprima.

Syntax analysis

Tokens are encoded according to syntax rules and output abstract syntax trees that are actually JSON objects

JS to generate abstract syntax tree site: astExplorer

For example: var a = ‘hello’

After parsing, the abstract syntax tree is generated as follows:

{
  "type": "Program"."body": [{"type": "VariableDeclaration"."declarations": [{"type": "VariableDeclarator"."id": {
            "type": "Identifier"."name": "a"
          },
          "init": {
            "type": "Literal"."value": "hello"."raw": "'hello'"}}]."kind": "var"}]."sourceType": "script"
}
Copy the code

Generate with esprima or AstExplorer.

It can also be generated with the esprima library:

const esprima = require("esprima");
const createAst = (code) = > esprima.parseModule(code, { tokens: true });
const code = `var a="hello"`;
const ast = createAst(code);
console.log(ast);
Copy the code

Notice here that the code generates the AST and then compiles it, but you can also use the AST to convert it into code

It may seem like the AST is a bit distant from everyday code, but it’s been around:

• webpack 和 lintThe core is throughASTReview and analyze the code
• UglifyJSthroughASTImplement code compression obfuscation
• babelThe core implements code conversion through the AST

The following is a step by step practice

Modify the AST to generate new code

How does an AST turn into code

In the same way that esprima converts code to AST, escodeGen converts AST to code.

const esprima = require("esprima");
// code becomes the AST function
const createAst = (code) = > esprima.parseModule(code, { tokens: true });

const escodegen = require("escodegen");
// AST becomes a function of code
let astToCode = (ast) = > escodegen.generate(ast);

const code = `var a="hello"`;
const ast = createAst(code);

const newCode = astToCode(ast);
// var a = 'hello';
console.log(newCode);
Copy the code

How to modify AST

The AST is actually a JSON object, and of course you can modify the AST as much as you want.

For example, if you want to change var a=’hello’ to const a=’hello’, look at the AST of both codes and change the former to be the same as the latter.

Look closely only, kind there is different, so it is simple!

/ /... With the above
const code = `var a="hello"`;
const ast = createAst(code);

// Modify kind directly
ast.body[0].kind = "const";

const newCode = astToCode(ast);
// const a = 'hello';
console.log(newCode);
Copy the code

But it’s easy to write things like this, and when code gets complicated, nested, or changed, it’s hard to write things like this, using another tool library, Estraverse, that travels around to find out what’s needed, and writes about it

How do I traverse the AST

Whereas the AST is a JSON object that can be understood as a tree, estraverse loops the AST in a depth-first fashion.

Anything with all attributes of Type is a node.

Estraverse averse averse averse averse averse averse averse averse averse averse averse

const esprima = require("esprima");
// code becomes the AST function
const createAst = (code) = > esprima.parseModule(code, { tokens: true });

const code = `var a="hello"`;
const ast = createAst(code);

/ / traverse
const estraverse = require("estraverse");
let depth = 0;
// The deeper the level, the more indentation
const createIndent = (depth) = > "".repeat(depth);

estraverse.traverse(ast, {
  enter(node) {
    console.log(`${createIndent(depth)} ${node.type}Enter the `);
    depth++;
  },
  leave(node) {
    depth--;
    console.log(`${createIndent(depth)} ${node.type}Leave `); }});Copy the code

In contrast to the generated AST, it is clear that this is the process of deep traversal:

Program goes into VariableDeclaration goes into VariableDeclarator goes into Identifier goes into Identifier leaves Literal goes into Literal leaves VariableDeclaration leaves ProgramCopy the code

Modify AST using estraverse

Var a=’hello’; Var b=’world’ var = const

/ /.. CreateAst astToCode function same as above
const estraverse = require("estraverse");

const code = `var a='hello'; var b='world'`;
const ast = createAst(code);

estraverse.traverse(ast, {
  enter(node) {
    // All var variables are const
    if (node.kind === "var") {
      node.kind = "const"; }}});const newCode = astToCode(ast);
// const a = 'hello'; const b = 'world';
console.log(newCode);
Copy the code

Through estraverse, an old AST is conveniently averse to adding, subtracting, and adapting its nodes to generate the desired new AST!

Use babel-types to quickly generate an AST

To generate an AST, a code must be generated first.

babel-types!!!!

The AST is made up of nodes, and you can simply generate nodes that are described accordingly.

Arbitrary nodes can be generated with babel-types!

Such as a const a = ‘hello’, b = ‘world’ :

{
  "type": "VariableDeclaration"."declarations": [{"type": "VariableDeclarator"."id": {
        "type": "Identifier"."name": "a"
      },
      "init": {
        "type": "Literal"."value": "hello"."raw": "'hello'"}}, {"type": "VariableDeclarator"."id": {
        "type": "Identifier"."name": "b"
      },
      "init": {
        "type": "Literal"."value": "world"."raw": "'world'"}}]."kind": "const"
}
Copy the code

Init + ID => VariableDeclaration + kind => VariableDeclaration

const kind = "const";
const id = t.identifier("a");
const init = t.stringLiteral("hello");

const id2 = t.identifier("b");
const init2 = t.stringLiteral("world");

// a=hello
const variableDeclarator = t.variableDeclarator(id, init);
const variableDeclarator2 = t.variableDeclarator(id2, init2);

const declarations = [variableDeclarator, variableDeclarator2];
const ast = t.variableDeclaration(kind, declarations);

// The ast is expanded above
console.log(ast);
Copy the code

The type name is usually the corresponding API name, and other attributes of the same class are API parameters.

How to write a Babel plugin

One of the most important features of Babel is the ability to convert ECMAScript 2015+ code into backwardly compatible JavaScript syntax.

Because there are so many conversions, Babel generally takes the form of a plug-in that connects the plug-in through babel-core to transform the code.

If we convert const to var:

const t = require("babel-types");

/ / the plugin
const ConstPlugin = {
  visitor: {
    // Path is the path of the corresponding type. Node can also be understood as an object describing const expressions
    // When iterating through the VariableDeclaration, change kind to var
    VariableDeclaration(path) {
      let node = path.node;
      console.log(node);
      if (node.kind === "const") {
        node.kind = "var"; ,}}}};// Try writing the plugin
const code = `const a='hello'`;

const babel = require("babel-core");
let newCode = babel.transform(code, { plugins: [ConstPlugin] });
// var a = 'hello';
console.log(newCode.code);
Copy the code

Handwriting implementation plug-in babel-plugin-arrow-functions

Babel-plugin-arrow functions convert arrow functions to normal functions

var a = (s) = > {
  return s;
};
var b = (s) = > s;

/ / into
var a = function (s) {
  return s;
};
var b = function (s) {
  return s;
};
Copy the code

Let’s take a look at the difference between an arrow function and a normal AST:

• Notice that the body type is notBlockStatementIf so, change it toBlockStatement; If it is, don’t worry about it
• The type isArrowFunctionExpressionCan be replaced byFunctionExpressionnode

Since we are writing the Babel plug-in, we must use babel-core to convert to new code:

// npm i babel-core babel-types
const t = require("babel-types");

/ / the plugin
const ArrowPlugin = {
  visitor: {
    ArrowFunctionExpression(path) {
      let node = path.node;
      let { params, body } = node;
      // Generate a code block if not a code block
      if(! t.isBlockStatement(body)) {// Generate a return statement
        let returnStatement = t.returnStatement(body);
        // Create code block statements
        body = t.blockStatement([returnStatement]);
      }
      // Use t to generate an expression object equivalent to a normal function
      const functionJSON = t.functionExpression( null, params, body, false.false );
      // Replace the current pathpath.replaceWith(functionJSON); ,}}};// Try writing the plugin
const arrowJsCode = `var a = (s) => { return s; }; var b = (s) => s; `;

const babel = require("babel-core");
let functionJSCode = babel.transform(arrowJsCode, { plugins: [ArrowPlugin] });
// var a = function (s) { return s; }; var b = function (s) { return s; };
console.log(functionJSCode.code);
Copy the code

reference

• Javascript You Didn’t Know, Part 1
• JavaScript parsing, AST, V8, JIT
• Walk you through the AST abstract syntax tree
• AST Abstract Syntax tree – one of the most basic javascript essentials that 99% of people don’t know at all
• A quick start guide to Babel