Build the column series directory entry

Zhou Shitie, Front end dislocation engineer of medical Support group, Front End Technology Department of Micro Medical

preface

When we write business code, AST is probably used by so few people that most students don’t know AST well. Some students have learned, but do not go to practice, after a period of time and forget about. If you look at this, you’ll see it’s you. I heard that your circle is now writing articles to make up stories, from time to time, but also dot facial expressions. Is it true? As the head of the company, I won’t let go.

This article will study AST through the following aspects

  1. Basic knowledge of

    • What is the AST

    • What is AST useful for

    • How an AST is generated

  2. Practical examples

    • Remove the debugger

    • Modify the console.log argument executed in the function

  3. conclusion

Basic knowledge of

ASTWhat is the

First post the official explanation

In computer science, an abstract syntax tree (AST for short), or syntax tree, is a tree-like representation of the abstract syntactic structure of source code, specifically the source code of a programming language.

To help you understand abstract syntax trees, let’s look at a concrete example.

var tree = 'this is tree'
Copy the code

The JS source code will be converted into the abstract syntax tree below

{
  "type": "Program"."start": 0."end": 25."body": [{"type": "VariableDeclaration"."start": 0."end": 25."declarations": [{"type": "VariableDeclarator"."start": 4."end": 25."id": {
            "type": "Identifier"."start": 4."end": 8."name": "tree"
          },
          "init": {
            "type": "Literal"."start": 11."end": 25."value": "this is tree"."raw": "'this is tree'"}}]."kind": "var"}]."sourceType": "module"
}
Copy the code

You can see that a statement consists of several lexical units. This lexical unit is like 26 letters. Create a hundred thousand words, through the combination of different words and can write different content of the article.

For details of the lexical units, see the AST object documentation or see the Advanced Front-end Basics of JavaScript Abstract Syntax Tree AST, which lists syntax tree nodes and explanations.

A tool for online AST converters is recommended. Try the downconversion yourself on this website. Click on the word in the statement and the node in the abstract syntax tree on the right is selected, as shown below:

What is AST useful for

  • IDE error prompts, code formatting, code highlighting, code completion, etc
  • JSLint, JSHint checks for code errors or styles, etc
  • Webpack, rollup for code packaging, etc
  • CoffeeScript, TypeScript, JSX, etc. Convert to native Javascript.
  • Compile the vUE template and react template

How an AST is generated

By now, you should have an idea of what an abstract syntax tree looks like. So how is an AST generated?

The AST parsing process is divided into two steps

  • Lexical Analysis: The input source code strings are scanned to generate a series of tokens. These lexical units include numbers, punctuation marks, operators, etc. Lexical units are independent of each other, meaning that at this stage we don’t care how each line of code is put together.
  • Syntax Analysis: To establish relationships between parsing Syntax units

Var tree = ‘this is tree

Formal understanding

Lexical analysis

After lexical analysis, input source code strings are scanned to generate a series of tokens. These lexical units include numbers, punctuation marks, operators, etc

Syntax analysis

In the grammar analysis stage, the list of tokens generated in the previous stage will be converted into the AST as shown in the following figure.

Informal understanding

Solemnly declare: I zhou someone language rarely pass, roughly meaning can understand good.

Example: “It’s a pig.”

Lexical analysis

After lexical analysis, input source code strings are scanned to generate a series of tokens. These lexical units include numbers, punctuation marks, operators, etc

Syntax analysis

In the grammar analysis stage, the list of tokens generated in the previous stage is transformed into the AST as shown in the figure below

JsParser

JavaScript Parser is a Parser that converts JavaScript source code into abstract syntax trees.

  • acorn
  • esprima
  • traceur
  • @babel/parser

Practical examples

Example 1: Go to the debugger

The source code:

function fn() {
  console.log('debugger')
  debugger;
}
Copy the code

How to get rid of the debugger

  1. First, convert the source code toAST
  2. traverse**AST**Find the node of**debugger**And delete the node
  3. Will be convertedASTgeneratingJScode

Copy the source code toOn line AST converterAnd click on the left areadebuggerYou can see the one on the leftdebuggerThe node is selected. So you just pick the right onedebuggerAbstract syntax tree nodedeleteWill do. ​

This is a simple example, so let’s go straight to the code.

For this example I used @babel/ Parser, @babel/traverse, and @babel/ Generator for parsing, conversion, and generation respectively.

​

const parser = require('@babel/parser');
const traverse = require("@babel/traverse");
const generator = require("@babel/generator");

/ / the source code
const code = ` function fn() { console.log('debugger') debugger; } `;

// 1. Source code is parsed into ast
const ast = parser.parse(code);


/ / 2. The conversion
const visitor = {
  // traverse will traverse tree nodes whenever the type of the node is present in the visitor object
  DebuggerStatement(path) {
    // Delete the abstract syntax tree node
    path.remove();
  }
}
traverse.default(ast, visitor);

/ / 3. Generate
const result = generator.default(ast, {}, code);

console.log(result.code)

// 4. Log output

// function fn() {
// console.log('debugger');
// }
Copy the code

The core Babel logic processing is in the visitor. Traverse traverses the tree nodes, and whenever the type of the node is present in the visitor object, the method corresponding to that type is called, where path.remove() is called to remove the current node. Some API of path used in demo can refer to Babel-Handbook.

Example 2: Modify the console.log argument executed in the function

Sometimes we log in a function, but we want to see which function logs in the console. In this case, we can use AST to parse, transform, and generate the desired code.

The source code:

Function funA() {console.log(1)} function funA() {console.log('from function funA:', 1)}Copy the code

Before we start coding, we need to get our heads around it. This time you need to use the online AST converter to analyze.

The tool found that to implement this case, you simply insert the segment node in front of Arguments.

So let’s do the same thing we did in example 1

  1. use@babel/parserParse the source code intoast
  2. Listening to the@babel/traverseTo traverse theCallExpression
  3. After firing, determine if the method executed isconsole.logWhen toarguments unshiftaStringLiteral
  4. Will be convertedastThe generated code

Parsing JS code into an AST is consistent with generating JS code from an AST and going to the Debugger example, which is not described here.

First listen for CallExpression traversal

const visitor = {
  CallExpression(path) {
    // console.log(path)}}Copy the code

Looking at the tree parsed by the online AST converter, we just need to determine the presence of object console and property in the callee of path. You can then unshift a StringLiteral to the current path arguments

The types object here uses a new package, @babel/types, to determine the type.

The isMemberExpression used above, isIdentifier getFunctionParent, stringLiteral can be found in the Babel – faced document, this article explains.

const visitor = {
  // When iterating to CallExpression
  CallExpression(path) {
    const callee = path.node.callee;
    // Determine whether the currently executing function is a combined expression
    if (types.isMemberExpression(callee)) {
      const { object, property } = callee;
      if (types.isIdentifier(object, { name: 'console' }) && types.isIdentifier(property, { name: 'log'{}))// Find the nearest parent function or program
        const parent = path.getFunctionParent();
        const parentFunName = parent.node.id.name;
        path.node.arguments.unshift(types.stringLiteral(`from function ${parentFunName}`)}}}}Copy the code

conclusion

As mentioned in the preface, AST is seldom used in our daily work, so that most students do not know AST well. But knowing the AST helps us better understand development tools, compilers, and tools that make our code more efficient. Remember I had a problem on the previous front end team. Our project was an SSR project, and the life cycle of executing on the server side did not allow code to be executed on the client side. However, team members sometimes accidentally write, resulting in degraded server rendering. Before learning AST, IN order to solve this problem, I wrote a Loader to match and verify through the re. At that time, I was really driven to death. The re needs to be adapted to various scenarios. After I learned the AST, I wrote an ESLint plugin to implement client-side code validation.

reference

Advanced front End Basics -JavaScript Abstract Syntax tree AST