Javascript Abstract Syntax Tree part 2

Author: Chen Xiaoqiang

The last part introduced the basics of AST. This part introduces the application of AST

Three key points of AST applications

You need a parser to convert the code into an AST
A traverser is required to traverse the AST and easily add, delete, modify, and check AST nodes
You need a code generator that can convert the AST into code

Esprima and Babel

Two common toolkits that meet the above three points are Esprima and Babel

Esprima related packages and use are as follows

const esprima = require('esprima');   // code => ast
const estraverse = require('estraverse'); / / ast traversal
const escodegen = require('escodegen'); // ast => code
let code = 'const a = 1';
const ast = esprima.parseScript(code);
estraverse.traverse(ast, {
    enter: function (node) {
        // Node operation}});const transformCode = escodegen.generate(ast);
Copy the code

Babel related packages and usage are as follows

const parser = require('@babel/parser');  //code => ast
const traverse = require('@babel/traverse').default; // ast traversal, node add, delete, change, scope processing, etc
const generate = require('@babel/generator').default; // ast => code
const t = require('@babel/types'); // Lodash library for AST nodes, node construction, validation, etc
let code = 'const a = 1';
let ast = parser.parse(sourceCode);
traverse(ast, {
  enter (path) { 
    // Node operation}})const transformCode = escodegen.generate(ast);
Copy the code

Babel is currently much better than Esprima, both ecologically and documentatively, so it is recommended that you use the Babel tool, which is used for the examples in this article.

Use the Babel tool to manipulate the AST

As shown in the previous section

@babel/parserUsed to convert code to AST
@babel/traverseIt is used to traverse the AST, including node addition, deletion, modification, and scope
@babel/generatorUsed to convert the AST into code
@babel/typesLodash library for AST node operation, node construction, validation, etc

See Babel Manual for more APIS [1]

The following is a simple case to introduce how to operate the AST. Note that the case is only an example. As the space only explains some boundary problems, you need to consider them thoroughly in the actual development process.

Case 1: Remove console.log() from code

The implementation code

const parser = require('@babel/parser');
const traverse = require('@babel/traverse').default;
const generate = require('@babel/generator').default;
const t = require('@babel/types');
let sourceCode = ` function square(n) { console.log(n); console.warn(n); return n * n; } `
let ast = parser.parse(sourceCode);
traverse(ast, {
 CallExpression(path) {
  let { callee } = path.node;
  ifCallee.type === 'MemberExpression' && callee.object.name === 'console'&& callee.property.name ===' log ') {path.remove();Log (); // If you want to remove the global}}})console.log(generate(ast).code);
Copy the code

The processing results

function square(n) {
- console.log(n);
  console.warn(n);
  return n * n;
}
Copy the code

This case involves knowledge points

How do I traverse specific nodes with traverse
Identifies console.log() as a function call expression in the specification, and the node type isCallExpression.
The console. The log is itselfcalleeIs a method on the object Console, soconsole.logIs a member expression of typeMemberExpression.
MemberExpressionThere is one according to the specificationobjectProperty represents the object being accessed, and there is onepropertyRepresents the visiting member.
throughpath.remove()The API can delete nodes.
Identification of code nodes can be aided by https://astexplorer.net/. Pay attention to the choicebabylon7That is babe7 corresponds to@babel/parser

Case 2: Variable confusion

The implementation code

const parser = require('@babel/parser');
const traverse = require('@babel/traverse').default;
const generate = require('@babel/generator').default;
const t = require('@babel/types');
let sourceCode = ` function square(number) { console.warn(number); return number * number; } `
let ast = parser.parse(sourceCode);
traverse(ast, {
  FunctionDeclaration(path) {
    let unia = path.scope.generateUidIdentifier("a");
    path.scope.rename("number",unia.name); }})console.log(generate(ast).code);
Copy the code

The processing results

-function square(number) {
+ function square(_a) {
- console.warn(number);
+ console.warn(_a);
- return number * number;
+ return _a * _a;
}
Copy the code

This case involves knowledge points

path.scopeSaves information about the current scope
Variable names in scope can be changed in batches through the API
throughpath.scopeYou can obtain the unique identifier of the current scope, avoiding variable name conflicts

Case 3: Transform the arrow function and remove unused arguments

The implementation code

const parser = require('@babel/parser');
const traverse = require('@babel/traverse').default;
const generate = require('@babel/generator').default;
const t = require('@babel/types');
let sourceCode = ` new Promise((resolve,reject)=>{ setTimeout(()=>{ resolve(1); }, 200)}); `
let ast = parser.parse(sourceCode);
traverse(ast, {
  ArrowFunctionExpression (path) { 
    let { id, params, body } = path.node;
    for(let key in path.scope.bindings){   // Note the arrow function's this property. If you find a call to this in the function body, you need to bind this to its parent's scope in the current scope
      if(! path.scope.bindings[key].referenced){ params = params.filter(param= >{
          return param.name!==key;
        })
      }
    }
  path.replaceWith(t.functionExpression(id, params, body)); 
  }
})

console.log(generate(ast).code);
Copy the code

The processing results

-new Promise((resolve,reject)=>{
+new Promise(function(resolve){
- setTimeout(()=>{
+ setTimeout(function(){
    resolve(1);
  },200)
});
Copy the code

This case involves knowledge points

Arrow function node:ArrowFunctionExpression
Path. scope identifies variables that are referenced, whether they are referenced, and by which paths
@babel/types makes it easy to build nodes of any type
throughpath.replaceWith()Node replacement can be performed

Case 4: Tree-shaking of JINGdong Shopping Miniprogram

Delete redundant code in the small program, some code examples are as follows

const parser = require('@babel/parser');
const traverse = require('@babel/traverse').default;
const generate = require('@babel/generator').default;
const t = require('@babel/types');
let sourceCode = ` export function square (x) { return x * x; } export function cube (x) { return x * x * x; } `
let ast = parser.parse(sourceCode);
traverse(ast, {
  ExportNamedDeclaration (path) {
    let unused = ['cube']   // With webpack, we can find out which of the exported methods are unused
    let { declaration = {} } = path.node;
    if (declaration.type === 'FunctionDeclaration') {
      unused.forEach(exportItem= > {
        // references=1 indicates that there is only one reference, that is, the reference of export, and it is not invoked elsewhere
        if (declaration.id.name === exportItem && path.scope.bindings[exportItem].references === 1) { path.remove(); }}); }}})console.log(generate(ast).code);
Copy the code

The processing results

export function square (x) {
    return x * x;
}
-export function cube (x) {
- return x * x * x;
-}
Copy the code

This case involves knowledge points

Export nodes:ExportNamedDeclaration

Case 5: Converting code to AN SVG flowchart

Flowchart; JS-code-to-SVG-flowchart this case is an interesting open-source project of Git. Flowchart for converting code to SVG using AST; see jS-code-to-SVG-flowchart [2]

Try it out: Demo [3]

As you can see from the examples above, we can do a lot of interesting things with the AST

AST applications in other languages

In addition to Javascript, AST is widely used in other languages such as HTML, CSS, SQL, etc. Here you can find the parser for the corresponding language to open the door of AST.

conclusion

In the AST site above, you can see that the HTML parser has a VUE option. Those of you who have read the vue source code should know that vue templates will first convert the template to the AST and then generate the Render Function to generate VirtualDOM before converting to HTML. We don’t use AST as much as we normally do, but we see it everywhere: Babel, Webpack, ESLint, Taro, and so on. I hope to inspire students to produce more excellent tools and projects based on AST in their teams.

Relationship between short-term flowsheet and flowsheet; Relationship between short-term flowsheet and flowsheet; References [1] Babel Handbook [2] JS-code -to- SVG-Flowchart [3] Demo

If you think this article is valuable to you, please like it and follow our official website and WecTeam:

Javascript Abstract Syntax Tree part 2

Three key points of AST applications

Esprima and Babel

Use the Babel tool to manipulate the AST

Case 1: Remove console.log() from code

Case 2: Variable confusion

Case 3: Transform the arrow function and remove unused arguments

Case 4: Tree-shaking of JINGdong Shopping Miniprogram

Case 5: Converting code to AN SVG flowchart

AST applications in other languages

conclusion

Related Posts

The smallest number of k

Send you N interesting input box!

Do you really understand script tags and link tags?