Babel plug-in practice (1) Analysis of the principle of Babel compilation

preface

We all know that before the advent of front-end build tools, front-end projects were implemented using syntaxes recognized by es5 browsers. (jquery, ES5…) . With the development of front-end technology (es6 even introduced updated syntaxes), browsers were unable to recognize these new syntaxes. Then there are the compile-build tools, of which Babel plays an important role. So what is Babel?

What’s a Babel?

The official introduction

Babel is a JavaScript compiler.

Babel is a toolchain for converting code written in ECMAScript 2015+ syntax into backwardly compatible JavaScript syntax so it can run in current and older versions of browsers or other environments.

Simply put, in order for javascript to work in the browser, you need to translate the syntax that the browser doesn’t understand into a pre-release that the browser recognizes. This is what Babel does. In computer programming, this step is also called compilation.

In fact, compilation involves a lot of things, interested students can understand the “principle of compilation”, compilation principle mainly includes lexical analysis, syntax analysis, semantic analysis, intermediate code generation, code optimization, object code generation these steps, here will not do too much introduction, here omitted one million words…

The compilation process in Babel’s workflow and compilation principles is relatively simple. We can summarize the steps as follows:

Lexical analysis
Syntax analysis
Code conversion
Code generation

The overall workflow of Babel is shown below:

The two steps of lexical analysis and pre-analysis can be combined into a parse process

As you can see from the diagram above, one of the most important things to compile from start to finish is the knowledge of the abstract syntax tree /AST, or AST for short, which Babel relies on throughout the process of compiling code.

Abstract Syntax Tree (AST)

Abstract syntax trees are a bridge between high-level programming languages (Java, JavaScript, etc.) and machine languages. The parser analyzes the code string according to the ECMAScript standard “JavaScript Language Specification”, divides it into lexical units, and then iterates through each lexical unit for parsing to construct the AST. We analyze the principle through the following code:

let age = 10;
age = age + 20;
Copy the code

Lexical analysis

The lexical analysis stage is the “word segmentation” of the source code. It receives a piece of source code and then executes a tokenize function to divide the code into things called tokens. Tokens are an array of code fragments like numbers, punctuation marks, operation symbols, and so on. For example:

Here we use an online tool to analyze the above code and the results are as follows: a lexical analysis tool

[
    { "type": "Keyword", "value": "let"},
    { "type": "Identifier", "value": "age"},
    { "type": "Punctuator", "value": "="},
    { "type": "Numeric", "value": "10"},
    { "type": "Punctuator", "value": ";"},
    { "type": "Identifier", "value": "age"},
    { "type": "Punctuator", "value": "="},
    { "type": "Identifier", "value": "age"},
    { "type": "Punctuator", "value": "+"},
    { "type": "Numeric", "value": "20"},
    { "type": "Punctuator", "value": ";"}
]
Copy the code

As you can see from the lexical analysis, the end result is to parse the code into words (let, age, +, =, etc.) using the Babel-tokenizer method

Syntax analysis

After lexical analysis, grammar analysis translates tokens into AST. Babel transforms tokens into AST

The AST abstract syntax tree is the core concept of the Babel plug-in. It is also used when writing custom Babel plug-ins, because the code transformation is actually performed on the nodes of the AST syntax tree

Here is a recommended tool for generating AST syntax trees online

The generated AST is too long to be shown here, so you can try it online if you are interested.

An AST tree, as the name implies, is a typical data type of data structure – tree, so we also know that trees have a root node, but also many child nodes. The AST syntax tree will have a root node whose type value is Program, as follows

{
  "type": "Program",
  "start": 0,
  "end": 29,
  "body": [],
  "sourceType": "module"
}
Copy the code

After observing the child nodes, in fact, the child nodes (including the root node) have the same data structure, as follows

{ "type": "VariableDeclaration", "start": 0, "end": 13, "declarations": [...] , "kind": "let" }Copy the code

{
  type: "Identifier",
  name: ...
}
Copy the code

{ type: "BinaryExpression", operator: ... , left: {... }, right: {... }}Copy the code

These are just a few examples of the different types of nodes (note: some attributes have been removed for simplification purposes) that make up the AST syntax tree and together describe program syntax for static analysis.

Each node has a type field representing the node type, and additional attributes are defined to further describe the node type.

Babel compilation

Babel compilation process code demo

We have also shown a flowchart for compiling code with Babel. Let’s see how it works

Create an empty project as follows:

Create a folder and use NPM init -y to create package.json

Then create the SRC /index.js file under the project

let name = "hello babel";
console.log(name);
Copy the code

Add execution scripts in package.json

"scripts": {
  "build": "node src/index.js"
}
Copy the code

To make it easier for us to interrupt debug later, here we use the vscode tool to generate a launch.js file for us and add our own launch configuration

My launch.js content is as follows

{" version ":" 0.2.0, "" configurations: [{" type" : "pwa - node", "request" : "launch", "name" : "Debug", "runtimeExecutable": "npm", "restart": true, "console": "integratedTerminal", "runtimeArgs": ["run-script", "build"], } ] }Copy the code

Specific configuration, please search your friends…

Then we click on the breakpoint where we want the breakpoint to create the breakpoint, click the debug button above to run it, as shown below

More about vscode debugging tools please learn by yourself, here do not do too much about

Now back to compiling Babel, we need to install three plug-ins that are officially provided by Babel

npm install -D @babel/parser @babel/generator @babel/traverse
Copy the code

@babel/ Parser Official address
@babel/ Traverse official address
@babel/generator Official address

To look at the simple usage of these three packages, modify the SRC /index.js code as follows

const Parser = require("@babel/parser") const traverse = require("@babel/traverse").default; const generator = require("@babel/generator").default; // Source code const compilerCode = 'let age = 10; age = age + 20; Parse (compilerCode, {}) const AST = parser. parse(compilerCode, {}); // Traverse (AST) nodes on the AST syntax tree; // ast syntax tree generates final code const codeObj = generator(ast, {}, compilerCode); console.log(codeObj.code);Copy the code

This is a simple code example of how Babel compiles code.

The compiled code produces the following

let age = 10;
age = age + 20;
Copy the code

There is no difference here compared to the source code because we are not using plug-ins to manipulate the code (compress, obfuscate, optimize, etc.)

Babel /traverse package The traverse method receives the AST abstract syntax tree and traverses it (deep traversal). You can add, update, and remove nodes during this process. This is the most complex process in Babel or any other compiler, and this is where plug-ins will come in. In the plug-ins section, we will look at the code that the @babel/generator method receives from the AST abstract syntax tree as a string. A sourceMap is also created (sourceMap, which controls whether the sourceMap is generated based on the parameters passed in)

As mentioned above, @babel/traverse traverse traverse traverse traverse traverse traverse through the entire tree, visiting all nodes in the tree. This is where the second parameter of the method comes into play. This argument is an object, and each property of the object is a hook function. The property values of this object include enter and exit, in addition to the type value that supports the AST syntax tree node. The exit hook function will be executed at the end of each node. The exit hook function will be executed at the end of each node. The exit hook function will be executed at the end of each node.

Modify the conversion code as follows:

Traverse (ast, {enter(path){console.log(path.type,"- enter ")}, exit(path){console.log(path.type,"- exit ")}});Copy the code

Debug runs the code again

Program - Enter VariableDeclaration - enter VariableDeclarator - enter Identifier - Enter Identifier - leave NumericLiteral - enter NumericLiteral - Leave VariableDeclaration - leave ExpressionStatement - enter AssignmentExpression - enter Identifier - Enter Identifier - Leave BinaryExpression - Enter Identifier - Enter Identifier - Leave NumericLiteral - Enter NumericLiteral - Leave BinaryExpression - Leave AssignmentExpression - Leave ExpressionStatement - Leave Program - LeaveCopy the code

As you can see from the print above, the enter and exit functions are executed when traversing each node. Compared with AST abstract syntax tree, it can be seen that it really belongs to depth-first recursive traversal

Next we add the following code to the VariableDeclaration hook function:

traverse(ast, {
    enter(path){
        
    },
    VariableDeclaration(path){
        console.log(path.type)
    },
    exit(path){
        
    }
});
Copy the code

Debug the code again, and the VariableDeclaration function is executed once, since our AST syntax tree has only one node of type VariableDeclaration.

Now, as many of you have noticed, the hook functionpathWhat do parameters do?

Path represents the path that connects two nodes while traversing the AST. You can use path.node to get the current node, path.parent. In this way, it can be modified by obtaining characteristic nodes under certain conditions.

One more question for those of you who are here, Babel probably defines a lot of node types, how do we know what the different types of nodes are?

The official gives all type points I view the type, here there are too many types, now use now check the document!!

@babel/types

There is also a plugin for @babel/types, which includes a lot of APIS and official documentation. It creates, modifies, deletes, and finds AST nodes. In addition, the AST node is also divided into many types, such as ExpressionStatement, ClassDeclaration, VariableDeclaration, etc., the same types of these types are corresponding to their creation method: T. expressionStatement, T. classDeclaration, and t. viableDeclaration also correspond to the following methods: T.i sExpressionStatement, T.I sClassDeclaration, t.I VARIableDeclaration. This plugin is often used with traverse traversal because types can only operate on a single node, and is typically used for deep traversals of nodes.

By now, you have a basic understanding of Babel compilation and the AST abstract syntax tree. In the next article, we will practice writing a Babel plug-in.

Babel plug-in practice (1) Analysis of the principle of Babel compilation

preface

What’s a Babel?

Abstract Syntax Tree (AST)

Lexical analysis

Syntax analysis

Babel compilation

Babel compilation process code demo

@babel/types

Related Posts

Tuax Technology Weekly 2021-08-27

How to identify anonymous Windows in Chrome

Wechat small program Wxs chapter