1. What is Babel

In short, Babel can translate ECMAScript 2015+ code so that it can run in older browsers or environments.

// es2015 const arrow function const add = (a, b) => a + b; Var add = function add(a, b) {return a + b; };Copy the code

Babel’s function is pure. We pass a piece of source code to Babel, and it returns a new string of code to us. It doesn’t run our code, it doesn’t package our code. It’s just a compiler.

Taro also used Babel to convert React syntax into applets.

2. Babel usage

First, the usage and concept of Babel; Don’t want to see you skip zhuanlan.zhihu.com/p/58624930 1, 2, 3, 4 piece from scratch

Package composition of Babel

Core packages

  • Babel-core: The Babel translator itself, which provides Babel’s translation API, such as babel.transform, for translating code. Babel-loader like WebPack calls these apis to complete the translation process.
  • Babylon: JS lexical parser, AST generated
  • Babel-traverse: for AST traversal, for plugin
  • Babel-generator: Generates code according to the AST

Feature pack

  • Babel-types: Nodes used to validate, build, and change the AST tree

  • Babel-template: Helper function for building AST tree nodes from string code

  • Babel-helpers: a set of prefabricated babel-template functions that are used by plugins

  • Babel-code-frames: Used to generate error messages, print error error source frames, and indicate error locations

  • Babel-plugin-xxx: a plug-in used during Babel translation, where babel-plugin-transform-xxx is used by the transform step

  • Babel-preset – XXX: Series of plugins to be used in the transform stage (official written plug-ins)

  • Babel-polyfill: a shim for native objects and apis added to the JS standard, which is implemented as a wrapper around core-JS and Regenerator-Runtime packages

  • Babel-runtime: functions like babel-polyfill and is commonly used in libraries or plugins because it does not pollute the global scope

kit

Babel-cli: command line tool of Babel, which translates JS code through the command line

Babel-register: automatically translates js code files referenced by require by binding node.js’s require

Babel8 changes the package name to @babel, see below

Blog.csdn.net/weixin_3372…

Principle 3.

Babel transform JS code can be broken down into three major steps:

  • Parser: This process takes pre-conversion source code and outputs an AST (abstract syntax tree). The package responsible for this in Babel is Babel/Parser;

  • Transform: This process takes the AST output from Parser (abstract syntax tree) and outputs the transformed AST (abstract syntax tree). The package responsible for this in Babel is @babel/traverse;

  • Generator: This process takes the new AST output from the Transform and outputs the transformed source code. The package responsible for this process in Babel is @babel/generator.

So you should know about AST in advance

Babel is a transpiler that feels more accurate than a compiler, called transpiler, because it simply translates higher-version rules of the same language into lower-version rules, unlike a compiler that outputs lower-level code in another language. Similar to the compiler, however, the Babel translation process is divided into three stages: parsing, transforming, and generating. Taking ES6 code translation as an example, the specific process of Babel translation is as follows:

(1) code –> AST

The first step is to convert our ES6 code strings into the ES6 AST

The conversion tool is Babel’s Parser

How do you understand the transformation as normal transformation AST, the simple example will be put at the end

(2) the Transform

What this step does is manipulate the AST. Convert ES6 AST operation JS to ES5 AST

Transform iterates through the AST, adding, removing, updating, and so on to the AST structure, depending on the plugin provided by the developer. Babel provides two “enter node Enter” and “exit node exit” opportunities for each AST node, which can be used by third-party developers to operate on the old AST. It’s worth noting that the Transform step is the most complex part of Babel and where third-party plug-ins come into their own.

This step is the most important, like Webpack, where plugins take effect, or you can add your own hand-written plugins.

The Transform process follows a typical visitor pattern for those of you unfamiliar.

We can see that there are many similar elements in the AST that have a type attribute, and such elements are called nodes. A node usually contains several attributes that can be used to describe part of the AST.

For example, here is one of the most common Identifier nodes:

{
    type: 'Identifier',
    name: 'add'
}
Copy the code

Indicates that this is an identifier.

So, you can manipulate the AST, which is the nodes in it, and you can add, delete, or modify these nodes to convert them into the desired AST. Babel traverses the AST depth-first. For each branch on the AST, Babel traverses down to the end, then up to exit the node it just traversed, and then looks for the next branch.

{"type": "Program", "body": [{"type": "Program", "body": [{"type": "VariableDeclaration", // VariableDeclaration" declarations": [ // Variable declaration "ID ": {"type": "Identifier", // Identifier (basic) "name": "add" // function name}, "init": {"type": "ArrowFunctionExpression", // Arrow function "id": null, "expression": true, "generator": false, "params": [// parameter {"type": "Identifier", "name" : "a"}, {" type ":" Identifier ", "name" : "b"}], "the body" : {/ / function body "type" : "/ / binomial BinaryExpression", "left" : {/ / on the left side of the binomial "type" : "Identifier", "name" : "a"}, "operator" : "+", "right" / / binomial operators: {/ / binomial on the right side of the "type" : "Identifier", "name" : "b"}}}}], "kind" : "const"}], "sourceType" : "the module"}Copy the code

The root node is not a declarations but is traversed from the declarations:

  1. A variable is declared, its internal properties are known (ID, init), and then we access each property and its child nodes from that.
  2. Id is an Idenrifier with a name attribute representing the variable name.
  3. Next comes init, which also has several internal properties:
  • Type is ArrowFunctionExpression, which indicates that this is an ArrowFunctionExpression
  • • Params is the input parameter to this arrow function, where each parameter is a node of type Identifier;
  • • The body property is the body of the arrow function, which is a BinaryExpression binomial: left, operator, and right, representing the left, operator, and right variables of the binomial respectively.

Here’s the vernacular form for traversing AST, and let’s see how Babel does it:

Babel maintains an object called Visitor, which defines the methods used to get specific nodes in the AST.

Visitor

Babel traverses the AST actually passes through the node twice: traversal and exit, so the Visitor in Babel should actually look like this:

var visitor = { Identifier: { enter() { console.log('Identifier enter'); }, exit() { console.log('Identifier exit'); }}};Copy the code

Let’s take this visitor to iterate over an AST like this:

Params: [/ / {" type ":" Identifier ", "name" : "a"}, {" type ":" Identifier ", "name" : "b"}]Copy the code

The process might go something like this…

  • Enter the Identifier (params [0])
  • Come to an end
  • Exit the Identifier (params [0])
  • Enter the Identifier (params [1])
  • Come to an end
  • Exit the Identifier (params [1])

Of course, the Visitor pattern in Babel is far more complicated than that…

Back up, the arrow function is a syntax that ES5 doesn’t support, so Babel has to convert it to a normal function and iterate through the layers until it finds the ArrowFunctionExpression node, at which point it needs to be replaced with the FunctionDeclaration node. So, the arrow function might be handled like this:

import * as t from "@babel/types"; var visitor = { ArrowFunction(path) { path.replaceWith(t.FunctionDeclaration(id, params, body)); }};Copy the code

Those interested in details can browse the source github.com/babel/babel…

(3) Generate code

The previous step is to convert THE ES6 AST operation JS to the ES5 AST

This step is to convert the AST of ES5 into an ES5 code string

After the above two phases, the code that needs to be translated has been transformed to generate a new AST, and the last phase of course is to output the code according to this AST.

Babel is available via github.com/babel/babel… To finish. And, of course, depth-first traversal.

Generator Generator can be seen as the reverse of the Parser, generating code from the new AST, which is essentially generating strings that have no meaning on their own and that the compiler gives meaning to the strings into what we call “code”. Babel traverses the AST depth-first and builds a string that represents the transformed code.

class Generator extends Printer { constructor(ast, opts = {}, code) { const format = normalizeOptions(code, opts); const map = opts.sourceMaps ? new SourceMap(opts, code) : null; super(format, map); this.ast = ast; } ast: Object; generate() { return super.generate(this.ast); }}Copy the code

After these three stages, the code is translated by Babel.

4. Simple implementation

Const add = (a,b) => a + b; const add = (a,b) => a + b; const add = (a,b) => a + b;

Define the code string to convert:

/** * const codeString = 'const add = (a, b) => a + b';Copy the code

(1) ES6 code –> AST

Generating an AST requires both string lexing and syntax analysis

The first is lexical analysis

/** * Param codeString Tokens stream */ function Tokens (codeString) {let Tokens = [];  // Let current = 0; While (current < codeString. Length) {let char = codeString[current]; / / processing brackets if first (char = = = '(' | | char = = =') ') {tokens. Push ({type: 'parens' value: char}); current++; continue; Const WHITESPACE = /\s/; const WHITESPACE = /\s/; const WHITESPACE = /\s/; if (WHITESPACE.test(char)) { let value = ''; while (current < codeString.length && WHITESPACE.test(char)) { value = value + char; current++; char = codeString[current]; } tokens.push({ type: 'whitespace', value: value }); continue; } // let NUMBERS = /[0-9]/; if (NUMBERS.test(char)) { let value = ''; while (current < codeString.length && NUMBERS.test(char)) { value = value + char; current++; char = codeString[current]; } tokens.push({ type: 'number', value: value }); continue; } const LETTERS = /[a-za-z \$\_]/; const LETTERS = /[A-za-z \$\_]/; if (LETTERS.test(char)) { let value = ''; While (current < codeString.length && /[A-zA-z0-9 \$\_]/.test(char)) {value = value + char; current++; char = codeString[current]; } tokens.push({ type: 'identifier', value: value }); continue; } // const COMMA = /,/; if (COMMA.test(char)) { tokens.push({ type: ',', value: ',' }); current++; continue; } / / processing OPERATOR const OPERATOR = / = | | \ + > /; if (OPERATOR.test(char)) { let value = ''; while (OPERATOR.test(char)) { value += char; current++; char = codeString[current]; } // If (value === '=>') {tokens. Push ({type: 'ArrowFunctionExpression', value,}); continue; } tokens.push({ type: 'operator', value }); continue; } throw new TypeError(' not yet added to this character processing ${char} '); } return tokens; }Copy the code

Syntax analysis

/** * param tokens stream * @returns AST */ const Parser = tokens => {// Declare a full time pointer, It will always exist let current = -1; Const tem = []; // Declare a temporary pointer to const tem = []; // Let tokens = tokens[current]; Const parseDeclarations = () => {// hold current pointer setTem(); // next(); // If (token.type === 'identifier' && token.value === 'const') {const declarations = {type: const declarations; 'VariableDeclaration', kind: token.value }; next(); If (token.type! == 'identifier') { throw new Error('Expected Variable after const'); } / / we get to the name of the variable declarations. IdentifierName = token. The value; next(); If (token.type === 'operator' && token.value === '=') {declarations parseFunctionExpression(); } return declarations; }}; const parseFunctionExpression = () => { next(); let init; / / if the '=' followed by parentheses or character that basic judgment is an expression of the if ((token. Type = = = 'parens' && token. The value = = =' (') | | token. Type = = = 'identifier') {  setTem(); next(); while (token.type === 'identifier' || token.type === ',') { next(); } // If (token.type === 'parens' && token.value === ')') {next(); if (token.type === 'ArrowFunctionExpression') { init = { type: 'ArrowFunctionExpression', params: [], body: {} }; backTem(); // parse.params = parseParams(); Init. body = parseExpression(); } else { backTem(); } } } return init; }; const parseParams = () => { const params = []; if (token.type === 'parens' && token.value === '(') { next(); while (token.type ! == 'parens' && token.value ! == ')') { if (token.type === 'identifier') { params.push({ type: token.type, identifierName: token.value }); } next(); } } return params; }; const parseExpression = () => { next(); let body; while (token.type === 'ArrowFunctionExpression') { next(); } // If (token. Type === 'identifier') {body = {type: 'BinaryExpression', left: { type: 'identifier', identifierName: token.value }, operator: '', right: { type: '', identifierName: '' } }; next(); if (token.type === 'operator') { body.operator = token.value; } next(); if (token.type === 'identifier') { body.right = { type: 'identifier', identifierName: token.value }; } } return body; }; // const next = () => {do {++current; token = tokens[current] ? tokens[current] : {type: 'eof', value: ''}; } while (token.type === 'whitespace'); }; // Const setTem = () => {tem.push(current); }; // Const backTem = () => {current = tem.pop(); token = tokens[current]; }; const ast = { type: 'Program', body: [] }; while (current < tokens.length) { const statement = parseDeclarations(); if (! statement) { break; } ast.body.push(statement); } return ast; };Copy the code

It can be assumed that the process of transforming into an AST is a series of continuous operations such as loops, regularization, and identifier alignment

(2) the Transform

Traverser = (ast, visitor) const traverseArray = (array, traverser) parent) => { array.forEach((child) => { traverseNode(child, parent); }); }; Const traverseNode = (node, parent) => {const methods = visitor[node.type]; if (methods && methods.enter) { methods.enter(node, parent); } switch (node.type) { case 'Program': traverseArray(node.body, node); break; case 'VariableDeclaration': traverseArray(node.init.params, node.init); break; case 'identifier': break; default: throw new TypeError(node.type); } if (methods && methods.exit) { methods.exit(node, parent); }}; traverseNode(ast, null); }; /** * Transform procedure * @param ast The ast to be converted * This function calls the traverser, Const transformer = (AST) => {// newAst const newAst = {type: 'Program', body: []}; // Add a _context attribute to the ast, pointing to the same memory address as newast. body. Newast. body ast._context = newast. body; traverser(ast, { VariableDeclaration: { enter(node, parent) { let functionDeclaration = { params: [] }; if (node.init.type === 'ArrowFunctionExpression') { functionDeclaration.type = 'FunctionDeclaration'; functionDeclaration.identifierName = node.identifierName; functionDeclaration.params = node.init.params; } if (node.init.body.type === 'BinaryExpression') { functionDeclaration.body = { type: 'BlockStatement', body: [{ type: 'ReturnStatement', argument: node.init.body }], }; } parent._context.push(functionDeclaration); }}}); return newAst; };Copy the code

(3) generate

/** * Generator process * @param node new AST * @returns new code */ const Generator = (node) => {switch (node.type) {// If yes 'Program' node, then we iterate over each node in its' body 'property and recursively // call the codeGenerator again on these nodes, printing the result onto a new line. case 'Program': return node.body.map(generator) .join('\n'); // If it's FunctionDeclaration we'll iterate through the array of arguments and the body property case 'FunctionDeclaration': return 'function' + ' ' + node.identifierName + '(' + node.params.map(generator) + ')' + ' ' + generator(node.body); // For 'Identifiers' we simply return' node 'identifierName case 'identifier': return node. IdentifierName; Case 'BlockStatement': return '{' + Node.body. map(Generator) + '}'; Case 'ReturnStatement': return' return' + '+ generator(node.argument); // If the statement is ReturnStatement we call the left and right nodes and concatenate case 'BinaryExpression': return generator(node.left) + ' ' + node.operator + ' ' + generator(node.right); Default: throw new TypeError(Node.type); // Throw new TypeError(node.type); }};Copy the code

(4) The whole process is connected in series to complete the call chain

let token = tokens(codeString);
let ast = parser(token);
let newAST = transformer(ast);
let newCode = generator(newAST);
 
console.log(newCode);
Copy the code

5. Other extended knowledge

It is also important to note that Babel only translates syntaxes introduced by new standards, such as ES6 arrow functions into ES5 functions; The new standard introduces new native objects, new prototype methods for some native objects, new apis (such as Proxy, Set, etc.) that Babel does not translate. Users need to introduce polyfill to solve the problem

plugins

Plug-ins are used in the translation process of Babel, especially in the second stage, which is a transforming process. If no plug-ins are used in this stage, Babel will output the code as it is.

We focused mainly on the transforming plug-ins, which didn’t need to be configured because the Transform plug-ins would automatically use the corresponding lexical plug-ins.

presets

It would be too painful to preset the types of plug-ins to be used in translation, so Babel has created a set of preset plug-ins called PRESET, and all we need to use is preset. Using the JS standard as an example, Babel offers the following preset:

• es2015

• es2016

• es2017

• env

Es20xx presets only refer to the approved standard for that year, and ENV refers to the latest standard, including Latest and ES20XX years. There are also stages prior to stage-0 to stage-4, which are experimental versions of preset and are recommended not to be used.

polyfill

Polyfill is a SHIM for the ES2015+ environment. Implementatively, the babel-Polyfill package simply wraps core-JS and ReGenerator Runtime, which are where the real implementation code is (more on core-JS later).

Using Babel-Polyfill brings the ES2015+ environment into your code environment as a whole, allowing your code to directly use the new native objects, new apis, and so on introduced by the new standard, generally for individual applications and pages.

runtime

The difference between Polyfill and Runtime (must see)

Using Babel-Polyfill directly is fine for situations where your application or page environment is under your control. However, using polyfill in the Library is not feasible. The Library is intended for external use, but the external environment is outside the library’s control, and Polyfill contaminates the original global environment (because new native objects, apis, and so on are introduced directly from Polyfill). It’s easy to get into conflict, so this is where babel-Runtime comes in handy.

zhuanlan.zhihu.com/p/58624930

Finally, writing should not be, please point to like, forwarding please note everywhere thank ~