This article continues with the study notes for Mustache in the previous installment, Mustache Template Engine 01

In the last part we gave an overview of what a template engine is and a few small examples to help you understand it. The main content of this paper is divided into two parts: the basic token idea of mustache, and the formal beginning of handwriting mustache

1. The low token idea of Mustache

You know the mustache template engine is used to turn a string template into a DOM template and then attach the data to the DOM tree for rendering. In the process Mustache introduced a concept called Tokens to act as a “go-between.” A picture is worth a thousand words



In short, tokens are JS nested array representations of template strings. It is an array containing many tokens, each of which is an array generated based on rules.

When a loop exists in a template string, it is compiled into more deeply nested tokens

We can print tokens directly from the browser console by modifying mustache source code: Find the parseTemplate function in the source code, and at the end of the function body, return nestTokens(squashTokens(tokens)) is actually tokens. Make the following minor changes so that you can print them in your browser

const myTokens = nestTokens(squashTokens(tokens))
console.log(myTokens)
return myTokens
Copy the code

For example, you have a template string like this

const templateStr = '
      

       {{name}} basic information 
      
'
Copy the code

Then the final printed tokens will be shown in the figure below

Start writing mustache by hand

Now that you know tokens, start writing your own My_TemplateEngine object to implement Mustache. We will write separate functions into separate JS files, usually a separate class, and each individual function should be able to perform a separate unit test.

Implements compilation of template strings as Tokens

Realize the Scanner class

An instance of the Scanner class is a Scanner that scans the template string provided as an argument during construction.

Attributes –

  • Pos: pointer to record the current position of the scanned string
  • Tail: tail, the string after the current pointer (including the character to which the pointer is currently pointing)

Method –

  • Scan: No return value, so that the pointer skips the passed end identifier stopTag
  • ScanUntil: Pass in a specified content stopTag as the identifier for pos to end the scan and return the scanned content
// Scanner.js
export default class Scanner {
  constructor(templateStr) {
    this.templateStr = templateStr
    / / pointer
    this.pos = 0
    / / tail
    this.tail = templateStr
  }

  scan(stopTag) { 
    this.pos +=  stopTag.length // The pointer skips the stopTag. For example, if stopTag is {{, pos will be added by 2
    this.tail = this.templateStr.substring(this.pos) // Substring truncates the second argument to the end
  }

  scanUntil(stopTag) {
    const pos_backup = this.pos // Record the pointer position at the start of the scan
    // Continue to move the pointer when the end of the sweep is not reached and when the beginning of the tail is not stopTag
    // Pay attention to the necessity of && to avoid the occurrence of infinite loops
    while (!this.eos() && this.tail.indexOf(stopTag) ! = =0) {this.pos++ // Move the pointer
      this.tail = this.templateStr.substring(this.pos) // Update the tail
    }
    return this.templateStr.substring(pos_backup, this.pos) // Return the scanned string, excluding this.pos
  }
  
  // If the pointer has reached the end of the string, return the Boolean eos(end of string)
  eos() {
    return this.pos >= this.templateStr.length
  }
}
Copy the code

Generate tokens from the template string

With the Scanner class in place, you can begin to generate an array of Tokens from the template string passed in. The first item of each token array in the eventually generated tokens is identified by name(data) or text(non-data text) or #(start of loop) or /(end of loop). Create a new parseTemplateToTokens. Js file to do this

// parseTemplateToTokens.js
import Scanner from './Scanner.js'
import nestTokens from './nestTokens' // This will be explained later

/ / function parseTemplateToTokens
export default templateStr => {
  const tokens = []
  const scanner = new Scanner(templateStr)
  let word
  while(! scanner.eos()) { word = scanner.scanUntil('{{')
    word && tokens.push(['text', word]) // Make sure the tokens have a value
    scanner.scan('{{')
    word = scanner.scanUntil('}} ')
    /** * check whether the word collected between {{and}} begins with the special character # or /. If yes, the first element of the token corresponds to # or /. Otherwise, it is name */
    word && (word[0= = =The '#' ? tokens.push([The '#', word.substr(1)]) : 
      word[0= = ='/' ? tokens.push(['/', word]) : tokens.push(['name', word]))
    scanner.scan('}} ')}return nestTokens(tokens) // Return the folded tokens, see below
}
Copy the code

Introduce parseTemplateToTokens in index.js

// index.js
import parseTemplateToTokens from './parseTemplateToTokens.js'

window.My_TemplateEngine = {
  render(templateStr, data) {
    const tokens = parseTemplateToTokens(templateStr)
    console.log(tokens)
  }
}
Copy the code

In this way, we can initially convert the passed templateStr into tokens, such as templateStr for

const templateStr = ` < ul > {{# arr}} < li > < div > the basic information of the {{name}} < / div > < div > < p > {{name}} < / p > < p > {{age}} < / p > < div > < p > interests: 
 
      
    
        {{#hobbies}} 
       
  1. {{.}}
    2.  {{/hobbies}} 
      
    {{/arr}}  `
Copy the code

So the current parseTemplateToTokens processing will get the following tokens

The next step is to find a way to make the part of the loop between # and/the third element of the token whose # is the first element of the array. Insert the red box into [“#”,”arr”] as the third element; In the same way, insert the blue box into [“#”,”hobbies”] as the third element

Achieve nesting of tokens

Create a new file nestTokens. Js, define the nestTokens function to do the nesting of tokens, and return the passed tokens as an array containing nested nestTokens.

Then introduce nestTokens in parseTemplateToTokens.js and return nestTokens(tokens) at the end.

  • Implementation approach

In nestTokens, we iterate over each token passed to us. The first item we encounter is # and /, and the rest is treated as a default. The general idea is that when the first item of the traversed token is #, each token traversed until the matching/is encountered is placed in a container (collector), which is placed in the current token as the third element.

But there’s a problem: what if you run into a # before you run into a matching /? So how do you solve the problem of nested loops within loops?

The solution is to create an array of the stack data types. When a # is encountered, the current token is placed on the stack and the collector points to the third element of the token. The next # is encountered and the new token is put on the stack, with the collector pointing to the third element of the new token. The collector points to the token that has been removed from the stack. This takes advantage of the advanced, out-of-the-box nature of the stack to ensure that each token traversed is placed in the right place and that the collector points to the correct address.

  • Specific code
// nestTokens.js
export default (tokens) => {
  const nestTokens = []
  const stack = []
  let collector = nestTokens // Start by making the collector collector point to the returned array of nestTokens
  tokens.forEach(token= > {
    switch (token[0]) {
      case The '#':
        stack.push(token)
        collector.push(token)
        collector = token[2] = [] // Assign the same value
        break
      case '/':
        stack.pop(token)
        collector = stack.length > 0 ? stack[stack.length-1] [2] : nestTokens
        break;
      default:
        collector.push(token)
        break}})return nestTokens
}
Copy the code

One More Thing

Collector = token[2] = []

token[2] = []
collector = token[2]
Copy the code

Simple as it looks, it actually implies a small pit, unless you really understand it, otherwise try not to use. For example, I saw an example elsewhere,

let a = {n:1};
a.x = a = {n:2};
console.log(a.x); / / output?
Copy the code

The answer is undefined, did you do that right?

This is a study note of the principles behind {{}} used in vUE, followed by mustache Template Engine-03.