Learn to write MVVM framework in 3 days [2] : template parsing

In order to learn the source code of Vue, I decided to write a simplified version of Vue myself. Now I share what I’ve learned. If you’re using Vue but don’t know how it works, or if you’re looking to read the source code for Vue, hopefully these shares will help you understand how Vue works.

The target

Our goal today is for the following HTML templates:

<div class="outer">
  <div class="inner" v-on-click="onClick($event1)">abc</div>
  <div class="inner" v-class="{{innerClass}}" v-on-click="onClick">1{{name}}2</div>
</div>
Copy the code

We want to generate the following JS code:

with(this) {
  return _c(
    'div',
    {
      staticClass: "outer"
    },
    [
      _c(
        'div',
        {
          staticClass: "inner",
          on: {
            "click": function($event) {
              onClick($event, 1)
            }
          }
        },
        [_v("abc")]
      ),
      _c(
        'div',
        {
          staticClass: "inner",
          class: {
            active: isActive
          },
          on: {
            "click": onClick
          }
        },
        [_v("1" + _s(name) + "2")]])}Copy the code

(Note: For the generated code, line breaks and Spaces are added manually for the sake of presentation; For templates, the code that will be implemented next does not yet handle line breaks and whitespace correctly, which are also added here for demonstration purposes.

Parsing HTML

Our work will be carried out in two steps:

The template in the form of a string is first parsed and processed into the data format we need, which is called hereAST Tree(Abstract syntax tree).
Next, we’ll traverse the tree to generate our code.

First, we create the class ASTElement to hold our abstract syntax tree: The ASTElement instance has an array children to hold the children of this node, and the entry point of a tree is its root node. The node types are simply divided into two classes, text nodes and plain nodes (to be created with Document.createTextNode and Document.CreateElement, respectively). The text node has the text attribute, while the plain node will contain tag information and attrs lists. Attrs is used to hold various information such as class, style, V-if, @click, :class, etc. :

const ASTElementType = {
  NORMAL: Symbol('ASTElementType:NORMAL'),
  PLAINTEXT: Symbol('ASTElementType:PLAINTEXT')}; class ASTElement { constructor(tag,type, text) {
    this.tag = tag;
    this.type = type; this.text = text; this.attrs = []; this.children = []; } addAttr(attr) { this.attrs.push(attr); } addChild(child) { this.children.push(child); }}Copy the code

Parsing a template string starts at the head of the template string and loops through regular matches until the entire string is parsed. Let’s use a diagram to illustrate this process:

In the figure on the left, we can see that the sample template is divided into three categories: start tag, end tag, and text. The start tag can contain attribute pairs.

In the diagram of parsing process on the right, we can see that our parsing is a loop: in each loop, we first determine whether the next < character is the first character that follows. If so, we try to match the tag. The matching tag is divided into two cases: first, we try to match the start tag and the end tag. If not, the string between the current position and the next < character is treated as text (the inclusion of < in the text is ignored here to simplify the code). This loop continues until all templates are parsed:

const parseHtml = function (html) {
  const stack = [];
  let root;
  letcurrentElement; . const advance =function (length) {
    index += length;
    html = html.substring(length);
  };

  while (html) {
    last = html;

    const textEnd = html.indexOf('<');

    if (textEnd === 0) {
      const endTagMatch = html.match(endTag);
      if (endTagMatch) {
        ...
        continue;
      }

      const startTagMatch = parseStartTag();
      if (startTagMatch) {
        ...
        continue;
      }
    }

    const text = html.substring(0, textEnd);
    advance(textEnd);

    if (text) chars(text);
  }

  return root;
};
Copy the code

We declare several variables that represent:

Stack: depositASTElementStack structure, such as for<div class="a"><div class="b"></div><div class="c"></div></div>, will be in orderpush(.a) -> push(.b) -> pop -> push(.c) -> pop -> pop. This stack structure allows us to check that the labels in the template are correctly matched, but we will ignore this check and assume that all labels are correctly matched.
Root: indicates the entireASTElementThe root node of the tree for which the first start tag is encountered and createdASTElementThis value is set for instance. A template should have only the root node, which can also be passedstackTo check the state of the variable.
CurrentElement: current item being processedASTElementInstance, should also bestackElement at the top of the stack.

Handling closed labels

In the body of the loop, we use the regular endTag to try to match the closed tag, which is defined as follows:

const endTag = /^<\/([\w\-]+)>/;
Copy the code

As a graph:

\w matches any word character that includes an underscore, similar but not equivalent to “[A-za-z0-9_]”. This re matches strings like , , , and so on. Of course, there are many closed label forms that are excluded from the specification, but for the purposes of understanding the Vue principle this regular is sufficient for us.

If we match the closed tag, we need to skip the matched string (through advance) and continue the loop while maintaining the stack and currentElement variables:

const end = function () {
  stack.pop();
  currentElement = stack[stack.length - 1];
};

const parseEndTag = function(tagName) { end(); }; . const endTagMatch = html.match(endTag);if (endTagMatch) {
  const curIndex = index;
  advance(endTagMatch[0].length);
  parseEndTag(endTagMatch[1], curIndex, index);
  continue;
}
Copy the code

At this point we can make some fault tolerance judgments, such as whether the tag pairs match correctly and so on, we will skip all these steps.

Handling text

If the next character is not <, we will generate a text node for the string up to that point and add it to the current node as a child node:

const chars = function (text) {
  currentElement.addChild(new ASTElement(null, ASTElementType.PLAINTEXT, text));
};
Copy the code

Processing start tag

For the start tag, because we will write 0, 1, or more attribute pairs in the start tag, we need to deal with three parts: the start tag header, tail, and the defaultable attribute part. Therefore, we need to create three regular expressions:

const startTagOpen = /^<([\w\-]+)/; const startTagClose = /^\s*>/; const attribute = /^\s*([\w\-]+)(? (=) (? :"([^"] *)"+))? /;Copy the code

Represented by a graph (generated by RegExper.com) :

StartTagOpen and startTagClose are both relatively simple and won’t be repeated here. (Note that I’m not considering self-closing tags, such as .) For attribute pairs, we can see that = and the parts that follow are defaultable, such as disabled=”disabled” and disabled.

Therefore, the whole matching process is divided into three steps:

Match the head
Match attribute pairs one by one and join the currentASTElementAttribute pair of
Match the tail

Finally, push the newly created ASTElement to the top of the stack and mark it as the current element:

const start = function (match) {
  if(! root) root = match;if (currentElement) currentElement.addChild(match);
  stack.push(match);
  currentElement = match;
};

const parseStartTag = function () {
  const start = html.match(startTagOpen);
  if (start) {
    const astElement = new ASTElement(start[1], ASTElementType.NORMAL);
    advance(start[0].length);
    let end;
    let attr;
    while(! (end = html.match(startTagClose)) && (attr = html.match(attribute))) { advance(attr[0].length); astElement.addAttr([attr[1], attr[3]]); }if (end) {
      advance(end[0].length);
      returnastElement; }}}; const handleStartTag =function (astElement) {
  start(astElement);
};

const startTagMatch = parseStartTag();
if (startTagMatch) {
  handleStartTag(startTagMatch);
  continue;
}
Copy the code

The generated code

After the above steps, we can parse the template string and get a tree of astElements. Next, we need to walk through the tree and generate the code string to render the tree. Finally, after we get the code string, we pass it into the Function constructor to generate the render Function.

The first thing to do is wrap the entire code with with(this) :

const generateRender = function (ast) {
  const code = genElement(getRenderTree(ast));
  return 'with(this){return ' + code + '} ';
};
Copy the code

So when we specify this correctly, in the template we can write {{calc(a + b.c)}} instead of {{this.calc(this.a + this.b.c)}}.

GetRenderTree recursively traverses the entire tree:

const getRenderTree = function ({ type, tag, text, attrs, children}) {
  return {
    type,
    tag,
    text: parseText(text),
    attrs: parseAttrs(attrs),
    children: children.map(x => getRenderTree(x))
  };
};
Copy the code

In this process, we will do some more work on the original ASTElement tree, because the original book is all raw data, and we will need to do some more work on the data based on our rendering process.

The processing here is divided into two parts:

Handles the text of a text node
Working with property lists

Now let’s look at the code to see what preprocessing we need to do.

First, for the text node, we need to find the part that contains the method/variable, that is, the part that is contained by {{}}. For example, ABC needs to be converted to code ‘ABC’, {{getStr(item)}} needs to be converted to code getStr(item), ABC {{getStr(item)}}def needs to be converted to code ‘ABC’ + getStr(item) + ‘def’.

That is, we need to constantly match the {{}} parts of the text, preserve the contents, and convert the rest of the text to strings and finally concatenate them together:

const tagRE = /\{\{(.+?) \}\}/g; const parseText =function (text) {
  if(! text)return;
  if(! tagRE.test(text)) {return JSON.stringify(text);
  }
  tagRE.lastIndex = 0;
  const tokens = [];
  let lastIndex = 0;
  let match;
  let index;
  let tokenValue;
  while ((match = tagRE.exec(text))) {
    index = match.index;
    if (index > lastIndex) {
      tokenValue = text.slice(lastIndex, index);
      tokens.push(JSON.stringify(tokenValue));
    }
    tokens.push(match[1].trim());
    lastIndex = index + match[0].length;
  }
  if (lastIndex < text.length) {
    tokenValue = text.slice(lastIndex)
    tokens.push(JSON.stringify(tokenValue));
  }
  return tokens.join('+');
};
Copy the code

For the properties section (or directives), let’s start with the (rather limited) properties we’ll support:

Class: e.g.class="abc def", will be processed as'class': 'abc def'Key-value pairs like this.
V-class: : For examplev-class="{{innerClass}}", will be processed as'v-class': innerClassKey-value pairs like this. Here we are being lazy, not as dynamic as Vue is for nowclassImplement binding as an object or array.
V – on – [eventName] : for examplev-on-click="onClick", will be processed as'v-on-click': onClickKey-value pairs like this; whilev-on-click="onClick($event, 1)", will be processed as'v-on-click': function($event){ onClick($event, 1) }Key-value pairs like this.

Because of the simplicity of the re used to implement attribute matching, we can’t use :class or @click for binding at this point.

Support for V-classes is similar to working with text parts.

For events, you need to decide if you want to wrap the event with function($event){}. If the string contains only letters and so on, such as onClick, we consider it a method name and do not need to wrap it; If it contains more than letters, such as onClick(), flag = true, we wrap it around:

const parseAttrs = function (attrs) {
  const attrsStr = attrs.map((pair) => {
    const [k, v] = pair;
    if (k.indexOf('v-') = = = 0) {if (k.indexOf('v-on') = = = 0) {return `'${k}': ${parseHandler(v)}`;
      } else {
        return `'${k}': ${parseText(v)}`; }}else {
      return `'${k}': ${parseText(v)}`;
    }
  }).join(', ')
  return` {${attrsStr}} `; }; const parseHandler =function (handler) {
  console.log(handler, /^\w+$/.test(handler));
  if (/^\w+$/.test(handler)) return handler;
  return `function($event) {${handler}} `; };Copy the code

The processing required for different attributes/bindings in Vue is quite complex, and we have implemented a fairly limited number of attributes in a relatively simple way to simplify the code. Interested children can read the Vue source code or try to implement their own custom instructions.

Finally, we walk through the processed tree to piece together our code. Here we call the _c and _v methods to render plain nodes and text nodes. We will cover the implementation of these two methods in the next practice:

const genElement = function (el) {
  if (el.type === ASTElementType.NORMAL) {
    if (el.children.length) {
      const childrenStr = el.children.map(c => genElement(c)).join(', ');
      return `_c('${el.tag}'.${el.attrs}[${childrenStr}`]); }return `_c('${el.tag}'.${el.attrs}) `; }else if (el.type === ASTElementType.PLAINTEXT) {
    return `_v(${el.text})`;
  }
};
Copy the code

You can also try…

We didn’t consider comment nodes when parsing the template, and we made it easy for the regees that matched the tags because they didn’t match similar<? xml,<! DOCTYPEor<xsl:stylesheetLabels like this
We didn’t handle line breaks and whitespace correctly in the template
We do not consider if the string contains<How do you deal with that
We’re not thinking about autism and labels but we’re assuming that all labels have opening and closing labels
We do not make fault tolerance for mismatched tags and do not consider must-include/no-include tag relationships (e.gtableThe following should be included firsttbody, and should not directly includetr;pInternal cannot containdivEtc.)
We also made the regex for attributes too simple to match@clickand:srcThis form
The supported directives/attributes are quite limited, note that support for example is requireddisabledwithdisabled="disabled"Abbreviations like this

These are interesting feature points if you want to try them out for yourself.

conclusion

This time, we demonstrated how to parse a template string to generate an abstract syntax tree and generate rendering code from it.

In one final practice, we’ll put together what we’ve already done and finally finish rendering the front end.

Reference:

Learn to write MVVM framework in 3 days [2] : template parsing

The target

Parsing HTML

Handling closed labels

Handling text

Processing start tag

The generated code

You can also try…

conclusion

Related Posts

React Virtual DOM

“Javascript you Don’t Know” “Parts” Learning Notes (Part I of Volume 1)

Koa+Vue implements GitHub style personal blog system