MPM store visual building system — architecture process design

Before reading this article, may wish to have a look at: 1) VISUAL building system of MPM stores — element design

preface

This is the second part of MPM sharing series. In the last part of MPM Visual building system for stores — Element Design, we introduced MPM as a page visual building system for store scenes, what are the most basic system elements, and gave the derivation and design process of system elements. System elements are the link between each module of a huge system and the cornerstone of system design, while elements are the architecture and process of the system.

MPM system architecture

The MPM we are talking about is not only the operation of the store editing system directly faced by students. The store pages generated by MPM are also an important part of MPM. Therefore, in terms of architecture, MPM is mainly composed of an editing system and a page parsing engine.

The editing system is the MPM store editing interface directly used by the operation. After editing and configuration by the operation user, the editing system generates a PageData PageData. In different end environments, PageData will parse and render through different page parsing engines to generate corresponding store pages. From the MPM architecture diagram, we can also see that the entire MPM architecture process is based on four system elements, that is, both the editing system and the parsing engine have implementation of four system elements, which are used to complete the assembly and presentation of pages.

PageData

In MPM architecture, PageData is an important medium of link editing state (editing configuration) and display state (user browsing). It is essentially an abstract description of the page of the store. Based on PageData, the page parsing engine at each end can know how to generate the corresponding page of the store. In addition to the main page configuration data, it will also undertake some intermediate data artifacts during the page parsing process, such as request data caching. On the other hand, using PageData as isolation, we can easily achieve an edit, multi-terminal display. Therefore, proper design of PageData is a key step to implement MPM architecture.

In MPM, we use JSON to store PageData, which is the standard structure of PageData:

{" pageId ":" ", / / page ID "appType" : 1, page types: / / MPM activities | 1-2 - which | 3 - small program "pageConfig" : {" basic ": {" type" : 2, / / business types: ordinary page | 1-2 - jing xi page "env" : 2, / / channel type: 1 - hand Q | 2 - WeChat | 3 - M stand "bgColor" : "# FFF," / / "createDate" page background color: ", "/ /" customCode ": create time null, additional code" customCodeIn "content: / / 0, / / other information of additional code position: 0 - inserted at the top of the | 1 - at the bottom of the insert" expireTime ": ", "/ / expiration time" expireUrl ":" ", / / overdue jump links "name" : "", / / the name of the page" path ":" ", / / page path "status" : 1, / / page state: 1 - effective | 0 - invalid "floorSortId" : "", / / floor sequence id" floorSortType ":" ", / / sorting type 0 - floor sorting | 1 - TAB sort "forceLogin" : 0 / / whether mandatory login: 0 - no | 1 -}, "search" : {" topShow ": 0, / / whether the show at the top of the search box: 0 - no | 1 - is" topRuleId ":" ", / / at the top of the search box "topAd" gold finger parameters: Rd" topKeywordShow": 0, // whether to display "topKeywordRuleId": "", / / the bottom shuffling out gold finger parameters" bottomShow ": 0, / / whether the show at the bottom of the search module: 0 - no | 1 - is" bottomRuleId ":" "} / / search gold finger at the bottom of the parameters, the "share" : {" title ": "The jingdong shopping", / / share the title "desc" : ", fast, good, province ", / / share the headline "imgUrl" : "/ / wq.360buyimg.com/img/mpm/defaultShare.jpg" / / share photos}}, "the userInfo" : {" checkNewuser ": false, / / check whether the couple" checkVip ": "CheckBind ": false, // check whether the query is bound to "checkFirstBuy": }, "componentConfig": [], // componentConfig" template": {vueFnObj": {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj: {vueFnObj:}} {}, // Record< style id, extension method > "vueHook": {}, // Record< style id, hook function > "styleTpl": {}, / / Record > < style id, template rendering function/page/straight out of the "header" : "", / / page head" footer ":" "/ /} at the bottom of the page," dataCache ": {/ / to cache data" the userInfo ": {}, // User identity data "floorSortData": {}, // floor ordering data "dsCache": {} // Floor service interface data cache}}Copy the code

This is a page save generated by a PageData, although a bit complex but the structure is still clear:

1, pageId

Unique logo on MPM store page.

2, appType

There are three basic page types of MPM: the most used activity type (also the default type), the library area solidified operation of the library area type, small program type.

3, pageConfig

Page level configuration. Include:

• PageConfig. Basic: Basic configuration of a page, including page name, page status, page creation/expiration time, page service/channel type, etc.

• PageConfig. Search: page search box configuration, including the detailed configuration of the top search box and the bottom search module of the page;

• PageConfig. Share: share configuration of the page.

4, the userInfo

The user-level configurations are not controlled by the operation, but are automatically analyzed and set by the MPM editing system when saved. For example, if the operation uses a component that needs to determine whether the user is a VIP, the MPM will automatically set userinfo.checkVIP to true. In the page parsing engine, user-level configuration is handled in the preload phase, as we’ll discuss later.

5, componentConfig

The floor level configuration, which stores the detailed floor configuration of the operation, is the configuration that the parsing engine cares most about. ComponentConfig is an object array, each object represents a floor, which contains the configuration data of the floor components, including the public attribute configuration and private attribute configuration of the floor, the parsing engine will traverse this array, one by one rendering floor.

6, the template

Page some logic code, mainly including some common logic and page to use the MPM template. The parsing engine assembles template and configuration data into page content for presentation, which is only used for H5 pages. The specific structure of MPM template can be found in visual building System of MPM Store – Element Design.

• Template. vueFnObj: map object, which stores the relational mapping between the template and its corresponding extension function.

• Template. vueHook: a map object containing a relational map of the template and its corresponding declaration cycle hook function;

• Template. styleTpl: map object, which holds a relational mapping between a template and its corresponding template.

• Template. header: an incomplete block of HTML code that contains some common priority logic for the page, and where the style code for an MPM template is stored during the save and edit phase.

• Template. footer: An incomplete HTML block that contains some common bottom logic for the page.

7, dataCache

Some data cache after the page request interface, the main function is to avoid repeated requests, especially to avoid the outbound data has been completed after pulling the client to be repeated requests.

The generation of PageData

PageData is generated by the editing system and maintained in a SQL data table in real time. When a page is created, MPM initializes the page’s PageData and adds an SQL record. When save edit content, edit system will be PageData real-time synchronization to SQL database; When the page is published, PageData will be processed into a standard JSON structure and provided to each end parsing engine for processing.

PageData database model

In editing the system, when the operation staff creates a page of the store, the system will generate a default page ID to uniquely identify the page, and at the same time generate an initial PageData and write it into the SQL database. SQL database specially designed a data table to store the page created by operations, it in addition to page_id as a unique identifier, Page_name, page_path, page_type, creator, creation time, page_create_date, page_content, page_content, page_name, page_path, page_type, page_creator, page_create_date, page_content, etc. Where page content page_content is the component configuration of PageData componentConfig, is a serialized JSON string.

The structure model of PageData in the data table is slightly different from its standard structure. Some people may ask: why change the structure model of PageData? Page name, page type, create time and so on, are not all belong to the data attributes of PageData, why to carry out a separate, open a table field to store, serialize the entire PageData directly to store it? So design, because both for management personnel, operation personnel or MPM MPM is the need to provide some necessary retrieval functions, such as “search to create all the pages before time XX”, this time, the independent deposit table fields can make us very convenient to complete the retrieval, and even more complex multi-table join query.

So why is page_content serialized as a JSON string instead of being expanded? This is because the internal structure of page_content is variable and difficult to keep consistent (new template will appear a new combination of attribute fields), and the current retrieval requirements rarely involve harsh retrieval deep into the component configuration, so the maintenance cost of page_content expansion storage is high and the benefit is small. So we simply serialize it and store it in a table field.

How to generate PageData

In the MPM editing system, the editing of a store page generally goes through four stages of loading, editing, saving and publishing, which is also the process of PageData generation.

loading

Loading is also the initialization of the editing system. In this section, MPM pulls PageData from the server for the current MPM page to be edited, which contains all the previous configuration data. Note that the PageData spit out by the server is not yet a standard structure, as mentioned earlier.

Of course, the loading process does much more than that, including user authentication, system configuration, new page, and other judgment execution operations, will be done in this section.

The editor

In this part, PageData will change in real time with the user’s edit configuration. At the same time, with the help of Vue’s monitoring capability (Watch), we have realized an efficient and convenient edit preview, and the effect of each edit can be displayed in the preview area in real time.

save

When a page is saved during editing, MPM submits the latest PageData to the server and updates it to the database.

At the same time, the save operation will also generate a preview page link, which is easy to browse on the terminal. Therefore, when submitting the database, we will also transform PageData into a standard JSON structure, which is provided to the page parsing engine for parsing. For pages of different end environments, the saving process will also be somewhat different, and more specific details will be discussed in the subsequent end parsing process.

release

The save operation does not update the edits online, but simply generates an alternate preview link. To change the results on the line, you need to publish the page. Release will be a default save operation, and according to different page types (different end environment), the implementation of different release process, more detailed we will be in the next PageData analysis of the situation.

In addition, we perform pre-operations such as page diagnosis and RD generation before release, and post-operations such as page accessibility testing and automated testing after release.

PageData analytical

MPM provides different page parsing engines in different end environments, and the parsing process is basically similar but somewhat different. MPM supports H5 and applets. The H5 page supports static and direct access by default. Therefore, the end environment involved in MPM includes static H5, direct H5, and applets.

Static H5 parsing

Static H5 provides a static link, and the page is actually a static HTML file containing the necessary JS (such as the H5-side parsing engine), so static H5 parsing is actually a VUe-based client-side rendering. Before that, static access used to be the main access mode of ONLINE MPM pages. Later, MPM gradually promoted the direct export service. Static links were reduced to Dr Links and only used for Dr Access and preview access.

Save publication

In the save phase, we did a page assembly to generate static pages.

      
<html lang="zh-CN">
  <head>
    <meta charset="UTF-8" />
    <title>{{title}}</title>
    <! -- header JS/CSS -->
    <script>{{topScript}}</script>
    <style>{{topCss}}</style>
  </head>
  <body>
    <! -- Application container -->
    <div class="mpm-app"></div>
    <script>
      // page data
      window.__PAGE_DATA__ = {{pageData}}
      // Render the template
      window.__COM_TPL__ = {{template}}
      // Template extension method
      window.__COM_FNOBJ__ = {{fnObj}}
      // The component hook function
      window.__COM_VUEHOOK__ = {{vueHook}}
    </script>
    <! - engine - >
    {{engineCore}}
    <! -- bottom JS -->
    <script>{{bottomScript}}</script>
  </body>
</html>
Copy the code

This is a simplified assembly template, it’s a string, and you can see there’s a lot of double curly braces in it, and it’s a simple placeholder. Through this assembly template, we replace the standardized PageData, together with MPM template elements that the page depends on, including template, fnObj, vueHook, to the corresponding placeholder, and mount it on the window object of the generated page. The CSS code that depends on it is already incorporated into topCss and placed at the top of the page.

In addition, MPM will also fetch the current version of the parsing engine engineCore through the interface, which is assembled at the bottom of the page. EngineCore is a reference to an external JS:

<script src="//wq.360buyimg.com/martpagemakerv3/web/src/wq.vue2.engine.simplified.acf4f576ed9c5cb9f76d.js" crossorigin="true"></script>
Copy the code

When we publish, we submit the assembled HTML string (in other words, the entire generated page) to the server, which stores it as a static.html file that can be accessed online.

Engine parsing

When the client loads the page, the engine code executes. Because of the order in which the code is executed, the window has already mounted PageData (that is, PageData) and the associated render template, and the engine can be used directly to build the page.

1. Preloading

Before you build a page, the engine actually has a preload. This stage, the engine will be based on user identity configuration __PAGE_DATA__. The userInfo and floor sorting configuration __PAGE_DATA__ pageConfig. Basic. FloorSortId through interfaces for user identity information in advance and floor ordering information page. Obviously, preloading deals mainly with non-floor level data dependencies.

2. Sort & filter

When all the preload interface requests are in place, the engine starts to perform floor sorting. According to the obtained floor BI data, the original page floor configuration __PAGE_DATA__. ComponentConfig is sorted. Then, the FilterPipe of the engine further excludes some floors that do not need to be displayed, including those that are not valid, do not meet the user identity, do not meet the channel type, and finally, based on the new __PAGE_DATA__. ComponentConfig, The engine assembled a floor skeleton, which was added to the MMP-app node as a container for subsequent floor renderings.

<div class="mpm-app">
  <div id="com_1001_con"><div com-root></div></div>
  <div id="com_1002_con"><div com-root></div></div>
  <div id="com_1003_con"><div com-root></div></div>
  <div id="com_1004_con"><div com-root></div></div>
</div>
Copy the code

3. Registration of Vue components/instructions

After that, the engine registers Vue components and Vue directives globally to ensure that they can be used in the template during subsequent page rendering. It is worth mentioning here that in order to reduce the size of the engine JS, we have creatively split the engine into two versions – the full version which contains all the Vue components/instructions, and the simplified version which contains only some commonly used Vue components/instructions and is nearly 150 KB smaller than the full version. Whether a page uses the full version engine or the simplified version engine, we will do a static analysis of the code when the editing system saves the page to determine whether the page uses Vue components/instructions that are not included in the simplified version engine. If so, we will switch to the full version engine. Through regular statistical maintenance, we have enabled more than 85% of our pages to use the simplified engine.

Of course, there is still a lot of unused code, so why not dynamically package a page’s own engine based on dependency analysis when the page is saved? This is mainly due to the risk of dynamic packaging JS code being directly launched without rigorous testing, so we chose the conservative packaging method. Dual-version engine is still a reasonable practical scheme before a perfect test scheme.

4. Floor rendering

Once these steps are complete, it’s just a matter of walking through __PAGE_DATA__. ComponentConfig, using Vue step by step instance nodes and mounting them, and you’re ready to render.

// Page render entry
function renderPage () {
  __PAGE_DATA__.componentConfig.forEach(createComponent);
  // ...
}

function createComponent (com) {
  const comId = com.id;
  // El is the page node generated in Step 2 and is the floor container
  const el = document.querySelector(` #${comId}_con>[com-root]`);

  const data = utils.copy(__PAGE_DATA__[comId]); // Make a deep copy of configuration data
  data.fnObj = __COM_FNOBJ__[comId]; // Render function

  new Vue({
    el, / / container
    data, // Configure data
    render: __COM_TPL__[comId], // Render function
    mounted () {
      __COM_VUEHOOK__[comId]['mounted'] ();// The hook function}}); }Copy the code

As for the business interface data of the floor itself, it has been mentioned in the visual building system of MPM store — Element Design before that it is completed by a DS component. The DS exists in the template and is executed when the upper floor is instantiated to initiate requests, receive data, and trigger rendering again.

Straight out the resolution of H5

To address the problem of poor first screen experience of static H5, MPM created a highly available Node service that provides straight out capability for all H5 pages. On the outbound side, the MPM page parsing engine renders only the first screen, and the rest of the page is rendered by the client after it reaches the client.

Save publication

When H5 saves publication, on the one hand, it will assemble and generate static H5 for direct out Dr, and on the other hand, it will submit PageData (including configuration data, MPM template, page header and tail and other unassembled code blocks) to Node server and write into Redis.

Engine parsing

MPM Node outbound terminal is based on Express framework design, carrying MPM outbound parsing engine. Similarly, the MPM outbound engine has the same set of Vue components built into it as the static H5 engine logic.

1. Read Redis

The outbound end is a unit of access, and each access has a separate context. When a User accesses a direct outbound page, through Express middleware, MPM direct outbound will first initialize an access context, which contains the URL parameter of the page, request Cookie, user-Agent and other information. At the same time, according to the request parameter pageID, Page data is retrieved from Redis and incorporated into the access context for subsequent page assembly.

2. Preload, sort & filter

Preloading and sorting filter links also exist at the straight out end, which is not much different from the static H5 end, and will not be described here.

3. Page data request

This is one of the biggest differences between the outbound model and the client model. On the client side we can render for several times, so we use the update of the Vue responsive, let data request delay treatment, but on the straight out of the end, we implement the streaming rendering, one and only one train rendering, rendering requirements before rendering all data must be in place, so in straight out of the end, we must advance to request data page.

So in this section, we’re actually doing what the DS component does ahead of time. We collected the business interface requests to be launched by each floor, processed them uniformly and stored them in the cache object. When rendering is performed to DS component, DS will first check whether there is cache data in the cache object. If so, it will be directly used. Of course, here we only consider the first 20 floors as the first screen of the page, the remaining floors are rendered by the client.

In fact, the specific implementation scheme here is not the best, on the contrary, there are many unreasonable, so I will not elaborate here. We are currently working on this. In future articles, we will discuss the evolution of the MPM data model and what data model can better fit the front and back rendering of MPM.

4. Page rendering

After data is in place, we use vue-server-renderer to complete vUE server-side rendering:

import { createRenderer } from 'vue-server-renderer';
const renderer = createRenderer();

// Page render entry
async renderPage () {
  // Outputs the page header
  await context.res.write(pageTop);
  // Output page floor content
  await renderApp();
  // Output the end of the page
  await context.res.write(pageBottom);
}

// Render the page body
renderApp () {
  return Promise((resolve, reject) = > {
    // Create a Vue instance of app
    let app = new Vue({
      data: rootData,
      methods: rootMethods,
      render: renderFn
    });

    // Create a render stream
    const stream = renderer.renderToStream(app);

    // segment output
    stream.on('data', chunk => {
      // ...
      context.res.write(chunk);
    });

    stream.on('end', () = > {// ...app.$destroy(); resolve(); }); })}Copy the code

In this step, instead of creating a Vue instance for each floor, we will merge PageData and the template and render it with a single root Vue instance so that we can create a render stream using renderToStream. The segmented output to the client is complete.

Small program parsing

MPM in advance in the business small program to create a set of system components/template one to one correspondence, UI 100 percent reduction of small program components, small program rendering is actually according to PageData, these have been prepared small program components out of the combination into the desired page.

Save publication

MPM applet pages are separate page types from H5 pages, and the editing process is separate. When the MPM applet page is saved and published, it just submits the standardized PageData to the server and generates a JSON file.

Engine parsing

Get PageData

When an MPM page is opened in a small program, the engine will first request the page’s corresponding PageData.

2. Render the page

Applets page parsing, in essence is also a client rendering, so this step is actually no different from static H5 rendering, similarly, first preloading and floor sorting, and then according to PageData floor configuration, select the corresponding applets component rendering, and finally render the entire page.

At the end of

Once you understand the entire MPM flow mechanism, it’s easy to see that the implementation logic of different end engines is too similar! We had to change the static side, the straight out side, and the small side of the code at the same time every time we had a requirement iteration, and repeated testing was extremely costly and difficult to maintain (you really can’t guarantee that the same code in three places won’t be different). In this case, three-terminal isomorphism, as an effective solution appears in front of us, “one writing, three-terminal operation” is undoubtedly a good vision. However, in addition to the technical difficulty of three-terminal isomorphism, THE DEVELOPMENT of MPM has been very large, its internal logic is very complex, to complete the reconstruction of this volume, how to achieve isomorphism compilation, how to complete the system compatibility transition and other issues, need to invest more thought.

---

If you think this article is valuable to you, please like it and follow us on our official website and WecTeam. We have excellent articles every week