This article mainly around the monkey small program debugging technology third edition.

As mentioned in the last introduction article, the debugging part of little Monkey program has gone through three versions. This article will describe in detail how debugging for developers is implemented.

The following sections will be described:

  • Debug the basic communication relationship structure of the implementation.
  • How to implement full DOM review capabilities.
  • How to implement Console.
  • How to implement Source and breakpoint debugging.
  • How to implement the review of network records.
  • How to implement page-based data review.

Overview of basic communication relationship structure

The figure above completely expresses the relationship between the key parts of the implementation of small program debugging technology. It is divided into the following key roles:

  1. Debugging panel: DevTools frontend.
  2. Example Debugging the trunk service.
  3. Render layer page stack.
  4. The logical layer runs the container.

The debug panel

The debug panel is consistent with the developer tools that we developers use all the time. It comes from the Chrome DevTools Frontend project, a developer tool embedded in the Chrome browser.

But here I’m using the May 1, 2020 version. Since this release, the entire Chrome DevTools Frontend is written in TS. This makes debugging awkward, as each change takes about 10 minutes to compile and run.

Here’s an article that gives a brief overview of DevTools Frontend

] (zhaomenghuan.js.org/blog/chrome…).

Chrome DevTools Frontend uses Websockets to communicate with the outside world. The internal message is: Chrome Devtools Protocol. By default, after frontend is started, a debug source is passed to frontend via a Url. Similar to this: http://localhost:8090/front_end/devtools_app.html? Ws = 127.0.0.1:9222 / devtools/browser / 2 b82a4a4 – c047-40 f8 – bbc4-7 a09fdc501d3.

So where do you get the debug source information? Often engines that execute JS code expose debug source information to outsiders. For example, the Node:

The node - inspect - BRK index. Js Debugger listening on the ws: / / 127.0.0.1:9229/51 d4ee96 e04-3759-4-8 b2a - c0fd8a1c5db2 For help, see: https://nodejs.org/en/docs/inspectorCopy the code

For example, Electron:

/ / external debug information released by the following way. App.com mandLine appendSwitch (" remote - was debugging - port ", 8820); App.com mandLine. AppendSwitch (" remote - was debugging - address ", "http://127.0.0.1"); / / for debugging information by the following way, the interface will return all the processes in the Electron source debugging http://127.0.0.1:8820/json/list / / return the result: [{" description ": "", "devtoolsFrontendUrl": "/ devtools/inspector. HTML? Ws = 127.0.0.1:9333 / devtools/page / 547 b0c088951191189b7878999ba8048", "id" : "547 b0c088951191189b7878999ba8048", "title" : "127.0.0.1:8886 / dashboard. HTML", "type" : "page", "url" : "Http://127.0.0.1:8886/dashboard.html", "webSocketDebuggerUrl" : Ws: / / "127.0.0.1:9333 / devtools/page / 547 b0c088951191189b7878999ba8048"}, {" description ":" ", "devtoolsFrontendUrl" : "/ devtools/inspector. HTML? Ws = 127.0.0.1:9333 / devtools/page / 6457 e98873e6402b53c79c8a374723f6", "id" : "6457 e98873e6402b53c79c8a374723f6", "title" : "127.0.0.1:8886 / main HTML", "type" : "page", "url" : "Http://127.0.0.1:8886/main.html", "webSocketDebuggerUrl" : Ws: / / "127.0.0.1:9333 / devtools/page / 6457 e98873e6402b53c79c8a374723f6"}]Copy the code

Specific information can check: www.electronjs.org/docs/latest…

With the debug source in hand, you can make frontend establish a WS connection with the debug source for code debugging control. This is not enough for an applet developer because Frontend only connects to one source, and applet is actually separated from the rendering layer by two sources. So either you debug the logical code, or you do DOM review. So what to do?

The initial solution was to create two instances of Frontend, each connected to the debug source for the logic layer and rendering layer, and then switch the two frontend instances in a clever way when switching DOM or Console. But ultimately feel this way is not very orthodox, just a way to cheat.

Thus the debug relay service was born.

Debugging the Trunk Service

The pink area in the figure above represents the debug relay service. A relay service does some things:

  • Undertake communication with frontend.
  • Responsible for communication with logic layer, rendering layer debug sources (DOM, Console, Source, Network, Memory, etc.).
  • Undertake communication with the logical layer runtime container (page data review).

Therefore, the relay service plays the role of message distribution and message summary. It is responsible for the seamless communication between frontend and the logic layer and the rendering layer debug source.

Debugging the trunk service has one WebSocketServer and two WebSocketClients. WebSocketServer communicates with Frontend. The two WebSocketClients are responsible for communicating with the logical layer debug source and the render layer debug source respectively. In addition, it holds a reference to the logical layer run container, which is responsible for executing some JS scripts, which are used in the data review function.

Render layer page stack

Applets are multi-page, which is quite different from regular H5 applications. So the render layer page stack appears. It is responsible for managing all pages, such as loading, unloading, destroying, and so on. The object that the debug relay service connects to is always the debug source of the BrowserView at the top of the page stack.

The logical layer runs the container

The logical layer runtime container is responsible for the execution of logical code. After the applet code is compiled, a file like logic.js is generated. In order to create a clean operating environment, a worker is added to the operating environment of the logical layer to ensure that the logical code written by the developer cannot obtain external API, such as window or Document.

Ok, now that I’ve covered the four key roles, I’ll describe in detail how each debugging tool is implemented.

The implementation of debugging tools

As front-end development, we often use the following debugging tools:

  • Elements is used for DOM review and CSS style review.
  • Console is used to view logs and execute scripts on the Console.
  • Source is used to view running code and control the execution of debugging.
  • Network Is used to view all Network access records. Includes resources, XHR, WS, and so on.
  • Inspect is not integrated in the browser, but we often use debugging tools like vue-devtools and react-devtools. It allows you to view the data declared on the page and a modification to the data.

Before getting into the panels, LET me briefly describe what goes into debugging when the applets developer tool is launched:

  1. Small program developers start.
  2. Example Start debugging the trunk service.
  3. Start the Frontend static resource service.
  4. Tell the applets developer tool to load frontend.
  5. Frontend establishes a WS connection with the debug relay service.
  6. Start waiting for the applet to load.

What does the relay service do after the applet loads?

  1. Scan all the debug sources supplied by Electron.
  2. Filter out logic layer debug source and render layer debug source.
  3. Establish WS connections with logical layer debug sources and render layer debug sources respectively.
  4. Send frontend messages to both logical and render debug sources.
  5. Finally, the debug source communicates properly with frontend.

Elements

An implementation of Elements is shown above. Since we are using the chrome DevTools Frontend source code directly, nothing needs to be changed here. However, in order to ensure the normal use of functions, there are some additional processing needs to be done:

  1. Because Frontend starts right after the developer tool starts. Frontend immediately sends messages to the debugged source, which in this case is received by the debug relay service. The applets haven’t been loaded yet, so you can’t do simple forwarding yet. The relay service needs to cache the message first, and then send the message to the logical layer debugging source or the rendering layer debugging source respectively after the applets are loaded. As shown below:

2. Because the applet is multi-page, the DOM structure of the current page needs to be refreshed to Elements whenever a new page is opened or returned. Elements’ DOM refresh presents some challenges here. Since page stack switching is an external initiative, how do we tell frontend that it’s time to update the DOM? We have to take a closer look at the source code of Frontend to understand how frontend first gets DOM information. See how it can be triggered to request the DOM again based on its internal context. I was inspired by the browser’s page redirection feature. Finally, after a lot of exploration and practice, WE know that DOM updates can be triggered in the following ways:

// Trigger page DOM update
frontendWS.send('{"method":"DOM.documentUpdated","params":{}}');
Copy the code

But is that it? No, there’s more. It turns out that neither the hover element nor the style modification function works. Therefore, after another round of investigation, it was found that some abilities needed to be turned on before they could be used:

// Send a message to the render layer
viewWS.send('{"id":2,"method":"Page.enable"}')
viewWS.send('{"id":3,"method":"Page.getResourceTree"}')
viewWS.send('{"id":5,"method":"DOM.enable","params":{}}')
viewWS.send('{"id":6,"method":"CSS.enable"}')
viewWS.send('{"id":10,"method":"Overlay.enable"}')
viewWS.send('{"id":11,"method":"Overlay.setShowViewportSizeOnResize","params":{"show":true}}')
viewWS.send('{"id":25,"method":"Page.setAdBlockingEnabled","params":{"enabled":false}}')
Copy the code

The sequence diagram at this point is as follows:3. Because of the special nature of the relay service, it is necessary to identify which messages are destined for the rendering layer debug source and which are destined for the logical layer debug source. That’s what you need to doChrome DevTools ProtocolThere is a very clear understanding. The basic logic is as follows:

if (message.includes('DOM') || message.includes('CSS') || message.includes('Page') || message.includes('Overlay')) {
    // Send to the render layer debug source
} else {
    // To the logical layer debug source
}
Copy the code

Console & Source & Network & Memory

All four tabs are connected to logical layer debugging sources, so they are not as complex as Elements. You only need to ensure that the information between frontend and the debugging source of the logical layer is unblocked. However, the following three points still need special treatment:

  1. Pages cannot be connected to the logical layer debug source again as they go in and out of the stack.
  2. The logical layer running container needs to be reload when the applet is reloaded.
  3. If you need to listen for network conditions, you need to add Ajax requests to your logical code.
  4. To properly execute certain scripts on the Console, you need to select the execution context of the worker.

DataInspect

In general, the logic of the above five parts is not complicated. You just need to ensure that the information is distributed properly, the order is not out of order, and the data is not lost. But the data review that I’m about to introduce is a lot more complicated.

Data review is similar to vue-DevTools in that it provides the ability to view and modify data declared on a page. This capability must be independently developed from zero to one. Because Frontend doesn’t have that capability.

The first question I was asked was: How do I add a TAB to frontend?

Add tabs to frontend

This question has puzzled me for a long time. The ability to successfully add a TAB to the debug panel has previously been used with Chrome Extensions, but this ability is only available in Chrome itself or using the developer tools that come with Electron. And we here in order to highly customized, had to use chrome DevTools frontend source code to do. Frontend does not support this capability, so I started on the path of in-depth reading of frontend source code.

After a lot of debugging + reading, I know how frontend loads a TAB. I have to say a little bit about the TAB loading mechanism of Frontend.

The frontend root project has a file like this:

All modules that need to be dynamically loaded by Frontend are declared in devTools_app. json. The name of each module represents the directory in which the module resides. For example, the data review directory is inspect, where you need to add the declaration {“name”: “inspect”}.

After you have configured the module name to load, you need to know how the module is loaded. You need to define a file named module.json in the root directory of each module. In this file, you need to specify key information such as the resource information to be loaded for the current module, the position of the TAB, and the panel corresponding to the TAB. For example, inspect/module.json is defined like this:

{
  "extensions": [{"type": "view"."location": "panel"."id": "inspect"."title": "Inspect"."order": 101."className": "Inspects.InspectsPanel"}]."dependencies": []."scripts": []."modules": [
    "inspect.js"."inspect-legacy.js"."ElementsPanel.js"]."resources": []}Copy the code

Once defined in this way, Frontend starts loading and executing these files in a certain order. The key codes are as follows:

// root/ runtime.js some key codes
_loadPromise(){...const dependencies = this._descriptor.dependencies;
  const dependencyPromises = [];
  for (let i = 0; dependencies && i < dependencies.length; ++i) {
    dependencyPromises.push(this._manager._modulesMap[dependencies[i]]._loadPromise());
  }
  this._pendingLoadPromise = Promise.all(dependencyPromises)
                                 .then(this._loadResources.bind(this))
                                 .then(this._loadModules.bind(this))
                                 .then(this._loadScripts.bind(this))
                                 .then(() = > this._loadedForTest = true);
  return this._pendingLoadPromise;
}

_loadModules() {
  if (!this._descriptor.modules || !this._descriptor.modules.length) {
    return Promise.resolve();
  }
  const namespace = this._computeNamespace();
  self[namespace] = self[namespace] || {};
  const legacyFileName = `The ${this._name}-legacy.js`;
  const fileName = this._descriptor.modules.includes(legacyFileName) ? legacyFileName : `The ${this._name}.js`;
  return eval(`import('.. /The ${this._name}/${fileName}') `);
}
Copy the code

This way you can load and run the file declared in module.json. But we haven’t created a TAB yet. We have to keep going down.

After all the resources have been loaded, the master comes out. Main/mainimp.js is responsible for building the entire panel. Inside the constructor, the _loaded method is triggered after the DOMContentLoaded event completes. This method triggers the instantiation of the entire Frontend page.

// main/MainImpl.js 306
_showAppUI(appProvider){...constapp = (appProvider).createApp(); . app.presentUI(document); . }Copy the code

The above part of the code will mount the entire panel to document to make the content appear on the page.

The top Tab will then be constructed using the UI/inspectorView.js class. Key code:

export class InspectorView extends VBox {
  constructor() {
    super(a); .// Create main area tabbed pane.
    this._tabbedLocation = ViewManager.instance().createTabbedLocation(
        Host.InspectorFrontendHost.InspectorFrontendHostInstance.bringToFront.bind(
            Host.InspectorFrontendHost.InspectorFrontendHostInstance),
        'panel'.true.true, Root.Runtime.queryParam('panel')); . }Copy the code

Then construct the corresponding panel class through dynamic instantiation:

_createInstance() {
  const className = this._className || this._factoryName; .const constructorFunction = self.eval((className)); .// instantiate the corresponding class
  return new constructorFunction(this);
}
Copy the code

The class we define in module.json is Inspects.InspectsPanel. The code for this class looks like this:

import * as UI from '.. /ui/ui.js';

export class InspectsPanel extends UI.Panel.Panel {

    constructor() {
        super('inspect');

        const vueRootNode = createElement('div');
        vueRootNode.id = 'vue-root'
        this.element.appendChild(vueRootNode);

        self.ee = new EventEmitter3();

        new Vue({
            render: createElement= > createElement(inspect_frontend),
            el: vueRootNode
        })

    }

    / * * *@return {! InspectsPanel}* /
    static instance() {
        return / * *@type {! InspectsPanel} * /(self.runtime.sharedInstance(InspectsPanel)); }}Copy the code

InspectsPanel the InspectsPanel class is the page to which the data review Tab corresponds. All of frontend’s pages are written with controls defined in the project, which is very complex, and there is no reference documentation, so the logic that will be implemented here is appropriate for Vue implementation.

In the above code, Vue renders an Inspect_frontend. Where did this come from? This is a Vue project that I wrote separately, and then built a library through VUe-CLI, which was mounted globally. It’s just a render function. The running effect is as follows:

Okay, so now I’m ready to write concrete stuff on frontend via Vue. We’ve reached a critical point. Now comes the part about how to get the review data.

Review the data acquisition process

In order to accomplish the basic capabilities, there are three basic functions needed:

  1. Gets all the current pages of the applet.
  2. Gets page data for a page of the applet.
  3. Modify data.

Since all the data in the applet is controlled by the logical layer, I just need to go to the logical layer to get the data. Although it sounds simple, the actual process is very complicated. So where’s the complexity?

Let me briefly describe the process:

  1. The Data review office sends messages through WS to the relay service.
  2. The relay service recognizes the message after receiving it.
  3. After identification, a message is sent to the logical layer container by executing the JS script.
  4. After receiving the message, the logical layer container sends the message to the worker through postMessage.
  5. The worker executes the query after receiving the message.
  6. After worker’s functions are executed, messages are sent to external logical layer containers through postMessage.
  7. After receiving the message, the external logical layer container sends the message to Electron through IPC.
  8. Eletron receives the message and sends it to the relay service for identification.
  9. After the trunk service is identified, the corresponding callback method is found and triggered.
  10. The callback method remessages frontend via WS.
  11. The frontend then identifies the corresponding callback to the upper-level caller.

The logic here can be seen in the third edition of the operation diagram (red arrow direction below).

The entire process goes through nine objects, with the data wrapped layer by layer and then unwrapped layer by layer. It’s as complex as a computer network.

Query page data

When opening the data review panel, the first thing you need to know is which pages are currently open. As shown below:

With the procedure described in the previous section, it is relatively easy to query the page data here. In business code, you just need to query like this:

getPages() {
  sendMessage('getPages').then(res= > {
    this.pages = res;
  }).catch(err= > {
    console.error(err); })}Copy the code

SendMessage is intended to facilitate data review using abstracted public methods:

export function sendMessage(method, params) {
    return new Promise((resolve, reject) = > {
        self.target._router.sendMessage(""."DataInspect".`DataInspect.${method}`, params, (error, result) = > {
            if (error) {
                console.error('Request ' + method + ' failed. ' + JSON.stringify(error));
                reject(null);
                return; } resolve(result); })})}Copy the code

The third line, self.target, is a basic communication object inside frontend for upper-level business use. It is responsible for sending messages to the outside world via WS and calling the received messages back and forth to the corresponding upper-level callback methods.

Self. target, a mechanism to ensure that callback messages don’t mess up, was an eye-opener for me.

After receiving the message, the relay service performs a simple identification:

// Data review message distribution
if (message.includes('DataInspect')) {
    const dataInspectMessageBody = JSON.parse(message);
    if (dataInspectMessageBody.method === 'DataInspect.getPages') {
        // Get all page information
        const result = await this.executeCodeInAdapter('globalVar.getRoutes()');
        this.sendMessageToDataInspect(dataInspectMessageBody.id, result); }}Copy the code

The executeCodeInAdapter above is responsible for executing the specified script from the logical layer container. The globalvar.getroutes () argument is a function defined in the logical code. It will be passed layer by layer and finally run in worker:

// The adapter.js that the worker is running
function dispatchMessage(message) {
    switch (message.type) {
        case 'loadCode':
            loadLogicFile(message.content);
            break;
        case 'runCode':
            const result = eval(message.content);
            syncExecuteResult({
                executeResult: result,
                executeExpression: message.content,
                executeId: message.id,
            });
            break;
        default:
            console.warn('Unprocessed message types, please note:${message.type}`);
            break; }}Copy the code

Finally, globalvar.getroutes () is executed by the eval function. And synchronously returns a result. However, the result cannot be returned to the outside world synchronously. It has to be sent to the outside world through a series of processes, and the result is finally distributed in ELECTRON:

onMessage({ type, args }) {
  // Synchronize the execution result
  if (type === 'syncExecuteResult') {
    const resolveCallback = this.resolveMap[args.executeId];
    if(resolveCallback) { resolveCallback(args.executeResult); }}else {
    // Other communication messages, such as communication with the rendering layer
    this.options.onMessage(messageObj); }}Copy the code

ResolveMap in the above function is the callback method that retains all asynchronous messages. When there is a result, it can execute the corresponding method according to the ID.

The logic above is borrowed from the self. Target logic of frontend.

When executed resolveCallback, above the await this. ExecuteCodeInAdapter (‘ globalVar. GetRoutes () ‘) will continue to execute, and get the result. The result is returned to the business invocation via WS via sendMessageToDataInspect.

At this point, query all pages of information logical path complete.

Query all data for a specified page

Similar to querying page data, it is based on the same communication mechanism. Just in a different way:

sendMessage('getPageData', {
  pagePath: this.path // Page path, for example, pages/index/index
}).then(res= > {
  this.pageData = res;
}).catch(err= > {
  console.error(err);
})
Copy the code
Sets an item of data for a specified page

This is the same with querying data:

sendMessage('setPageData', {
  pagePath: path,
  data: {
    // Edit key and value
    [this.editedKey]: value
  }
}).then(res= > {
  this.$emit('submit-edit');
  self.ee.emit('refresh');
}).catch(err= > {
  console.error(err);
})
Copy the code

Sending messages is the same, but the logic behind sending messages is a little different. After sending the edit message, you need to re-query the data for the current page and notify DOM updates.

Listen for active changes in the render layer

What does it mean to listen for active changes in the render layer? That is, there will be active behavior triggered by the render layer. For example: click checkbox, enter content in the input field, etc. These situations also require real-time feedback to the data review panel. The previous three methods are active to the logic layer of the process, but this is the rendering layer active trigger. So how do you solve this problem?

Here’s my idea:

  1. When the data review panel is turned on, actively register a method with the logical layer.
  2. When data changes in the rendering layer, the logic layer is notified through a series of procedures.
  3. The logical layer then triggers the method registered in step 1 to trigger the callback in frontend.
  4. When frontend receives a callback, it registers again.

The specific implementation logic is as follows:

  1. Active registration in frontend.

This step is no different from the above three data query methods.

// Actively register callbacks in frontend
sendMessage('registerRefreshEvent').then(this.getPageData);
Copy the code
  1. Active registration in the relay service.

After the secondary service receives the active registration request, it will make special treatment. Expose the Resolve method externally.

// Relay service message distribution
if (dataInspectMessageBody.method === 'DataInspect.registerRefreshEvent') {
       // Register data update callback
       console.info(Registered data update callback);
       await new Promise(resolve= > this.refreshEventResolve = resolve);
       this.sendMessageToDataInspect(dataInspectMessageBody.id, "");
}
Copy the code
  1. The relay service actively registers the method into Electron.
this._messager.registerEventCallback('1'.command= > {
   if (command === 'refresh') {
       // Actively refresh
       console.info('Request data review refresh');
       this.refreshEventResolve(); }})Copy the code
  1. Waiting to be called back. The process of the callback is the same as the message communication above. After receiving the render layer message from the logical layer, actively send the message to Electron. After Electron receives the message, it calls back the trunk service. The refreshEventResolve method that the relay service resolve is just waiting for.
  2. The trunk service then sends messages to Frontend.

The sequence diagram of this process is as follows:

So this is how messages in frontend communicate. Let’s talk about how pages are made.

Data Review UI

As shown above. The initial idea was to write a completely self-contained page, but then I thought, this must be a lot of work, why not use a third party UI component?

As a result, the page data was initially represented with ElementUI. This will be done in no time. But what about specific data viewing and editing?

Because data viewing involves a tree structure review and all sorts of editing interactions. So it feels like a lot of work. Can I take the vue-DevTools code and use it?

After a lot of code transfer and data adaptation. It was a difficult process, but it was successfully grafted. Considerable amount of work has been saved. There is little more to this process than a lot of reading of the vue-DevTools source code.

This is a detailed description of all the panel implementations in debugging. Finally, to sum up.

Again, the above picture is the end of the illustration.

First, to solve the problem of one debug panel versus two running containers, the debug relay service is introduced. The relay service is responsible for the distribution and aggregation of messages. Just make sure that the Chrome Devtools Protocal protocol flows properly to complete basic debugging capabilities. It is also compatible with new page loading and small program reloading.

But this is not enough, because data review needs to be implemented from zero. In order to solve the problem of data review, I tried to figure out how to add a TAB and how to add a panel to frontend from the source point of view. The panel is Vue used here. The data required for the data review went through nine object flows before it was opened. For UI, ElementUI plus vue-DevTools a lot of source code is used.

That’s all I want to share about debugging applets.