One, the introduction

Since Google released the first version of Chrome in 2008, the whole Web development field seems to have been injected with a fresh blood, gradually breaking the era of IE alone. Chrome and Firefox are strong supporters of the W3C Web standards, and as these two open source browsers gain market share, developers are enjoying a spring in their step. This ushered in a new professional division of labor – frontend-engineer, front-end engineer to promote the prosperity of Web applications, powerful debugging tools are essential. Google’s release of DevTools, which is based on open source, has been well received by web developers and has contributed to Chrome’s commercial success.

This article analyzes the technical implementation of DevTools for Chrome, especially in the browser kernel, to reveal the secrets behind this beloved development tool by millions of developers. This article is intended for front-end developers, application engineers who are interested in developing Hybrid application debugging tools or rewriting WebDriver to control Chrome or WebView.

Note: All code analysis in this article is based on Android Chromium 87.0.4280.141 version analysis. As the author’s team is mainly engaged in the development of Blink kernel on Android platform, the analysis process is mainly focused on mobile terminals, while other platforms are only different in data path, with little difference in implementation principles.

History of web debugging tools

Before 2006, this belongs to the ERA of IE. In the era of IE, the debugging means of JavaScript code writing mainly rely on window.alert() or output debugging information to the web page to analyze logic bugs. This means of hard debugging is no less than the low-level development of the system. It often takes a whole day for a small problem, and the development efficiency is extremely low.

In January 2006, Apple’s WebKit team released the first version of Web Inspector, which was relatively simple and could only look at the inheritance of DOM nodes and what CSS rules were applied to them. But this version has laid the foundation for years to come web debugging tool prototype, has the epochal significance.

The WebKit team iterated very quickly, and in June 2006 they released a heavyweight feature, breakpoint debugging for JavaScript, which was already in the shape of a developer artifact.

At the same time, a Firefox plug-in Firebug appeared in the open source camp, focusing on the debugging of Web development, laying the layout of the modern DevTools Web UI. Earlier versions supported JavaScript debugging, visual presentation of CSS Box models, and performance analysis of HTTP Archive. Later DevTools referenced the functionality and product positioning of this plug-in. Firebug was integrated into Firefox’s built-in debugging tool in 2016, and Firebug stopped updating in 2017, marking the end of a generation of magic tools.

At this time, a malicious Google team in the open source community joined the browser research and development based on WebKit, and launched Chrome to attract the attention of many IT geeks with “safety, speed and stability”. At the same time, in terms of developer tools, Google absorbed the excellent functions of many debugging tools. Introduced today’s main character DevTools.

Earlier versions now look a little primitive, but this is more than a decade old. Support DOM + CSS view, view resource loading analysis, script debugging and performance debugging. These are the basic features of DevTools used in development today.

DevTools of that era basically followed Firebug’s lead, but the difference was in the way they interacted. In 2007, Steve Jobs released the first generation oF iPhone, and Google successively launched Android phones. The development of the Internet came to the era of mobile Internet. DevTools began to leapfrog its peers by supporting remote real machine debugging. Chrome is a multi-process architecture, with DOM and JavaScript running in child processes, so the underlying implementation of DevTools is completely different from its peers. Chrome’s architects have tuned the DevTools implementation architecture to client-server mode, which makes remote real machine debugging possible. Chrome developed a data encapsulation protocol called Chrome DevTools Protocal(CDP) to facilitate the transfer of data over the Web, and over the next few years this architecture has been tweaked in the open source world.

Yan Dahl designed Node.js based on the Chromium JavaScript virtual machine V8, and the launch of Node.js took JavaScript, a Web scripting language, out of the browser. This opens up the possibility of server-side programming and desktop programming using JavaScript. As DevTools’s client-server architecture and the number of Node.js developers grew, DevTools quickly took off, and the Chrome team began supporting Node.js debugging in 2016. DevTools has evolved from a Web debugging tool to an important part of the JavaScript ecosystem, enabling more developers to create more good code. The node.js ecosystem is inseparable from DevTools, For example, The desktop development framework Electron, the developer’s favorite editor Visual Studio Code, the front-end architecture Vue.js, and Stetho, Facebook’s open source Android performance analysis tool.

DevTools architecture

DevTools is a client-server architecture. A client is a Web UI interface for user operations. It receives user operation instructions and sends them to the browser kernel or Node.js for processing, and displays the processing results on the Web UI. The server starts two types of services, one HTTP service; Another WebSocket service.

The HTTP service provides the kernel information query capability. Such as getting the kernel version, getting a list of debug pages, and starting or closing debugging.

The WebSocket service provides the ability to communicate real data with the kernel, distribute and process all operation instructions passed from the Web UI, and send the results back to the Web UI for display.

The diagram below shows the overall architecture of Android DevTools. From the left, the developer initiates action commands through the Web UI. Browser Core (Browser Core runs Chrome kernel applications, such as Chrome Browser, Android WebView, NodeJs applications, etc.).

The Android platform cleverly uses the ADB Forward capability to solve the problem of connecting the web user interface (WebUI) on a PC to the Chrome kernel on an Android phone. The ability to easily implement remote debugging is not to be sniffed at, which is a huge productivity boost for front-end developers. Because the front-end developer’s working environment, currently basically in PC (Windows, Mac, Linux collectively referred to as PC), through the implementation of remote debugging ability, so that the development of mobile terminal to achieve wySIWYG.

It was the Chrome team’s web-based approach to communication, as the underlying DevTools communication framework, that laid the foundation for subsequent Web development teams to blossom. TCP/IP is the foundation of the Internet. There is no language or platform that doesn’t support TCP/IP. The DevTools selection of TCP/IP directly smooths out the differences between platforms or system frameworks.

Chrome DevTools Protocol(CDP) this group of open Protocol, once again the implementation of DevTools, truly achieve cross-platform. CDP is a set of data encapsulation protocols in JSON format. JSON is a lightweight text exchange protocol that can be parsed on any platform or language. For this reason, there are nearly ten languages that are officially recommended to support CDP. Google officially recommends the Node.js version of Puppeteer, which fully implements the CDP protocol and provides a similar way for the Chrome kernel to communicate. Subsequently, the open source world has launched CDP libraries of multiple language versions. The CDP protocol is described in detail in a later chapter.

The Chrome architect abstracts the underlying architecture of DevTools into TWO parts, TCP/IP and CDP, through a high degree of abstraction, which lays the foundation for The cross-platform and cross-terminal capabilities of DevTools. When the WebSocket implementation was still in the draft stage, the Chrome architect boldly adopted WebSocket to implement the main protocol part of the debugging protocol. Now it seems that the real-time screenshots of the daily use of the developer can be used to observe the interface displayed in the remote web page in real time. This real-time capability is provided based on WebSocket. I also admire the vision and design energy of the Chrome architects, who have taken web developer tools to new heights.

DevTools communication protocol

Chrome DevTools Protocol (CDP) This Protocol consists of two parts: HTTP and WebSocket. The Web UI of DevTools sends control commands to the browser kernel. Control commands, parameters, and returned values are encapsulated through THE CDP. Commands are encapsulated by the Web UI and sent to the browser kernel through the WebSocket. After receiving the result from the browser kernel, unpack it according to the protocol and distribute it to the Web UI.

To analyze the communication between the Web UI and the Android browser kernel, you need to prepare the environment.

4.1 Environment Preparations

In order to access the data in the kernel, the browser kernel needs to have DevTools Server enabled. PC Chrome and Android Chrome/WebView are enabled in slightly different ways.

Add a startup parameter -remote-debugging-port=9222 when Chrome starts on PC. DevTools Server will listen on the local port. You can get data from DevTools by making an HTTP/WebSocket request to http://localhost:9222.

Android Chrome is slightly different from WebView. WebView is disabled by default. You need to manually enable WebView on the client to start the Server.

// WebView only uses Blink kernel after Android 4.4, so it needs to be in this version or later.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
    WebView.setWebContentsDebuggingEnabled(true);
}
Copy the code

At this point, The Android Chrome/WebView Server has been started in the mobile phone, but to be accessible on the PC, you need to use the port forwarding capability of the ADB tool.

ADB port forwarding You can use the forward command to set any port forwarding to forward requests on a specific host port to other ports on the device. The following example sets the host port6100Port to device7100Adb forward TCP:6100 tcp:7100The forward command enables the PC and Android device to access each other on the networkCopy the code

Android Chrome/WebView Server with Unix Domain socket

Chrome_devtools_remote and webview_devtools_remote_ are chrome and WebView linkers respectively. WebView connection Because different applications may use webViews, the process ID (PID) is used as a suffix to distinguish between them.

adb shell cat /proc/net/unix | grep "devtools_remote"
0000000000000000: 00000002 00000000 00010000 0001 01 528176 @chrome_devtools_remote
0000000000000000: 00000002 00000000 00010000 0001 01 276394 @webview_devtools_remote_23119
Copy the code

Use ADB Forward to connect the PC to the Android device and run the following command:

# Listen on PC9222Port for localhost:9222Adb Forward TCP on webview_devtools_remote_23119:9222 localabstract:webview_devtools_remote_23119
Copy the code

At this point, you can access the debug page on your Android device from your PC via 9222.

4.2 HTTP Protocol Analysis

4.2.1 Obtaining kernel Version Information

$curl = $curl $curl = $curl//localhost:9222/json/version
curl http://localhost:9222/json/version                       
{
   "Android-Package": "com.vivo.browser"."Browser": "Chrome / 87.0.4280.141"."Protocol-Version": "1.3"."User-Agent": "Mozilla / 5.0 (Linux; Android 8.1.0; Vivo X20Plus A Build/OPM1.171019.011; Wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/87.0.4280.141 Mobile Safari/537.36"."V8-Version": "8.7.220.31"."WebKit-Version": "537.36 (@ 9 f05d1d9ee7483a73e9fe91ddcb8274ebcec9d7f)"."webSocketDebuggerUrl": "ws://localhost:9222/devtools/browser"
}
Copy the code

From the value returned above, the following information can be obtained:

Android-package, using the Package name of the WebView application.
Browser, the kernel version number.
Protocol-version: indicates the Protocol Version of the CDP. The current Version is 1.3, starting from 1.0, and 1.1 and 1.2.
User-agent: UA information of the browser.
V8-version, the Version number of the JavaScript engine used.
Webkit-version. Since the Blink kernel is developed based on WebKit 537.36, there will be this Version information.
WebSocketDebuggerUrl, which is the WebSocket debug URL.

4.2.2 Obtaining the List of Debuggable Pages

$curl = $curl $curl = $curl//localhost:9222/json/list
curl http://localhost:9222/json/list  [{"description": "{\"attached\":true,\"empty\":false,\"height\":1812,\"never_attached\":false,\"screenX\":0,\"screenY\":72,\"visible\":tr ue,\"width\":1080}"."devtoolsFrontendUrl": "https://chrome-devtools-frontend.appspot.com/serve_rev/@9f05d1d9ee7483a73e9fe91ddcb8274ebcec9d7f/inspector.html?ws=loca lhost:9222/devtools/page/B86E67DEA526D5EEE83A170B1F62A72C"."faviconUrl": "https://mat1.gtimg.com/www/mobi/2017/image/logo/v0/192.png"."id": "B86E67DEA526D5EEE83A170B1F62A72C"."title": "QQ.COM"."type": "page"."url": "https://xw.qq.com/#news"."webSocketDebuggerUrl": "ws://localhost:9222/devtools/page/B86E67DEA526D5EEE83A170B1F62A72C"
}, {
   "description": "{\"attached\":false,\"empty\":true,\"never_attached\":true,\"screenX\":0,\"screenY\":0,\"visible\":true}"."devtoolsFrontendUrl": "https://chrome-devtools-frontend.appspot.com/serve_rev/@9f05d1d9ee7483a73e9fe91ddcb8274ebcec9d7f/inspector.html?ws=loca lhost:9222/devtools/page/3F9E05905F1919D563DF01BAEC64D2E4"."id": "3F9E05905F1919D563DF01BAEC64D2E4"."title": "about:blank"."type": "page"."url": "about:blank"."webSocketDebuggerUrl": "ws://localhost:9222/devtools/page/3F9E05905F1919D563DF01BAEC64D2E4"}]Copy the code

Each debug page takes up one data element. As you can see from the above value, vivo browser in my environment opened two pages, one xw.qq.com/#news and one about:blank.

Description, which is a JSON object, displays the status of the current page. For example, page width, height, offset on the screen, whether the WebView has been attached to the view, only the attached page will be displayed, whether it can be debugged.
DevtoolsFrontendUrl, this value is a URL, is the daily use of DevTools WebUI control panel address, the Web APP will cache a copy of the browser after a visit. This page is hosted in an address that cannot be accessed normally in China, so it often appears that the panel cannot be opened and the blank screen is displayed. Chrome is packaged with a built-in web user interface (WebUI) version that matches the current kernel, so Chrome can debug its own pages directly.
Id, which is a randomly generated GUID value for each open page used to generate WebSocket links to differentiate between pages.
Title, open the title of the page, corresponding to the content of the title tag in the page head.
Type indicates the type of a page, including page, iframe, worker, and service_worker.
URL, the URL of the currently open page.
WebSocketDebuggerUrl. This parameter is the URL of the WebSocket connection.

HTTP has several other subcommands, such as protocol, new, activate, etc., mainly the page control class, I will not introduce them in detail.

4.2.3 WebSocket Protocol Analysis

The WebSocket protocol consists of four parts: Domain, Method, Event, and Type.

1) Domain, namespace, similar to C++/Java namespace or package name, used to separate different commands. Subcommands are used to classify subcommands into classes for easy invocation and to prevent Method name conflicts. Using CDP 1.3, 15 domains are divided.

Browser: Used to manage Browser objects.
Debugger: Classes used to debug JavaScript, such as breakpoints, call stacks, etc.
DOM: All DOM node operations are performed in this Domain, including DOM node modification and traversal.
DOMDebugger: The Domain that manages DOM node debugging. DevTools nodes modify breakpoints through Method provided in this group of domains.
Emulation: This is a set of environment emulators. DevTools modifiers such as device size and UserAgent are implemented by this Domain.
Input: Collection of event distribution methods.
IO: set of I/O flow operations.
Log: set of Log control methods.
Network: web browser Network traffic data, which may be captured through this Domain.
Page: Based on the Page Method collection in Blink, such as refresh, open URL.
Performance: Integrates the Performance analysis Method.
Profiler: The Method of the sampling analyzer is integrated under this Domain.
Runtime: Methods that communicate with JavaScript are integrated into this Domain, such as executing JavaScript code.
Security: Security operations, such as certificate errors.
Target: Some control classes for DevTools connections are under this Domain.

2) Method, the name of the Method, each Domain will have a set of methods, specify the specific browser kernel functions. It consists of three parts: name, parameter, and return value. This is consistent with the method description in C++/Java.

Name: Debugger. SetBreakpointByUrl;
Parameters: lineNumber INTEGER [, URL String, urlRegex String, scriptHash String, columnNumber INTEGER, condition String];
Return value: breakpointId breakpointId, actualLocation Location.

. / / the Debugger setBreakpointByUrl into the kernel, with the following parameters
{
   "lineNumber":1."url":"snippet:///Script%20snippet%20%231"."columnNumber":0."condition":""
}
 
// The kernel returns a breakpoint success message
{
   "breakpointId":"1:1:0:snippet:///Script%20snippet%20%231"."locations":[]
}
Copy the code

The web page will have many status notifications, which need to be synchronized to the WebUI or other control terminal. Events are used to notify these events. For example, when a DOM attribute changes, the DOM. AttributeModified event is received; When you pass JavaScript to the kernel for execution, you will receive debugger.scriptparsed events and parameters as follows:

{
   "scriptId":"238"."url":""."startLine":0."startColumn":0."endLine":0."embedderName":""."endColumn":7."endLine":0."executionContextAuxData": {"isDefault":true."type":"default"."frameId":"2059AA1A2C1A535CF4C480DC01E7FDEC"
   },
   "frameId":"2059AA1A2C1A535CF4C480DC01E7FDEC"."isDefault":true."type":"default"."executionContextId":5."hasSourceURL":false."hash":"035a9e1738252e22523ed8f1c52d9dbf81abe278"."isLiveEdit":false."isModule":false."length":7."scriptId":"238"."scriptLanguage":"JavaScript"."sourceMapURL":""."startColumn":0."startLine":0."url":""
}
Copy the code

4) Type, which is the complex data Type of Method or Event passing parameters, and these types correspond to objects in the kernel. For example, the dom. Node type corresponds to a DOM Node in Blink. The main attributes are as follows:

NodeId: nodeId is also Type, which is the nodeId. Based on this value, the corresponding node can be found in the kernel.
ParentId: NodeId Is also Type, indicating the ID of the parent node.
NodeType: integer indicates the nodeType.
NodeName: string, nodeName.
NodeValue: string, node content.
Children: array, an array of child nodes.
Nodes use these attributes to describe the memory footprint of nodes in the DOM tree. The Element panel in DevTools’ Web UI displays a DOM tree using the dom.getDocument Method.

Through the data organization mode of CDP, the kernel can be operated by passing control commands and receiving kernel status notification (Event). CDP allows the browser to do just about anything and get more information than using Chrome’s graphical interface. As a result, Google has released Chrome Headless, which is widely used for automated web testing, crawlers, and web sandboxes.

When debugging a mobile browser, you can see the screen being viewed on the mobile device in real time. How does this work?

ScreencastFrame event to send base64 images back to the Web UI.

Meta data from the Page. ScreencastFrame notification event:

{
   "data":"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAYEBQYFBAYGBQYHBw..."."metadata": {"deviceHeight":604."deviceWidth":360."offsetTop":60."pageScaleFactor":1."scrollOffsetX":0."scrollOffsetY":832.6666870117188."timestamp":1631018056.565802
   },
   "sessionId":2
}
Copy the code

You can display the image description on the WebUI. A screenshot is nearly 1M in size, and because DevTools takes advantage of WebSocket’s two-way long link feature, it is extremely smooth and clear.

4.3 DevTools kernel implementation

The above chapter introduces the overall process of transferring operation commands to Browser Core on mobile terminals from the perspective of Web developers, as well as related content of CDP communication protocol. This section focuses on the implementation process in Browser Core. First, it introduces the implementation of DevTools in the Browser kernel. Later, THE author will select how JavaScript is passed from string to V8 to execute the process.

4.3.1 Kernel Architecture Introduction

DevTools starts with a Web Server and sends the call command to the corresponding processing module. The overall architecture is shown as follows:

DevTools has roughly four layers in the kernel:

The Server layer receives operation requests from the external network.
The Agent layer disassembs requests from the Server layer and distributes them to different Agents for processing according to operation types.
Session layer. Session is a layer of abstraction for different business modules. After passing the Session layer, you will enter different business modules, including V8, Blink, etc.
The business layer is the specific functional module, such as the V8 module, which is responsible for supporting JavaScript debugging related capabilities.

The Server layer is managed by the singleton DevToolsManager. Since it is a singleton, a process can only have one Manger object, so as to prevent the repeated creation of more than one Manger object, resulting in state disorder.

4.3.2 Web Server Data Receiving Entry

All requests received by the Server are sent to the DevToolsHttpHandler class, which is responsible for responding to data requests from the network Client and sending the results back to the network Client. There are two important methods in this class, OnJsonRequest and OnWebSocketMessage, which handle the HTTP protocol and WebSocket protocol respectively.

void DevToolsHttpHandler::OnJsonRequest(
    int connection_id,
    const net::HttpServerRequestInfo& info) {
  // Query the kernel version
  if (command == "version") {
    base::DictionaryValue version;
    version.SetString("Protocol-Version",
                      DevToolsAgentHost::GetProtocolVersion());
    // ...
    SendJson(connection_id, net::HTTP_OK, &version, std::string());
    return;
  }
  // Get the protocol supported by the kernel
  if (command == "protocol") {
    DecompressAndSendJsonProtocol(connection_id);
    return;
  }
  // Get the debugable page
  if (command == "list") {
    DevToolsManager* manager = DevToolsManager::GetInstance();
    DevToolsAgentHost::List list =
        manager->delegate() ? manager->delegate()->RemoteDebuggingTargets()
                            : DevToolsAgentHost::GetOrCreateAll();
    RespondToJsonList(connection_id, info.GetHeaderValue("host"),
                      std::move(list));
    return;
  }
  // Start a new debug
  if (command == "new") {
    // ...
    std::string host = info.GetHeaderValue("host");
    std::unique_ptr<base::DictionaryValue> dictionary(
        SerializeDescriptor(agent_host, host));
    SendJson(connection_id, net::HTTP_OK, dictionary.get(), std::string());
    return;
  }
  // Enable or disable a debug
  if (command == "activate" || command == "close") {
   // ...
  SendJson(connection_id, net::HTTP_NOT_FOUND, nullptr,
           "Unknown command: " + command);
}
 
void DevToolsHttpHandler::OnWebSocketRequest(
    int connection_id,
    const net::HttpServerRequestInfo& request) {
  // Create Agent for debugging
  if (base::StartsWith(request.path, browser_guid_,
                       base::CompareCase::SENSITIVE)) {
    scoped_refptr<DevToolsAgentHost> browser_agent =
        DevToolsAgentHost::CreateForBrowser(
            thread_->task_runner(),
            base::BindRepeating(&DevToolsSocketFactory::CreateForTethering,
                                base::Unretained(socket_factory_.get())));
    connection_to_client_[connection_id] =
        std::make_unique<DevToolsAgentHostClientImpl>(
            thread_->task_runner(), server_wrapper_.get(), connection_id,
            browser_agent);
    AcceptWebSocket(connection_id, request);
    return;
  }
 
  connection_to_client_[connection_id] =
      std::make_unique<DevToolsAgentHostClientImpl>(
          thread_->task_runner(), server_wrapper_.get(), connection_id, agent);
    // Accept websocket
  AcceptWebSocket(connection_id, request);
}
 
// WebSocket Data receiving interface through which all WebUI requests are distributed
void DevToolsHttpHandler::OnWebSocketMessage(int connection_id,
                                             std::string data) {
  auto it = connection_to_client_.find(connection_id);
  if (it != connection_to_client_.end()) {
    it->second->OnMessage(base::as_bytes(base::make_span(data)));
  }
}
Copy the code

DevToolsHttpHandler: : OnJsonRequest used in response to a HTTP request to track the status of the kernel, such as the kernel version, the current support agreement, will return the entire agreement content, convenient for developers to fit the corresponding support.
DevToolsHttpHandler: : OnWebSocketRequest to accept a WebSocket connection, according to this method to different Agent object is created.
DevToolsHttpHandler: : OnWebSocketMessage all debug request data, through the interface by Client distributed to different Agent.

Server layer data response through the above three interfaces to achieve data receiving and distribution capabilities.

4.3.3 JavaScript execution process

The V8 JavaScript engine is used to interpret and execute JavaScript scripts in web pages. It can also receive external scripts through DevTools. The scripts are executed in the Context of the current web page, so you can manipulate web page behavior through JavaScript. Such as modifying DOM node properties. In CDP, the Runtime. Evaluate interface is designed to implement JavaScript.

{
    allowUnsafeEvalBlockedByCSP: false,
    awaitPromise: false,
    contextId: 14,
    expression: "alert('hi');",
    generatePreview: true,
    includeCommandLineAPI: true,
    objectGroup: "console",
    replMode: true,
    returnByValue: false,
    silent: false
}
Copy the code

The most important parameter is expression, which is a string parameter used to store the content of the script to be executed. The above example will bring up an Alert box with the content hi on the web page.

The V8RuntimeAgentImpl module is used to support the CDP Runtime Domain and the V8DebuggerAgentImpl module is used to support the IMPLEMENTATION of the CDP Runtime Domain. The evaluate method in V8RuntimeAgentImpl is used to receive execution requests from DevTools.

void V8RuntimeAgentImpl::evaluate(
    const String16& expression, Maybe<String16> objectGroup,
    Maybe<bool> includeCommandLineAPI, Maybe<bool> silent,
    Maybe<int> executionContextId, Maybe<bool> returnByValue,
    Maybe<bool> generatePreview, Maybe<bool> userGesture,
    Maybe<bool> maybeAwaitPromise, Maybe<bool> throwOnSideEffect,
    Maybe<double> timeout, Maybe<bool> disableBreaks, Maybe<bool> maybeReplMode,
    Maybe<bool> allowUnsafeEvalBlockedByCSP,
    std::unique_ptr<EvaluateCallback> callback);
Copy the code

V8RuntimeAgentImpl: : evaluate will start a microtasks to execute the script, will eventually go the v8: : internal: : Execution: : Call, The Execution module parses and compiles the script into bytecode, and eventually schedules it to run on a virtual machine.

The execution process is shown in the figure above. The Web UI issues a string to execute the script, which is received by WebSocket’s OnWebSocketMessage and then distributed layer by layer to V8 via DevToolsSession. As Chrome is a multi-process architecture, divided into Browser process and Render process, communication between the IPC. In the figure above, the process is executed on the Browser side on the left and on the Render side on the right.

Render the DevToolsSession: : DispatchProtocolCommand is an important distribution interface, so to V8 or Blink control commands, passed through this interface. The control command is then sent to the V8RuntimeAgentImpl and is scheduled to be processed by different functional modules according to the command function.

4.4 Web Page Performance Tuning

4.4.1 Performance Analysis Panel

DevTools provides a powerful set of performance analysis tools for networking, JavaScript debugging, rendering, memory, and standards support. The following describes some Performance analysis functions in the Performance panel. The main interface is divided into these sections:

1) Frame rate (FPS) : linearly shows the frame rate of web rendering during Performance.

2) CPU Usage: CPU usage chart

3) Screen capture during loading: webpage screen capture performance is regularly collected

4) Network loading sequence: show the loading sequence and time consuming of network resources

5) Frames: displays the rendering time of each frame, and red indicates that there are Frames that take a long time.

6) Web Vitals Metrics: Google recommends a set of performance experience metrics, which are detailed below.

7) Main threads in the kernel: there are multiple threads in the browser kernel with different tasks. When a time-consuming frame occurs, it is necessary to check which thread is time-consuming in these threads. It is mainly divided into the following:

Main, which is the Blink Main thread, is responsible for layout, parsing, JavaScript execution, etc.
Raster, Raster thread, used to convert rendered objects into bitmaps.
GPU, hardware-accelerated rendering thread, draws Texture to the screen.
Chrome_ChildIOThread, responsible for network resources, file operations.
The Compositor thread is responsible for compositing the layers of rendering together and rasterizing them.
ThreadPoolForegoundWorker, the work of a Worker thread pool.

8) Information panel: used to display the detailed information of the selection module. Meanings of several indicators:

Loading: Time required for network requests and HTML parsing.
Scripting: JavaSript parsing, compilation, execution in the VIRTUAL machine, and GC time.
Blink Rendering time.
Painting: Painting time, including Painting, composition, image decoding, and screen loading.
System and Idle: indicates the System scheduling and Idle time.

4.4.2 General Ideas for Performance Analysis

The basic idea of performance analysis starts from the problem, the common webpage performance problem, the author encountered mainly have these several situations.

Required resources are not requested back in time. Troubleshoot server problems, resource request launched too late? Too many resources?
There are too many layers on the page, leading to Rendering and Painting taking too long.
Too much memory, too complex pages, too many and large resources, JavaScript block resource holding life cycle is too long.
Animation is too many and not removed after disappearing. JavaScript rotation animation, CSS animation, with animation image resources, such as GIF, SVG, WebP and so on.
Event listening is improper. Events are monitored too much and may be frequently touched, such as node changes, Move events, etc.

In general, whether webpage performance optimization or Native program optimization, as long as the balance of these two resources: CPU + memory. Once the problem spot is discovered, the performance problem will be solved, and DevTools can help with that in addition to source-level review.

For the general scenario summarized above, take a look at the Profile as a whole using DevTools’ performance analysis capabilities.

Whether the order and duration of network requests are reasonable;

Whether the long task of the Main Thread is reasonable.

Observe the sequence of resource requests from the Network board, and check whether there are time-consuming tasks that block the loading of resources on the first screen. If the required loading is not ensured, the screen will be blank for a long time.

After resource problems are solved, identify time-consuming tasks. First look at Long tasks in the Main Thread. For example, Long Tasks in the figure above are for Scripting, which takes a Long time. View the specific time points through bottom-up/CallTree and optimize them accordingly.

Here’s a tip for identifying specific optimization points. Often development environments are emulated on PCS, and problems emerge when the release is out. Due to the fragmentation of mobile devices, the performance of many users’ devices may not be good. So how do you optimize performance on a low-profile machine like this in a development environment? DevTools provides current-limiting emulation, limiting the network to 2G/3G and reducing CPU speed.

In the upper right corner there is a “Settings”, expand the configuration project, you can see the Network and CPU traffic limiting options, select and record the Profile again.

As mentioned above, there are too many layers of web pages, which greatly affects the rendering performance of web pages. What do I mean by “page layers”? At present, the browser rendering engine layers the web page to improve the rendering performance. The advantage of this is that only the modified layer is redrawn. If there is no change in the content of other layers, the last drawing result is directly taken, thus improving the drawing efficiency. Different WEB engines have different layering strategies, usually dividing ordinary WEB pages, CSS animations, Canvas, WebGL, Fix tags, and so on into one layer. Layering improves rendering efficiency, but also introduces memory overhead, which can affect performance. Can DevTools analyze page layers? Yes, in the Settings above there is an option “Enable Advanced Paint Instrumentation (slow)” to Enable it and re-do one-time recording.

There is a new “Layers” TAB in the Information panel. Select this TAB and you will see how the page is layered. If unreasonable layers exist, adjust the modes to combine them to improve performance.

4.4.3 Web Vitals

Web Vitals is a set of Web performance and experience metrics released by Google. The original measurement strategy is basically based on the “first word” and “first screen”, but from the perspective of users and technical optimization, there are some problems in these two indicators. So Google created the Web Vitals standard, which works with DevTools to help developers identify performance issues on the Web at the development stage. Since standards are constantly changing with the development of The Times, developers have been chasing the changes of the metrics is a little bit tired, fortunately, Google has made it clear that the current three metrics will not change in the short term, I don’t know how long this short period of time.

Largest Contentful Paint (LCP), 2.5 seconds or less. Mainly refers to the large area of text, pictures are displayed, even if the LCP has been reached.

The second indicator: First Input Delay (FID), the time point at which external Input events can be responded for the First time, which is excellent within 100 ms. This metric is from a user perspective, reaching the FID point in time, meaning the user can operate the web page.

The third indicator: Cumulative Layout Shift (CLS), typesetting jump index, 0.1 is excellent. This can be high if there is a large area of movement during the loading of a web page, where the elements appear in a well-lined layout. For example, the img tag in the web page does not set the width and height. After the image is loaded, the version is arranged according to the actual size of the image. This will trigger a relayout of the page, pushing the entire page down an image height from the user’s point of view, which Google considers a bad experience.

The LCP/FID/CLS metrics are essentially performance metrics from a user’s perspective, and developers can see where their games fall in these metrics.

V. The importance of tools in ecological construction

(Data from Statcounter.com)

Chrome has won over a large number of users with its excellent product features, security, speed and stability. As you can see from StatCounter’s statistics above, Chrome has become the absolute number one browser in the world, and it’s a natural commercial success. But DevTools is the first step in Chrome’s development of open source and ecosystem. Google has attracted a large number of front-end developers to switch to Chrome to develop their own products through DevTools’ superior features over competitors. The early ecosystem product was the Chrome plugin, and its success can be seen in the number of plug-ins in the Chrome Store.

When Node.js was introduced, DevTools was the first debugging tool to support Node.js, which helped popularize node.js. DevTools then quickly loops through Node.js. On the other hand, the open source world has also started to support DevTools project. Currently, there are more than 10 open source solutions supporting CDP protocol, and most commonly used languages are supported. This field is still in rapid development, and I look forward to better development in this field.

DevTools Web UI has been independent from the Chromium warehouse and can be cloned for secondary development. The Web UI is limited to space this time and the implementation principle analysis is not done. In fact, the Web UI is also a very good Web APP for front-end developers to explore in depth.

What we learn from good open source projects is not only code implementation and architecture, but also higher dimensions, such as product thinking and tool thinking, which can be applied to our own projects. Take a look at how web debugging evolved from a JavaScript plug-in to today’s front-end development ecosystem, and there are many lessons to be learned.

Sixth, concluding remarks

The author’s team has been committed to the research and learning of The Chromium kernel for a long time. Based on the products derived from it, we serve our ecological users and provide them with high-quality Internet experience. At the same time, the Web browsing service incubated by us also provides powerful, fast and stable Web service capabilities for applications in the ecosystem. If you are interested in Web underlying technology research, welcome to join us, grow together with a group of like-minded partners, but also can serve hundreds of millions of users.

7. References

[1] Google Chrome

[2] 10 Years of Web Inspector

[3] 10 years of Speed in Chrome

[4] Chrome DevTools

[5] Chrome DevTools Protocol protocol

[6] Web Vitals

Author: Li Qingmei, Kernel team of Vivo Internet Browser

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

DevTools implementation principle and performance analysis of actual combat

One, the introduction

History of web debugging tools

DevTools architecture

DevTools communication protocol

4.1 Environment Preparations

4.2 HTTP Protocol Analysis

4.2.1 Obtaining kernel Version Information

4.2.2 Obtaining the List of Debuggable Pages

4.2.3 WebSocket Protocol Analysis

4.3 DevTools kernel implementation

4.3.1 Kernel Architecture Introduction

4.3.2 Web Server Data Receiving Entry

4.3.3 JavaScript execution process

4.4 Web Page Performance Tuning

4.4.1 Performance Analysis Panel

4.4.2 General Ideas for Performance Analysis

4.4.3 Web Vitals

V. The importance of tools in ecological construction

Sixth, concluding remarks

7. References

DevTools implementation principle and performance analysis of actual combat

One, the introduction

History of web debugging tools

DevTools architecture

DevTools communication protocol

4.1 Environment Preparations

4.2 HTTP Protocol Analysis

4.2.1 Obtaining kernel Version Information

4.2.2 Obtaining the List of Debuggable Pages

4.2.3 WebSocket Protocol Analysis

4.3 DevTools kernel implementation

4.3.1 Kernel Architecture Introduction

4.3.2 Web Server Data Receiving Entry

4.3.3 JavaScript execution process

4.4 Web Page Performance Tuning

4.4.1 Performance Analysis Panel

4.4.2 General Ideas for Performance Analysis

4.4.3 Web Vitals

V. The importance of tools in ecological construction

Sixth, concluding remarks

7. References

Related Posts

React-hooks Fix state management using React-hooks

Come to package a one-stop service for dynamic sharing of applets (¨̮)

React Hook lets you load lists