Lighthouse Testing Insider

This article is contributed by KKDEV163, front-end engineer of netease cloud music, whose personal blog has also recorded some practical articles in the field of front-end monitoring.

preface

The cloud music front-end performance monitoring platform uses Lighthouse for auditing and scoring at the bottom. In the process of practice, we have accumulated some research experience on the internal implementation of Lighthouse, which we hope to share with readers through this article.

Based on Lighthouse 5.2.0, this article introduces the testing process, implementation of architecture modules, and calculation of performance indicators of Lighthouse. In this article, you’ll learn how Lighthouse runs automated tests, customizes audit entries on the Lighthouse framework, and simulates key performance metrics.

This article will be developed in the following four parts:

Lighthouse profile
Lighthouse Testing Process
Implementation of Lighthouse Module
Lighthouse Performance indicators are calculated

Lighthouse profile

Lighthouse is an open source automation tool designed to improve the quality of web applications. Once you give Lighthouse a web site to review, it will run a series of tests against that page, and then generate a report on the page’s performance.

Usage of Lighthouse

Currently, there are four official ways to use it:

Chrome Developer Tools
Chrome extension
Node CLI
Node Module

Under the Chrome Developer tools, for example, users can configure test platforms, test classes, speed limits, and so on to easily and quickly initiate a test.

Lighthouse Test Report

After the test is over, an HTML report is generated by default, as shown in the figure below, in which test scores of the 5 categories are covered:

Each category contains a set of audits, and Lighthouse provides specific optimization recommendations and diagnostics for the results of each audit.

This section provides a brief introduction to the usage of Lighthouse and the composition of the test report. The next section describes the test process of Lighthouse.

Lighthouse Testing Process

We ran tests using the Node CLI to analyze the Lighthouse testing process.

After the CLI is installed, run the following command to perform a test

lighthouse --only-categories=performance https://google.com
Copy the code

Note: The above command only tests the performance category.

The CLI output logs during the test process, as shown in the following screenshot. In the logs, you can see that the test is roughly divided into the following stages:

With the output logs, you can draw the Lighthouse test flow chart as follows:

Lighthouse connects to the browser.
Initial configuration of tests and loading of pages to be tested.
During page loading, a series of Gatherers are run, and each collector collects its own target information and generates artifacts.
Run a series of audits, each of which takes the data it needs from the artifacts and computes its own score.
Based on the scores of audit items, the scores of categories are calculated and summarized to generate reports.

This section describes the Lighthouse test process based on the Lighthouse test log. The next section describes the module implementation of the process.

Implementation of Lighthouse Module

With a primer on the basic testing process, let’s take a look at the official Lighthouse architecture diagram:

This diagram shows the main flow of the test, from which four main modules can be circled, which will be explained one by one in the following sections.

Driver module

Two-way communication and DevTools protocol

When Chrome is started, you can run the –remote-debugging-port parameter to set the remote debugging port. Run the following command to open Chrome and set the remote debugging port to 9222.

chrome.exe --remote-debugging-port=9222
Copy the code

You can then use the address http://localhost:9222 for remote debugging, such as the following command to open a new Tab in Chrome.

curl http://localhost:9222/json/new
Copy the code

The command also returns information about the Tab, notably the webSocketDebuggerUrl, which is the WebSocket connection address for the Tab.

{
    "id": "29989D..."."url": "about:blank"."webSocketDebuggerUrl": "ws://localhost:9222/devtools/page/29989D...". }Copy the code

The Driver module holds the Connection instance (responsible for communicating with the browser), which, when initialized, opens a new Tab by calling the /json/new directive of the remote debug port, And use the returned webSocketDebuggerUrl to establish a WebSocket connection with the browser, after which two-way communication can be carried out.

I’m going to open a new Tab

Establish a WebSocket connection

After the WebSocket connection is established, the two parties must use a data format Protocol to communicate. This Protocol is Chrome DevTools Protocol, which defines method names and parameters of instructions in JSON format.

Send the Pag. navigate directive to enable Chrome to navigate to the target Page, as shown below. Sending the page. captureScreenshot directive causes Chrome to generate a screenshot of the current Page.

In the document of the protocol, all control commands and events are divided into several Domains, such as Page, Network, etc. Open the Page field to find details for the sample directive pag.navigate:

In addition to navigate, captureScreenshot and other active directives, when we invoke the enable directive of a domain, we receive subsequent notification events from that domain.

Lighthouse controls the Chrome browser and senses events as the page loads using active commands and event notifications as defined by the Chrome DevTools Protocol.

logging

Two other important examples of the Driver module are DevtoolsLog and NetworkRecorder, which are used to store notification events sent by the browser in a structured manner. DevtoolsLog records all logs in all fields, NetworkRecorder only stores network logs, and analyzes the current network request status (busy, idle).

The stored log information is used in subsequent Audits modules, which we continue to cover below.

Emulation

The last part of the Driver module that was worth mentioning was the emulation (emulation), which simulated the test device, such as mobile/PC terminal, screen size, UserAgent, Cookie, network speed limit, etc.

These simulation functions are also set through the Connection module to send the corresponding domain control instructions to Chrome browser.

Now that we have analyzed the main components of the Driver module, let’s briefly summarize: responsible for two-way communication with the browser, logging events, setting up emulators, etc.

Gatherer module

An important concept of this module is pass, which is officially defined as follows:

controls how to load the requested URL and what information to gather about the page while loading.

That is, controlling how the page loads and deciding what information is collected during page loading

defines basic settings such as how long to wait for the page to load and whether to record a trace file. Additionally a list of gatherers to use is defined per pass. Gatherers can read information from the page to generate artifacts which are later used by audits to provide you with a Lighthouse report.

That is, to define the wait time for page loading, whether to record trace files, and so on. Each pass also defines a list of Gatherers, which can read the required information from the page and generate an intermediate, which will be used for subsequent audit analysis and ultimately generate test reports.

Now that you know the definition of pass, let’s look at a specific pass configuration:

{
  passes: [{
    passName: 'defaultPass'.recordTrace: true.// Whether to record Trace information
    useThrottling: true.// Whether to use speed limit
    gatherers: [ / / gatherers list
      'css-usage'.'viewport-dimensions'.'runtime-exceptions'.'console-messages'.'anchor-elements'.'image-elements'.'link-elements'.'meta-elements'.'script-elements'.'iframe-elements'./ / to omit],},.../ / to omit
}
Copy the code

Each Gatherer has a corresponding implementation file of the same name in the code repository, and all of them inherit from the same parent class Gatherer, which defines three template methods. The subclasses only need to implement the template methods they care about.

class Gatherer {
    // before page navigation
    beforePass(passContext) { }
    
    // After the page loaded
    pass(passContext){ }

    // After the page has loaded and the trace information has been collected
    afterPass(passContext, loadData) { }
}

Copy the code

Take a simpler Gatherer concrete implementation RuntimeExceptions that implements both beforePass and afterPass lifecycle template methods, Driver. on is the event listening implemented by the driver module described above.

const Gatherer = require('./gatherer.js');

class RuntimeExceptions extends Gatherer {
  constructor() {
    super(a);this._exceptions = [];
    this._onRuntimeExceptionThrown = this.onRuntimeExceptionThrown.bind(this);
  }

  onRuntimeExceptionThrown(entry) {
    this._exceptions.push(entry);
  }
 
  // Before page navigation, register event listener to collect error information
  beforePass(passContext) {
    const driver = passContext.driver;
    driver.on('Runtime.exceptionThrown'.this._onRuntimeExceptionThrown);
  }

  // Remove event listening after the page is loaded
  async afterPass(passContext) {
    await passContext.driver.off('Runtime.exceptionThrown'.this._onRuntimeExceptionThrown);
    return this._exceptions; }}Copy the code

With this reference example, we could also easily write a custom Gatherer, such as Gatherer for the page title:

const Gatherer = require('./gatherer.js');

function getPageTitle() {
    return document.title;
}

class PageTitle extends Gatherer {

    afterPass(passContext) {
        return passContext.driver.evaluateAsync(` (${getPageTitle.toString()}` ())); }}Copy the code

We just overwrite the afterPass method, which in the middle of its life sends the script to the browser for execution through the driver module and gets the results.

After all the Gatherers defined in Pass have run, an intermediate Artifact artifact is generated, and Lighthouse can then disconnect from the browser and use only artifacts for subsequent analysis.

To summarize, the Gatherer module defines how the page loads by passing the configuration, runs all gatherers configured to gather information during the page load, and generates intermediate artifacts. With artifacts, you can access the next Audits modules.

Audits module

Like Gatherers, audits that need to be run are defined in configuration files, and each audits has its own implementation file with the same name.

{
  audits: [
    'errors-in-console'.'metrics/first-contentful-paint'.'metrics/first-meaningful-paint'.'metrics/speed-index'.'metrics/first-cpu-idle'.'metrics/interactive'.'screenshot-thumbnails'.'final-screenshot'./ / to omit]./ / to omit
}
Copy the code

Let’s start with the simplest errors-in-console to see how the next audit is implemented.

In each audit, a static method meta() is defined that describes the audit and declares the required artifacts. ErrorLogs this audit declares the intermediates that need to be generated by RuntimeExceptions mentioned above.

class ErrorLogs extends Audit {
  static get meta() {
    return {
      id: 'errors-in-console'.title: str_(UIStrings.title),
      failureTitle: str_(UIStrings.failureTitle),
      description: str_(UIStrings.description),
      requiredArtifacts: ['ConsoleMessages'.'RuntimeExceptions']}; }}Copy the code

Another template method that the Audit instance needs to implement is Audit (), which can obtain the required intermediates and calculate the score and details of this Audit based on the intermediates.

static audit(artifacts) {
    // Get the desired intermediate
    const runtimeExceptions = artifacts.RuntimeExceptions;
    
    // Data filtering and conversion
    const runtimeExRows =
      runtimeExceptions.filter(entry= >entry.exceptionDetails ! = =undefined)
      .map(entry= > {
        const description = entry.exceptionDetails.exception ?
          entry.exceptionDetails.exception.description : entry.exceptionDetails.text;

        return {
          source: 'Runtime.exception',
          description,
          url: entry.exceptionDetails.url,
        };
      });

    // omit the table detail generation code.// Calculate the score of the audit item
    const numErrors = tableRows.length;
    return {
      score: Number(numErrors === 0),
      numericValue: numErrors,
      details,
    };
  }
Copy the code

With the example above, we can implement a custom audit item, such as the audit page title:

class PageTitle extends Audit {
    static get meta() {
        return {
            id: 'page-title'.title: 'title of page document'.failureTitle: 'Does not have page title'.description: 'This audit get document.title when page loaded'.requiredArtifacts: ['PageTitle']}; }static audit(artifacts) {
        return {
            score: artifacts.PageTitle ? 1 : 0.displayValue: artifacts.PageTitle || 'none'}; }}Copy the code

After running all audit items defined in the configuration file, the score and details of each audit item are obtained, and then the Report module is entered.

The Report module

In the configuration file, you define the audit items required for each test category and the weight assigned to each audit item. The following audit items are required for the performance test category:

{
  'performance': {
    title: str_(UIStrings.performanceCategoryTitle),
    auditRefs: [{id: 'first-contentful-paint'.weight: 3.group: 'metrics'},
    {id: 'first-meaningful-paint'.weight: 1.group: 'metrics'},
    {id: 'speed-index'.weight: 4.group: 'metrics'},
    {id: 'interactive'.weight: 5.group: 'metrics'},
    {id: 'first-cpu-idle'.weight: 2.group: 'metrics'},
    {id: 'max-potential-fid'.weight: 0.group: 'metrics'},
    / / to omit]}}Copy the code

In the final summary phase, Lighthouse calculates a performance score based on this configuration file and the scores for each audit item calculated in the previous session. Based on the score and type of each audit item, audit items are divided into pass and fail. For those audit items that fail, detailed test details and optimization guidelines are given.

Implement performance indicator audit items such as FCP

In introducing the overall testing process above, I chose the simplest audit items to introduce. In this section, I will introduce performance audit indicators that are of more concern to us, such as FCP.

First Contentful Paint (FCP) is the time when the First content is rendered from the DOM by the browser, starting with the page navigation.

The speed limit to simulate

As page performance is strongly influenced by host network and CPU frequency, Lighthouse provides three ways to simulate a poor host environment. The logic behind Lighthouse is that if a page achieves a good test score in a poor environment, most users will have a better experience of the page.

In Chrome Devtools’s Audits panel, you can see three different ways of limiting the speed:

The configuration items in the preceding figure correspond to the three rate limiting modes

simulated

Throttling is simulated, resulting in faster audit runs with similar measurement accuracy

That is, the speed limit is simulated (the speed limit is not performed when the page is loaded. After the page is loaded, the simulation calculates the performance index value under the speed limit condition), so the audit can be completed at a faster speed with similar test accuracy.

devtools

Typical DevTools throttling, with actual traffic shaping and CPU slowdown applied

With DevTools for speed limiting, pages are loaded on a real restricted network with a slow CPU.

no throttling

No network or CPU throttling used. (Useful when not evaluating performance)

Lighthouse doesn’t set any additional speed limits, and it’s usually used when no performance testing is in progress, or when the developers themselves set speed limits on the host.

Of the three speed limiting methods, Lighthouse truly limits the speed of the network and CPU only in the form of DevTools, which is implemented by sending instructions for the corresponding domain to The Chrome browser through the Driver module mentioned above:

// Enable CPU rate limiting
function enableCPUThrottling(driver, throttlingSettings) {
  const rate = throttlingSettings.cpuSlowdownMultiplier;
  return driver.sendCommand('Emulation.setCPUThrottlingRate', {rate});
}

// Enable network rate limiting
function enableNetworkThrottling(driver, throttlingSettings) {
  // Omit some code
  return driver.sendCommand('Network.emulateNetworkConditions', conditions);
}
Copy the code

The Trace information

When we introduced pass, we mentioned that there is a parameter that controls whether Trace information is collected. What is Trace information? What good is it?

In fact, most of us have already been exposed to Trace information, which can be visualized in the Performance panel of Chrome DevTools:

In this visual panel, you can see the key rendering nodes FP, FCP and FMP in the page loading process, as well as the dependencies of mainline Parse HTML, Layout and JS execution.

As soon as the Pass is configured to enable the collection of Trace information, Lighthouse, after the page is loaded, will provide you with the complete Trace information, from which you can know the key rendering nodes such as FCP and FMP at the time of page loading.

Simulation of FCP

When devTools and No throttling are used for speed limiting, Lighthouse does not require any additional processing because the page is loaded under real restricted network conditions and the FCP value given in the Trace message is the FCP value under speed limiting conditions.

However, under the speed limiting mode of simulated speed limiting, the page is loaded under the condition of no speed limiting, so the FCP in the Trace is the FCP under the condition of no speed limiting. Lighthouse needs to calculate the estimated VALUE of FCP under the given speed limiting condition by means of simulation calculation. Next we focus on the computation of FCP under simulated mode.

As mentioned above, the Driver module has A NetworkRecorder module that records the details of all Network requests during the page loading process. Lighthouse sets up a Network Node for each valid Network request event.

CPU execution events during page loading are also recorded in the Trace message. Lighthouse creates a CPU Node for each valid CPU event.

Next, Lighthouse locates the root node (the request Document node) from the Network request nodes, establishes the dependencies between CPU nodes and Network nodes according to the node dependency algorithm, and finally generates a directed acyclic graph of page loading dependencies:

Once the complete dependency graph required for page loading is set up, Lighthouse, combined with the TIME of FCP events in the Trace information, analyzes the dependency graph required for the FCP of the page:

Armed with the dependency map required for the FCP of a page, Lighthouse simulates the time it takes to request the resource in the dependency map and execute the CPU events in the dependency map under any given speed limit, providing an estimated VALUE for the FCP under any given speed limit.

Mock HTTP requests

Lighthouse simulates HTTP to calculate how long it takes to download resources under certain network conditions, rather than actually making a network request, so let’s take a look at how Lighthouse simulates this.

In the code above, Lighthouse simulates HTTP entirely, calculating how long a resource will take under a given network condition. And the simulation takes into account details like HTTP2 multiplexing, whether the request is KeepAlive, TCP three-way handshake, congestion Windows, and so on.

We use a figure to summarize and compare the difference of FCP calculation process between the two speed limiting methods:

Both of the two rATE-limiting methods are based on Trace information provided by DevTools. In Simulate rate-limiting mode, they need to Simulate the estimated value under rate-limiting conditions after obtaining the FCP value. Under the Simulate rate limit mode, other performance indicators such as FMP and SpeedIndex were simulated in a similar way. Thus, we analyzed the implementation principle of THE FCP audit item of the Lighthouse performance indicator.

conclusion

This article gives you a brief introduction to Lighthouse, analyzes its testing process and main module implementation, and finally introduces the simulation calculation method of FCP, a key performance indicator, in the hope that you can learn something from it. At the end of the article will post the module mentioned in the source code navigation, interested friends can see, welcome to exchange.

The source code navigation

Driver module

driver
connection
emulation
network-recorder

Gatherer module

gather-runner
gatherer
runtime-exceptions

Audit module

audit
error-in-console

FCP calculation

audit/fcp
computed/fcp
computed/lantern-fcp
computed/lantern-metric
computed/page-dependency-graph
dependency-graph/base-node
dependency-graph/tcp-connection

reference

Lighthouse-architecture
Chrome DevTools Protocol
Lighthouse Scoring Guide

This article is published from netease Cloud music front end team, the article is prohibited to be reproduced in any form without authorization. We’re always looking for people, so if you’re ready to change jobs and you like cloud music, join us!

preface

Lighthouse profile

Usage of Lighthouse

Lighthouse Test Report

Lighthouse Testing Process

Implementation of Lighthouse Module

Driver module

Two-way communication and DevTools protocol

logging

Emulation

Gatherer module

Audits module

The Report module

Implement performance indicator audit items such as FCP

The speed limit to simulate

simulated

devtools

no throttling

The Trace information

Simulation of FCP

Mock HTTP requests

conclusion

The source code navigation

Driver module

Gatherer module

Audit module

FCP calculation

reference

Related Posts

Front-end advanced salary must see -CSS handwriting

Unity Editor Extension – See how resources are referenced

Json Schema profile