Original is not easy, I hope you can pay attention to us, and then easily point a praise ~~

This article was first published on the front end team blog: Automated Web Performance Optimization Analysis Solutions

In the process of building a Web site, every detail can affect the speed of access to the site. If developers do not know about front-end performance, many factors that adversely affect website access speed may accumulate online, seriously affecting website performance, resulting in slow website access, poor user experience, and ultimately user loss. Page performance is critical to web pages. Therefore, the performance of the page detection analysis, developers can not be ignored. So how do we monitor and analyze pages and evaluate their performance? What are the rules for evaluating performance?

In terms of technology, front-end performance Monitoring is mainly divided into two ways, one is called Synthetic Monitoring (SYN) and the other is Real User Monitoring (RUM).

  • Synthetic monitoring, is in a simulation scenario, to submit a need to do performance testing page, through a series of tools, rules to run your page, extract some performance indicators, a performance report.

  • User monitoring, true is that users access on our page, visit can produce all kinds of performance data, we leave the page the user, the performance data uploaded to our log server, data extraction, cleaning, processing, finally on our monitoring platform to display a process.

The former focuses on “detection”, while the latter focuses on “monitoring”.

The following will introduce to you from multiple aspects how the Web performance optimization analysis system of ZooTeam in front of Zhengcai Cloud – “Baicai” collects page performance data and produces page performance reports through a series of calculation and processing.

Baicce, named after the historical figure Wei Zheng. Wei Zheng, styled Xuan Cheng. Tang Taizong Li Shimin’s remonstrate minister, dare to remonstrate directly, known as the minister. He was known as “Wei Baise” for repeatedly pointing out the faults of Li Shimin, emperor Taizong of the Tang Dynasty, and proposing effective policies.

How do I collect page performance data

The Best system uses the “synthetic monitoring” scheme mentioned above to obtain a set of performance data independent of the host environment. As for synthetic monitoring solutions, Lighthouse is the open-source automated analytics tool from the Google Chrome team.

Lighthouse’s workflow has several main steps. Some of the steps take place in the browser, and the rest are performed by the Lighthouse runner.

How Lighthouse works

Here are the components of Lighthouse:

  • The driver interacts with the Chrome Debugging Protocol.
  • Collectors use drivers to collect web page information. Minimize post-processing. The output of a collector is called an Artifact.
  • The Artifact is taken as input by the inspector, which runs a test against it and assigns pass/fail/score results.
  • Categories group the results of the review into user-facing reports (such as best practices). So you add the weights and you get the score.

In simple terms, the process is: establish a connection -> collect logs -> analyze -> generate reports.

On the basis of Lighthouse, our Baise system has been partially customized and innovated.

Different from Lighthouse’s features

1) Use Puppeteer

Puppeteer is a Node library that provides a high-level API for controlling Chromium or Chrome via the DevTools protocol. Compared to Selenium or PhantomJs, its biggest features are that its DOM manipulation can be simulated completely in memory, i.e. processed in V8 without opening the browser, and maintained by the V8 Chrome team with better compatibility and prospects.

To get the width and height of the viewport area, run the following code:

const puppeteer = require('puppeteer');

(async () = > {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');

  const dimensions = await page.evaluate((a)= > {
    return {
      width: document.documentElement.clientWidth,
      height: document.documentElement.clientHeight
    };
  });

  console.log('Dimensions:', dimensions);

  awaitbrowser.close(); }) ();Copy the code

Lighthouse’s solution is to use the lower-level DevTools protocol to interact with the browser. We chose Puppeteer because it is simpler and encapsulates a number of commonly used interfaces to meet our needs.

2) Simulate login before accessing the page

We need to detect not only the front pages that can be accessed anonymously, but also the back pages that can be accessed only after login. Therefore, we need to add a login operation before detection. Of course, the username and password is also optional, and the user will be reminded to enter the username and password only when it detects that the URL requires login.

3) The collected data is stored in the database

We saved every detection data into the database, sorted out the data, obtained the trend of page performance changes, and provided data guarantee for supporting subsequent statistical analysis.

After the transformation, the process is: establish connection -> login detection -> log collection -> data storage -> analysis -> report generation

How do I analyze page performance

Customize indicators

At the beginning of system design, we worked out the weights and thresholds of a series of performance indicators:

Lighthouse also has a number of performance metrics, such as:

  • Uses-webp-images: Provides images in a new generation format
  • Dom-size: indicates the total number of DOM nodes
  • Network-requests: number of requests when the page was loaded
  • … .

In addition, we customized more features such as whether OSS images use compression suffixes.

Because the company’s pictures are placed on Ali Cloud OSS, Ali Cloud OSS provides a very convenient way to optimize the image volume, that is, to add similar parameters to the end of the picture? X-oss-process =image/resize, W_187 /quality,Q_75/format,webp. Why is it theoretical? Because we found that if the image was compressed by other compression tools before uploading OSS, and then the compression suffix was added, the volume of the image would not change much, or even have negative growth. So instead of simply checking to see if the image has an OSS suffix added, we have adjusted the review algorithm to compare the volume difference between the current image and the volume after the OSS compression suffix is out of proportion.

Custom detection model

After testing the system for a while, we found that not all metrics are applicable to all types of pages. For example, our front page loads a lot of pictures and requires higher resolution, while the background page is less pictures. We have a traditional backend render + jQuery type page, a client-side render React page, and a pre-rendered Vue page.

Testing all types of pages to the same standard is obviously not appropriate, so we introduced the concept of a testing model. We perform performance detection according to the corresponding detection model according to the page type. A detection model can be associated with multiple indicators, and the weight and level of indicators can be configured.

The higher the weight is, the more points will be deducted, which can be divided into error and Warning. The deduction items of warning level are not included in the total score, for example, whether the request uses Gzip or not, because this item cannot be optimized by the front end and needs to be improved by the server.

Calculate page ratings

Provide optimization suggestions and trends

Optimization Suggestions

For each penalty point review item, we provide detailed reasons for penalty points and corresponding solutions:

Page performance trend change

In addition to developing new features, we also actively refactor old pages. In order to analyze the benefits before and after refactoring, we also provide the function to query historical data and change trends:

conclusion

Bce system provides performance optimization analysis services for front-end pages. Combining the differences of industry schemes and business scenarios, bCE system improves the best practices in line with the company’s business situation. Bce system has developed a set of Web page performance testing standards. Find out the problems affecting user experience, such as long request time, excessive request resources and unreasonable page structure, and provide corresponding optimization suggestions to solve the problems, so as to help developers produce pages with faster rendering speed, fewer resources and better experience.

In the future, we will share a series of “Bce System”. You are welcome to follow the wechat public account “Zhengcai Cloud Front End Team”, or follow the “Zhengcai Cloud front end Team” on digging Gold, so as to get the latest information in the first time. Share a preview:

  • Crawler Combat: Simulating user login to grab page Performance Data
  • Introduction to NestJS: I can develop background without writing SQL
  • Document Sites: Building Continuous Integration Document Sites without Gitlab CI
  • “Picture volume optimization: Round I save 100 million for the company”
  • Node Performance Optimization: How I Got Overwhelmed by Node Timed Batch Tasks

, recruiting

ZooTeam, more than 50 partners are waiting for you to join the wave ~ If you want to change things have been torturing, I hope to start torturing things; If you want to change, you’ve been told you need more ideas, but you don’t have a solution. If you want change, you have the power to make it happen, but you don’t need it. If you want to change what you want to accomplish, you need a team to support you, but you don’t have the position to lead people. If you want to change “5 years of work and 3 years of experience”; If you want to change the original savvy is good, but there is always a layer of fuzzy window… If you believe in the power of believing, believing that ordinary people can achieve extraordinary things, believing that you can meet a better version of yourself. If you want to be a part of the growth of a front end team that has deep business understanding, technology systems, value creation, and impact spillover as the business takes off, I think we should talk. Any time, waiting for you to write something and send it to [email protected]

Recommended reading

Visual Construction System for Front-end Engineering Practice (PART 1)

After reading this, you can play React Hooks, too

Automated Web performance optimization analysis solution