1. Enter the url

When you start typing in a url such as www.cnblogs.com, the browser will then search your bookmarks or history for relevant recommendations.

2, the browser to find the IP address of the domain name

The first step in navigation is to find out the IP address by the domain name you are visiting. The DNS lookup process is as follows:

  • Browser cache – The browser caches DNS records for a period of time. Interestingly, the operating system does not tell the browser how long to store DNS records, so different browsers store DNS records for a fixed amount of time (anywhere from 2 to 30 minutes).

  • System cache – If the desired record is not found in the browser cache, the browser makes a system call (gethostbyName in Windows). This allows you to obtain records in the system cache.

  • Router cache – The previous query request is then sent to the router, which typically has its own DNS cache.

  • ISP DNS cache – The next thing to check is the server where the ISP caches DNS. You can usually find the corresponding cached record here.

  • Recursive search – Your ISP’s DNS server performs a recursive search starting with the domain name server, from the.com top-level domain name server to the Facebook domain name server. The DNS server usually has the domain name in the.com domain name server in its cache, so the matching process to the top-level server is not necessary.

① After the request is initiated, the browser will first resolve the domain name. First, it will check the hosts file on the local disk to see if there is a rule corresponding to the domain name. If there is, it will directly use the IP address in the hosts file.

② If the IP address cannot be found in the local hosts file, the browser sends a DNS request to the local DNS server. The local DNS server is usually provided by your network access server provider, such as China Telecom and China Mobile.

③ After the DNS request for the url you entered reaches the local DNS server, the local DNS server will query its cache record first. If this record exists in the cache, you can directly return the result. This process is recursive query. If no, the local DNS server queries the DNS root server

④ The root DNS server does not record the mapping between domain names and IP addresses. Instead, the root DNS server informs the local DNS server that you can query the IP address of the domain server. The process is iterative

⑤ The local DNS server continues to make requests to the domain server, which in this case is the.com domain server. After receiving the request, the.com domain server does not directly return the mapping between the domain name and IP address. Instead, it tells the local DNS server the address of the resolution server for your domain name

6. In the end, the local DNS server to the domain name resolution server request, then will be able to receive a corresponding relationship between domain name and IP address, not only to the local DNS server IP address is returned to the user’s computer, and the corresponding relations between stored in the cache, arrange another time with other user query, can return the result directly, to speed up the network access.

Related issues:

What is DNS hijacking?

Domain Name hijacking, hijacking DNS (Domain Name System), refers to the hijacked intercept DNS request within the limits of network, analyze the request of the Domain Name, outside the scope of review requests, return false IP address or do nothing lose request response, its effect is to specific network can not access url is false or visit. To put it simply, you enter the website of Zhihu, but jump to the page of Baidu.

Communication process

TCP/IP is divided into four layers, each layer encapsulates data when sending:

1. Application layer: Sends HTTP requests

Having obtained the IP address of the server in the previous step, the browser will start constructing an HTTP message that contains:

  • Request headers: Request method, destination address, protocol to follow, and so on
  • Request body (other parameters)

Points to note:

  • Browsers can only send GET and POST methods, but open web pages using GET methods

2. Transport layer: TCP transmits packets

The transport layer initiates a TCP connection to the server. To facilitate transmission, the transport layer divides the data (in packet segments) and marks the numbers so that the server can accurately restore the packet information when receiving the data.

Before establishing a connection, the TCP three-way handshake is performed.

The TCP/IP three-way handshake has been vividly depicted in many paragraphs and pictures on the Internet. Related knowledge:

  1. SYN flood attack

3. At the network layer, query the Mac address using the IP protocol

Package the data segment, add the source and destination IP addresses, and find the transmission route.

Check whether the destination IP address and the current IP address reside on the same network. If yes, the IP address is sent based on the Mac address. Otherwise, use the routing table to search for the next-hop address and use ARP to query its Mac address.

Note: In the OSI reference model ARP is at the link layer, but in TCP/IP it is at the network layer.

4. Link layer: Ethernet protocol

Ethernet protocol

According to the Ethernet protocol, data is divided into packets in the unit of “frame”, and each frame is divided into two parts:

  • Header: Sender, receiver, and data type of a packet
  • Data: The content of the packet

The Mac address

The Ethernet provides that all devices connected to the network must have a NETWORK adapter interface. Data packets are transmitted from one network adapter to another. The ADDRESS of the network adapter is the Mac address. Each Mac address is unique and has one-to-one capability.

radio

The method of sending data is very primitive. The data is directly sent to all the machines on the local network through THE ARP protocol. The receiver accepts the data based on the comparison between the header information and its OWN Mac address.

Note: The receiver responds unicast.

Related knowledge:

  1. ARP attack

The server accepts the request

The process of acceptance is to reverse these steps, as shown above.

A flowchart

HTTPD

The most common types of HTTPD are Apache and Nginx, commonly used on Linux, and IIS on Windows.

It listens for the incoming request and then starts a child process to process the request.

Handle the request

After receiving a TCP packet, the system processes the connection, resolves the HTTP protocol (request method, domain name, and path), and performs some authentication:

  • Verify that a virtual host is configured
  • Verify that the virtual host accepts this method
  • This method can be used to authenticate the user (based on IP address, identity information, etc.)

redirect

If the server has configured HTTP redirection, it will return a 301 permanent redirect response, and the browser will re-send the HTTP request based on the response (rerun the process above).

The URL rewrite

It then looks at the URL rewriting rules, and if the requested file is a real one, such as an image, HTML, CSS, JS file, etc., it will be returned directly.

Otherwise, the server will rewrite the request to a REST-style URL as required.

It then decides what type of dynamic file interpreter to call to handle the request, based on the dynamic language script.

Taking the MVC framework of the PHP language as an example, it first initializes some environment parameters, matches the route from top to bottom based on the URL, and then gives way to the defined method to handle the request.

5. The browser accepts the response

After the browser receives the response resource from the server, it analyzes the resource.

Start by looking at the Response header and doing different things based on the different status codes (such as the redirection mentioned above).

If the response resource is compressed (such as gzip), you also need to decompress it.

The response resources are then cached.

Next, the response content is parsed based on the MIME type in the response resource (for example, HTML and Image are parsed in different ways).

3. Establish the TCP connection

After obtaining the IP address corresponding to the domain name, the system sends a TCP connection request to port 80 of the WEB server using a random port (1024~ 65535). The connection request is entered into the TCP/IP protocol stack of the kernel (used to identify the connection request, unpack the packet, and unpack it layer by layer). There may also be through the Netfilter firewall (module belonging to the kernel) filtering, eventually reached the WEB program, and finally established a TCP/IP connection, for the client and server TCP connection, must say is the “three handshake”.

The client sends a packet with the SYN flag to the server. After receiving the packet, the server sends a packet with the SYN/ACK flag to confirm the connection. Finally, the client sends another packet with the ACK flag to indicate that the handshake is complete and the connection is successful.

After popularization:

Client: I want to link with you

Server: Ok, agreed

Client: Ok

4. The browser sends an Http request to the WEB server

After a TCP connection is established, an HTTP request is made, which consists of three parts

Request method URI protocol/version

Request Header

Request body

Below is a complete request

5. Server-side processing

After the server receives the request, the Web server (HTTP server to be exact) processes the request, such as Apache, Ngnix, IIS, etc. The Web server parses the user request, knows which resource files need to be scheduled, processes the user request and parameters through these resource files, and calls the database information, and finally returns the results to the browser client through the Web server.

6. Close the TCP connection

To avoid resource consumption and wastage on both the server and client, either party can initiate a shutdown request when no request or response is delivered. Similar to the three-way handshake used to create a TCP connection, four handshakes are required to close a TCP connection.

Can be popularized:

Client: brother, I have no data to transfer, let’s close the link

Server: Ok, I got it. Let me see if I have any more to send

Server: I don’t have any either. Close it

Client: Ok

Three handshakes & four waves later

When the network process receives the status code of the response packet, it performs the corresponding operation. For example, when the status code is 200 OK, the content-Type header field in the response packet will be parsed. For example, application/javascript, Text/CSS and text/ HTML will appear in the content-type field in our process. That is, corresponding to Javascript files, CSS files, HTML files.

7. The viewer interprets resources

For access to HTML, CSS, JS, images and other resources.

The browser parses the HTML, generates the DOM tree, parses the CSS, generates the CSS rule tree, and then generates the render tree from the DOM tree and the CSS rule tree. Unlike DOM trees, render trees do not have nodes that do not need to be displayed, such as head or display none.

While parsing CSS, you can continue to load parsing HTML, but when parsing and executing JS scripts, it will stop parsing subsequent HTML, which will cause blocking problems. On JS blocking related issues, here but more elaboration, will be explained separately at the beginning.

8. Viewer layout rendering

Everyone knows that the page rendering process is a separate point in the interview, and it often leads to another question: how to avoid backflow and redraw.

The rendering process is the final step in the process from the editor input URL to the page rendering. The page rendering process can be divided into 9 steps:

  • parsingHTMLgenerateDOM 树
  • parsingCSSgenerateCSSOM
  • Load or executeJavaScript
  • Generate render tree (Render Tree)
  • layout
  • layered
  • Generate draw list
  • rasterizer
  • According to

2.5.1 Building a DOM Tree Because the network process transmits HTML strings to the renderer process, the renderer needs to convert the HTML strings into a DOM tree.

Note that this DOM tree is different from the ONE in the Element TAB in Chrome-Devtool. It exists in memory and provides JavaScript manipulation of the DOM.

Build CSSOM

The process of building CSSOM is to generate CSSOM by parsing CSS files, style tags, inline styles, and so on. And this process does a few things:

  • To standardize CSS, convert color: blue to color: RGB (), which can be understood as a process similar to ES6 to ES5

  • Calculating element styles, such as CSS styles, inherits parent styles, such as font size, color, etc.

The CSS Object Model is a set of apis that allow CSS manipulation with JavaScript. See MDN for details on the API

Loads the JavaScript

Typically, if JavaScript is loaded at the same time as building a DOM tree or CSSOM, the build of the former will be suspended. Of course, we can load JavaScript asynchronously by defer or sync. Both defer and sync can load JavaScript asynchronously, but the former waits for the CSSOM and DOM tree to build before executing the JavaScript, while the latter executes immediately after the asynchronous load, which still blocks using sync. The process of JavaScript execution, however, is the process of compiling and running JavaScript. Because JavaScript is an interpreted language. So the process would look something like this:

  • For each line of code, i.eToken 化
  • According to theTokenTo generate theAST(Abstract Sytanx TreeAbstract syntax tree and create context
  • The interpreter parses and executesASTGenerate bytecode.
  • The compiler generates machine code for the code that needs to be executed repeatedly to improve operation efficiency

Generate Render Tree

Once you have the DOM Tree and CSSOM, you need to combine the two to generate the Render Tree, and this process removes the display: node nodes. At this point, the render tree has the element and its style information.

layout

According to the Render Tree, each node in the Tree is calculated to determine the width, height and position of each node in the page.

It is important to note that the first process of determining the size and location of nodes is called layout, while the second is called backflow

layered

Because of the cascading context, the rendering engine creates layers for elements that have the cascading context, and the combination of layers creates some of the page effects we see. For example, some 3D effects and animations are based on layers.

It is worth mentioning that there is also stratification for content overflow with rollers

Generate draw list

For the part of the page that has layers, it needs to be drawn in order, and for this process, the rendering engine will divide the layer drawing into drawing instructions, and form a drawing list according to the layer drawing order.

rasterizer

Once you have a drawing list, the compositing thread in the render engine blocks the layer according to the size of the current viewport. The compositing thread then generates a bitmap of the blocks near the viewport, known as rasterization. The renderer process also maintains a rasterized thread pool for converting blocks into bitmaps. The rasterization process usually uses GPU acceleration, such as wil-change and opacity, which will be accelerated by GPU

According to

When all the blocks are rasterized, the compositing thread in the rendering engine generates instructions to draw the blocks and submits them to the browser process. The browser process then draws the page into memory. Finally, the memory drawing results are displayed on the user interface.

And this whole process from the generation of drawing list, raster, display, is often referred to as the process of redrawing

Conclusion The whole browser input URL to page rendering process involves a very wide range of knowledge, such as Chrome multi-process architecture, HTTP communication process, browser parsing JavaScript process, browser page drawing process and some basic computer knowledge and so on, and, This whole process analysis is actually closely related to Chrome-DevTools.