Here’s a question that comes up a lot in interviews these days: What happens when you type a URL into a browser and the entire page is displayed to the user? It’s a cliche, but there’s a lot going on here.

In general, it can be divided into the following processes:

  • Your URL
  • The DNS
  • A TCP connection
  • Sending an HTTP request
  • The server processes the request
  • The server responds to requests
  • The browser parses the rendered page
  • Connect the end of the

Enter the URL

URL Is a concise representation of the location and access method of a resource that can be obtained from the Internet. It is the address of a standard resource on the Internet. Every file on the Internet has a unique URL that contains information indicating where the file is and what the browser should do with it.

Protocol :// hostname[:port] / path / [;parameters][? Query]#fragment

  • Protocol
  • Hostname (hostname)
  • Port (Port number)
  • Path (path)
  • Parameters:
  • The query (query)
  • Fragment (information)

When we start typing urls into the browser, the browser is already intelligently matching possible URLS. It will look at history, bookmarks, etc., to find possible URLS for the entered string, and then give you an intelligent prompt to complete the URL. In the case of Google’s Chrome browser, it will even display the page directly from the cache, meaning it will appear before you press Enter.

The DNS

DNS resolution is the process of finding which machine has the resources you need. When you enter an address in the browser, such as www.baidu.com, it is not the real address of baidu website. The unique identification of every computer on the Internet is its IP address, but IP addresses are not easy to remember. Users prefer to find other computers on the Internet using easy-to-remember sites, such as baidu’s. So Internet designers need to make a tradeoff between user convenience and usability, and that tradeoff is a url to IP address translation, a process known as DNS resolution. It actually acts as a translator, realizing the translation from web address to IP address. How does url – to – IP address translation work?

Search order: browser cache > OPERATING system cache > local host file > router cache > ISP DNS cache > top-level DNS server/root DNS server

1, browser cache: first to the browser cache to read the last access to the record, in Chrome can be entered in the address bar chrome://net-internals/# DNS to view the current status of the cache.

2. Operating system cache: Find the cache stored in the system running memory. On a MAC, you can use the following command to clear the DNS cache in the system.

dscacheutil -flushcache
Copy the code

**hosts file: ** Check the hosts file on the local disk to see if there is a rule corresponding to this domain name. If there is, use the IP address in the hosts file directly.

4. Router cache: Some routers also have the function of DNS cache. The visited domain name is stored on the router.

5. ISP DNS cache: Internet service providers (such as China Telecom) will also provide DNS services, such as the well-known 114.114.114.114. If the DNS service cannot be found locally, it will query the DNS service to ISP. ISP will search for records in the cache of the current server, and if so, it will return the IP address. Requests to the root DNS server for queries begin.

6. Top-level DNS server/root DNS server: After receiving the request, the root DNS identifies which server is authorized to manage the domain name (.com) and returns the IP address of the top-level DNS server. After receiving the IP address of the top-level DNS server, the requester initiates a query to the server. If the server cannot be resolved, the server returns the IP address of the next-level DNS server (nicefilm.com). The local machine continues to search until the server finds the host (www.nicefilm.com).

6, in the end, the local DNS server to the domain name resolution server request, then will be able to receive a corresponding relationship between domain name and IP address, not only to the local DNS server IP address is returned to the user’s computer, and stored in the cache, the relation between arrange another time with other user query, can return the result directly, to speed up the network access.

The following chart perfectly illustrates this process:

optional title

The above picture is the process of finding the IP address of www.google.com. If no IP address is found, the local DNS server will send a request to the root DNS server. If the root DNS server does not have the domain name, the local domain name will send a request to the COM top-level DNS server, and so on. Until finally the local DNS server gets Google’s IP address and caches it locally for the next query. From the above process, we can see that the url resolution is a process from right to left: com -> google.com -> www.google.com. But did you notice something missing, the root DNS resolution process? Actually, the actual url is www.google.com. It’s not that I typed one more. This one. The root domain name server (root domain name server) is the root domain name server (root domain name server). By default, the root domain name server (root domain name server) is the root domain name server (root domain name server). ->.com -> google.com. -> www.google.com.

Supplement:

  1. What is DNS?

    The Domain Name System (DNS) is a distributed database that maps Domain names and IP addresses on the Internet. It enables users to access the Internet more conveniently without having to remember IP numbers that can be read directly by machines. The process of obtaining an IP address from a host name is called domain name resolution (or hostname resolution).

    In layman’s terms, we are more used to remembering the name of a website, such as www.baidu.com, than its IP address, such as 167.23.10.2. Computers are better at remembering the IP address of a website than links like www.baidu.com. DNS is like a phone book. If you’re looking for www.baidu.com, I’ll look in my phone book, and I’ll say, oh, its IP is 167.23.10.2.

  2. DNS query can be performed in two ways: recursive query and iterative query

    2.1. Recursive analysis

    When the local DNS server cannot answer DNS queries by itself, it needs to query other DNS servers. There are two ways to do this, and the one shown here is recursive. The local DNS server queries other DNS servers. Generally, the local DNS server queries the root domain server of the domain name first, and then the root DNS server searches the domain name from one level to the next. The query result is returned to the local DNS server, and the local DNS server returns the query result to the client.

    2.2. Iterative analysis

    If the local DNS server cannot answer DNS queries by itself, it can also resolve DNS queries by iterative query, as shown in the figure. The local DNS server does not query the domain name of other DNS servers. Instead, it returns the IP addresses of other DNS servers that can resolve the domain name to the client DNS program. The client DNS program then queries these DNS servers until the query result is obtained. In other words, iterative parsing just helps you find the relevant server, not look it up. For example, the server IP address of Baidu.com is 192.168.4.5. Please check it yourself. I am busy, so I can only help you here.

  3. DNS Indicates the organization mode of a domain name

    We talked earlier about root DNS servers, domain DNS servers, these are all ways of organizing the DNS domain name space. See the following table for an introduction to the five categories used to describe DNS names in their functional namespaces, along with examples of each name type

  4. DNS optimization

    Knowing the DNS process, what can we learn from it? The above mentioned request for Google’S IP address goes through 8 steps, and there are multiple requests in this process (UDP and TCP requests exist at the same time, why there are two request methods, please find by yourself). Is it too time-consuming to go through so many steps every time? How do you reduce the number of steps in the process? That’s the DNS cache.

    4.1 the DNS cache

    DNS has multiple levels of cache, sorted by distance from browser: browser cache, system cache, router cache, IPS server cache, root DNS cache, top-level DNS cache, and primary DNS cache.

    • Type :chrome:// DNS/into your Chrome browser and you can see chrome’s DNS cache.
    • The system cache is stored in /etc/hosts(Linux) :

    4.2 DNS Load Balancing

    When a site has enough users, if each resource request is on the same machine, that machine can pop at any time. The solution is DNS load balancing, The principle of it is for the same in the DNS server host name configure multiple IP addresses, in response to the DNS query, the DNS server for every query will with the IP address of the DNS host file record return different analytic results according to the order will lead to different client access machines, make different client access server, so as to achieve the negative The purpose of load balancing. For example, it can be based on the load of each machine, the distance of the machine from the user’s geographical location, etc.

Establishing a TCP Connection

After obtaining the IP address corresponding to the domain name, the browser sends a TCP connection request to the server’s WEB application (HTTPD,nginx, etc.) on a random port (1024< port <65535). After the connection request to the server (except within the intermediate routing device, through a variety of local area network), into the network card, and then into the kernel of the TCP/IP protocol stack (used to identify the connection request, unsealed packets, one layer of peel away), and may go through the Netfilter firewall filtering (belong to the kernel module), Finally, the WEB application is reached, and the TCP/IP connection is finally established.

The TCP connection is shown below:

Three – way handshake to establish TCP connection

First handshake: When establishing a connection, the client sends a SYN packet (SYN = J) to the server and enters the SYN_SENT state, waiting for confirmation from the server. SYN: Indicates the Synchronize Sequence number.

Second handshake: After receiving a SYN packet, the server must acknowledge the client’s SYN (ACK = J +1) and send a SYN packet (ACK = K). In this case, the server enters the SYN_RECV state.

Third handshake: After receiving the SYN+ACK packet from the server, the client sends an ACK packet (ACK = K +1) to the server. After the packet is sent, the client and the server enter the ESTABLISHED state (TCP connection is successful) and complete the three-way handshake.

Supplement:

Why do you need three handshakes?

The purpose of the “three-way handshake” is described in The fourth edition of Computer Networks “to prevent an invalid connection request segment from suddenly being sent to the server and causing an error.” An example of a “invalid connection request segment” occurs when: The segment of the first connection request packet sent by the client is not lost, but is detained on a network node for a long time. As a result, it is delayed to reach the server until a certain time after the connection is released. Originally, this is an invalid packet segment. However, after the server receives the invalid connection request packet segment, it mistakenly thinks it is a new connection request sent by the client. Then the client sends a confirmation message to agree to establish a connection. Assuming that the “three-way handshake” is not used, a new connection is established as soon as the server sends an acknowledgement. Since the client does not send a connection request, it ignores the server’s confirmation and does not send data to the server. However, the server assumes that the new transport connection has been established and waits for data from the client. As a result, many of the server’s resources are wasted. The three-way handshake prevents this from happening. For example, the client does not issue an acknowledgement to the server’s acknowledgement. When the server receives no acknowledgement, it knows that the client has not requested a connection.” . The main purpose is to prevent the server from wasting resources by waiting.

Sending an HTTP request

After the TCP connection is established, an HTTP request is made. A typical HTTP request header should include request methods, such as GET or POST. Unusual methods include PUT and DELETE, HEAD, OPTION, and TRACE. Normal browsers can only make GET or POST requests.

When the client makes an HTTP request to the server, there is some request information, including three parts:

  • Request method URI protocol/version
  • Request Header
  • Request body:

Here is a complete HTTP request example:

GET/sample. JspHTTP / 1.1 Accept: image/GIF image/jpeg, * / * Accept - Language: useful - cn Connection: Keep Alive - Host: localhost The user-agent: Mozila / 4.0 (compatible; MSIE5.01; The Window NT5.0) Accept - Encoding: gzip, deflate the username = jinqiao&password = 1234Copy the code

Note: The last request header is followed by an empty line, and the carriage return and newline characters are sent to inform the server that there are no more headers below.

The first line of the Request is “method URL discussion/version” : GET/sample.jsp HTTP/1.1 For example, the request header can declare the browser language, the length of the request body, and so on.

Accept:image/gif.image/jpeg.*/* Accept-Language:zh-cn Connection:Keep-Alive Host:localhost The user-agent: Mozila / 4.0 (compatible: MSIE5.01: Windows NT5.0) Accept - Encoding: gzip, deflate.Copy the code

Between the request header and the request body is an empty line. This line is very important because it indicates that the request header has ended and the request body is next. The request body can contain information about the query string submitted by the customer:

username=jinqiao&password=1234
Copy the code

The server redirects permanently

The server responds to the browser with a 301 permanent redirect response so that the browser visits http://www.google.com/ instead of http://google.com/.

Why does the server have to redirect rather than just send the content the user wants to see? One reason has to do with search engine rankings. If a page has two addresses, such as http://www.yy.com/ and http://yy.com/, search engines will treat them as two sites, resulting in fewer links per search and lower rankings. Search engines know what a 301 permanent redirect means, and will place all addresses with and without the WWW in the same ranking. There is also the cache friendliness of having different addresses. When a page has several names, it may appear in the cache several times.

Supplement:

1. Differences between 301 and 302:

Status code of 301 and 302 all redirection, said that after the browser to get the status code returned by the server will automatically jump to A new URL address, this address can be obtained from the response to the Location of the first (the effect of the user sees is he input address A moment into another address B) – this is that they have in common. The difference between them is. 301 indicates that the resource at the old address A has been permanently removed (the resource is no longer accessible), and the search engine will swap the old url for the redirected url while fetching new content. 302 means that the resource at the old address A is still there (still accessible). This redirection is only A temporary jump from the old address A to address B. The search engine will grab the new content and save the old url. Good SEO302 in 301

2. Redirection reason:

(1) Website adjustment (such as changing the structure of web directory); (2) The page is moved to a new address; (3) Webpage extension changes (such as the application needs to change.php. Html or. SHTML). In this case, if you do not do redirection, the old address in the user favorites or search engine database can only let visitors get a 404 page error message, the traffic is lost; Moreover some registered multiple domain name website, also need to access these domain name users automatically jump to the main site through redirection.

3. When do I jump to 301 or 302?

When a website or webpage temporarily moves to a new location within 24 to 48 hours, it is necessary to perform a 302 jump. The scenario where 301 jump is used is that the previous website needs to be removed for some reason and then accessed to the new address, which is permanent. To be clear, the following are the general scenarios for using the 301 redirect:

  • When your domain name expires, you don’t want to renew it (or you found a better domain for your site) and want to change your domain name.
  • In the search engine search results appeared without the WWW domain name, and with the WWW domain name is not included, this time you can use 301 redirect to tell the search engine our target domain name is which one.
  • The space server is not stable, when changing space.

The server processes the request

After all these steps, we finally send our HTTP request to the server. In fact, the previous redirect has already arrived at the server. So, how does the server handle our request?

After receiving A TCP packet on a fixed port, the backend processes the TCP connection, parses the HTTP protocol, and encapsulates the PACKET into an HTTP Request object for upper-layer use.

Some of the larger sites will send your requests to a reverse proxy server, because when the site is very heavily visited and the site is getting slower and slower, one server is no longer enough. The same application is deployed on multiple servers and requests from a large number of users are distributed across multiple machines. In this case, the client does not directly access a web application server through HTTP. Instead, Nginx first requests the application server, and Nginx then requests the application server, and then returns the result to the client. Here, Nginx acts as a reverse proxy server. It also has the advantage that if one of the servers goes down, users will not be affected as long as other servers are running properly.

As shown in the figure:

Supplement:

What is a reverse proxy?

The web administrator can add an Nginx in the middle. The client requests Nginx, and Nginx requests the application server, and then returns the result to the client. In this case, Nginx is the reverse proxy server.

The server returns an HTTP response

After the previous six steps, the server receives our request, processes our request, and at this point it returns the result of its processing, namely an HTPP response. An HTTP response is similar to an HTTP request. An HTTP response consists of three parts:

  • The status line
  • Response Header
  • A blank line
  • In response to the body
HTTP/1.1 200 OK Date: Sat, 31 Dec 2005 23:59:59 GMT Content-Type: text/ HTML; HTTP/1.1 200 OK Date: Sat, 31 Dec 2005 23:59:59 GMT Content-Type: text/ HTML; Charset =ISO-8859-1 Content-Length: 122 < HTML > < head > < title > HTTP < /title > < /head > < body > <! -- Body goes here -- > < /body > < / HTML >Copy the code

Status line: The status line is described by the protocol version, numeric status code, and corresponding status. Each element is separated by a space. Format: http-version status-code reason-phrase CRLF For example: HTTP / 1.1 200 OK \ r \ n | – protocol version: it is to use http1.0 or other version | – state description: state description gives a short description about the status code. Such as for a status code of 200 described as code: ok | – state status code consists of three digits, the first number defines the response categories, and there are five possible value. The following

1xx: information status code: indicates that the server has received the client request and the client can continue to send the request.

  • 100 Continue
  • 101 Switching Protocols

2xx: success status code, which indicates that the server has successfully received and processed the request.

  • 200 OK Indicates that the client request is successful
  • 204 No Content Is successful, but does not return the body of any entity
  • 206 Partial Content A Range request was successfully executed

3xx: redirection status code, indicating that the server requires client redirection.

  • 301 Moved Permanently redirected Permanently. The Location header of the response packet should contain the new URL of the resource
  • 302 Found Temporary redirection. The URL in the Location header of the response packet is used to locate the resource temporarily
  • 303 See Other The requested resource has another URI. The client should use the GET method to obtain the requested resource
  • 304 Not Modified The server content is Not updated and can be read directly from the browser cache
  • 307 Temporary Redirect Temporary redirection. Same meaning as 302 Found. 302 Do not allow POST to be changed to GET, but this is not always the case. 307 More browsers may follow this standard, but it depends on the browser implementation

4xx: indicates the client error status code, indicating that the client request contains invalid content.

  • 400 Bad Request Indicates that the client Request has syntax errors and cannot be understood by the server
  • 401 Unauthonzed indicates that the request is unauthorized. This status code must be used with the WWW-Authenticate header field
  • 403 Forbidden Indicates that the server receives a request but refuses to provide the service. The reason for not providing the service is usually given in the response body
  • 404 Not Found The requested resource does Not exist, for example, an incorrect URL was entered

5xx: indicates the server error status code. It indicates that an unexpected error occurs because the server fails to properly process requests from clients.

  • 500 Internel Server Error Indicates that an unexpected Error occurs on the Server. As a result, the client request cannot be completed
  • 503 Service Unavailable Indicates that the server cannot process requests from clients. The server may recover after a period of time

Header:

Response header: Consists of two keyword/value pairs, one on each line. The keyword and value are separated by colons (:). Typical response headers are:

Response body:

Contains the specific information we need, such as cookies, HTML,image, request data returned from the back end, and so on. It should be noted that there is a space between the response body and the response header, indicating that the information in the response header ends in the space. The following figure shows the request body captured by Fiddler, in the red box:

Browser displays HTML

After the browser receives the HTML,CSS, and JS files, how does it render the page to the screen? Below is the WebKit rendering process.

Build the DOM tree -> build the Render tree -> Layout the Render tree -> Render the render tree

Rendering process:

The browser is a process of parsing and rendering. First, the browser parses the HTML file to build the DOM tree, then the CSS file to build the render tree, and once the render tree is built, the browser lays out the render tree and draws it to the screen. The process is complicated and involves two concepts: reflow(reflow) and repain(redraw). Each element in the DOM node exists in the form of a box model, which requires the browser to calculate its position and size, a process called relow. Once the location, size, and other properties of the box model, such as color and font, are determined, the browser begins to draw the content, a process called repain. Pages will inevitably experience reflow and Repain when they first load. Reflow and Repain processes can be very performance draining, especially on mobile devices, and can ruin the user experience, sometimes causing pages to stagnate. So we should reduce reflow and repain as little as possible.

When a JS file is encountered during the document loading process, the HTML document will suspend the render (load parsing render synchronization) thread, and not only wait for the js file loading in the document, but also wait for the parsing execution to complete, before resuming the HTML document rendering thread. Because JS may modify the DOM, the most classic document.write, this means that all subsequent resource downloads may not be necessary until JS execution is complete, which is the root cause of JS blocking subsequent resource downloads. So I know that in normal code, JS is at the end of the HTML document.

JS parsing is done by the JS parsing engine in the browser. JS is single-threaded, that is, only one thing can be done at a time, and all tasks need to be queued so that the first task can finish before the next one can start. However, there are some tasks that are time-consuming, such as IO reads and writes, so you need a mechanism to do the synchronous and asynchronous tasks first.

The execution mechanism of JS can be regarded as a main thread plus a task queue. Synchronous tasks are tasks that are executed on the main thread, and asynchronous tasks are tasks that are executed on the task queue. All synchronization tasks are executed on the main thread, forming an execution stack. An asynchronous task will place an event in the task queue when it has a result. When a script runs, it first runs the execution stack in sequence, then extracts the events from the task queue and runs the tasks in the task queue. This process is repeated constantly, so it is also called Event loop.

Connect the end of the

Keep-alive is enabled by default to optimize request time, so the exact time a TCP connection is closed is when the TAB TAB is closed. This closing process is known as the four waves. Closing is a full duplex process, and the order of sending packets is not determined. Generally, the shutdown is initiated by the client, and the process is as follows.

For an established connection, TCP releases the connection using an improved three-way handshake (using a message segment with a FIN tag attached). The procedure for TCP to close the connection is as follows:

In the first step, when host A’s application notifies TCP that the data has been sent, TCP sends host B A packet segment with the FIN tag attached (FIN stands for Finish).

The second step, the host B after received the FIN message, do not immediately with FIN reply message segment host A, but first send A confirmation number to host A ACK, and notice their corresponding application: seeking close the connection (the first is the purpose of an ACK to prevent during this period, the other the retransmission FIN message section).

Third, host B’s application tells TCP: I want to close the connection completely, and TCP sends A FIN segment to host A.

Step 4 after receiving the FIN packet, host A sends an ACK to host B to indicate that the connection is released.

Supplement:

Why is there a three-way handshake when you connect and a four-way handshake when you close?

A: After receiving a SYN request packet from the Client, the Server sends a SYN+ACK packet. ACK packets are used for reply, and SYN packets are used for synchronization. However, when the Server receives a FIN packet, the SOCKET may not be closed immediately. Therefore, the Server can only reply with an ACK packet to tell the Client, “I received the FIN packet you sent.” I can send FIN packets only after all packets on the Server are sent. Therefore, THE FIN packets cannot be sent together. Therefore, a four-step handshake is required.

conclusion

So far, the whole article temporarily here, the content of the article is mostly reference online information, there are a lot of details need to sort out, if there is insufficient, I hope to point out more!

References:

Cliche – What happens from entering the URL to displaying the page

Typical front-end interview question: What happens from entering the URL to loading the page?

What happens from entering the URL to rendering the page?