1. Enter the address

When we start typing urls into the browser, the browser is already intelligently matching possible URLS. It will look at history, bookmarks, etc., to find possible URLS for the entered string, and then give you an intelligent prompt to complete the URL. In the case of Google’s Chrome browser, it will even display the page directly from the cache, meaning it will appear before you press Enter.

Second, DNS resolution

DNS resolution is the process of finding which machine has the resources you need. When you enter an address in the browser, such as http://www.baidu.com, it is not the real address of baidu website. The unique identification of every computer on the Internet is its IP address, but IP addresses are not easy to remember. Users prefer to find other computers on the Internet using easy-to-remember sites, such as baidu’s. So Internet designers need to make a tradeoff between user convenience and usability, and that tradeoff is a url to IP address translation, a process known as DNS resolution.

Analytical process:

1, find the browser cache: because the browser will generally cache DNS records for a period of time, different browsers may not be the same, generally 2-30 minutes, the browser to find these cache, if there is a cache, directly return IP, otherwise the next step.

2. Search the system cache: If the browser cannot find the IP address in the cache, the browser checks the hosts file on the local disk to see if there is a rule corresponding to the domain name. If there is, the browser directly uses the IP address in the hosts file.

3. If the IP address cannot be found in the local hosts file, the browser sends a DNS request to the local DNS server.

4. After the DNS request of the url you entered reaches the local DNS server, the local DNS server will query its cache record first. If this record exists in the cache, the local DNS server can directly return the result. If no, the local DNS server queries the DNS root server.

5. The root DNS server does not record the mapping between domain names and IP addresses. Instead, the root DNS server tells the local DNS server that you can go to the domain server to query the IP address of the domain server. The process is iterative.

6. The local DNS server continues to make requests to the domain server, in this case the.com domain server. After receiving the request, the.com domain server does not directly return the mapping between the domain name and IP address. Instead, it tells the local DNS server the address of the resolution server for your domain name.

7, in the end, the local DNS server to the domain name resolution server request, then will be able to receive a corresponding relationship between domain name and IP address, not only to the local DNS server IP address is returned to the user’s computer, and stored in the cache, the relation between arrange another time with other user query, can return the result directly, to speed up the network access.

Knowledge expansion:

1) What is DNS?

The Domain Name System (DNS) is a distributed database that maps Domain names and IP addresses on the Internet. It enables users to access the Internet more conveniently without having to remember IP numbers that can be read directly by machines. The process of obtaining an IP address from a host name is called domain name resolution (or hostname resolution).

In layman’s terms, we are more used to remembering the name of a website, such as www.baidu.com, than its IP address, such as 167.23.10.2. Computers are better at remembering the IP address of a website than links like www.baidu.com. DNS is like a phone book. If you’re looking for www.baidu.com, I’ll look in my phone book, and I’ll say, oh, its IP is 167.23.10.2.

2) Two ways of DNS query: recursive query and iterative query

1. Recursive resolution When a local DNS server cannot answer DNS queries by itself, it needs to query other DNS servers. There are two ways to do this, and the one shown here is recursive. The local DNS server queries other DNS servers. Generally, the local DNS server queries the root domain server of the domain name first, and then the root DNS server searches the domain name from one level to the next. The query result is returned to the local DNS server, and the local DNS server returns the query result to the client.

3) Organization of DNS domain name space

We talked earlier about root DNS servers, domain DNS servers, these are all ways of organizing the DNS domain name space. See the following table for an introduction to the five categories used to describe DNS names in their functional namespaces, along with examples of each name type

4)DNS load balancing

When a site has enough users, if each resource request is on the same machine, that machine can pop at any time. The solution is DNS load balancing, The principle of it is for the same in the DNS server host name configure multiple IP addresses, in response to the DNS query, the DNS server for every query will with the IP address of the DNS host file record return different analytic results according to the order will lead to different client access machines, make different client access server, so as to achieve the negative The purpose of load balancing. For example, it can be based on the load of each machine, the distance of the machine from the user’s geographical location, etc.

3. Establish a TCP connection

After obtaining the IP address corresponding to the domain name, the system sends a TCP connection request to port 80 of the WEB server using a random port (1024~ 65535). The connection request is entered into the TCP/IP protocol stack of the kernel (used to identify the connection request, unpack the packet, and unpack it layer by layer). There may also be through the Netfilter firewall (module belonging to the kernel) filtering, eventually reached the WEB program, and finally established a TCP/IP connection, for the client and server TCP connection, must say is the “three handshake”.

TCP three-way handshake:

First handshake: The client sets the SYN flag bit to 1 and randomly generates a packet with the value of SEq =J (the value of J is in the range of =1234567) to the server. The client enters the SYN_SENT state and waits for the server to confirm.

Second handshake: When the server receives the packet, the flag bit SYN=1 indicates that the client requests to establish a connection. The server sets the flag bit SYN and ACK to 1, ACK =J+1, randomly generates a value seq=K, and sends the packet to the client to confirm the connection request. The server enters the SYN_RCVD state.

Third handshake: After receiving the acknowledgement, the client checks whether the ACK is J+1 and ACK is 1. If the ack is correct, the client sets the flag ACK bit to 1 and ACK =K+1, and sends the packet to the server. The server checks whether the ACK is K+1 and ACK is 1. Data can then be transferred between client A and server B.

After popularization:

Client: I want to link with you

Server: Ok, agreed

Client: Ok

Knowledge development:

1) Why three handshakes are needed:

According to the fourth edition of Computer Networks, the purpose of the “three-way handshake” is “to prevent the invalid connection request message segment from suddenly being sent to the server and causing an error.”

The example in the book is as follows: “Invalid connection request segment” is generated when the first connection request segment sent by the client is not lost, but is delayed at a network node for a long time. As a result, it does not reach the server until a certain time after the connection is released. Originally, this is an invalid packet segment. However, after the server receives the invalid connection request packet segment, it mistakenly thinks it is a new connection request sent by the client. Then the client sends a confirmation message to agree to establish a connection.

Assuming that the “three-way handshake” is not used, a new connection is established as soon as the server sends an acknowledgement. Since the client does not send a connection request, it ignores the server’s confirmation and does not send data to the server. However, the server assumes that the new transport connection has been established and waits for data from the client. As a result, many of the server’s resources are wasted. The three-way handshake prevents this from happening. For example, the client does not issue an acknowledgement to the server’s acknowledgement. When the server receives no acknowledgement, it knows that the client has not requested a connection.” . The main purpose is to prevent the server from wasting resources by waiting.

The browser sends an HTTP request to the Web server

In fact, this part can also be called HTTP from the front end engineer’s point of view, and it mainly happens on the client side. The process of sending an HTTP request is to construct an HTTP request packet and send it to the specified port on the server through TCP (HTTP 80/8080, HTTPS 443). An HTTP request packet consists of three parts: the request line, the request header, and the request body.

The request line

The format is as follows:

Method Request-URL HTTP-Version CRLF

Eg: the GET index. HTTP / 1.1 HTML

The common methods are GET, POST, PUT, DELETE, OPTIONS, and HEAD.

The request header

The request header allows the client to pass additional information about the request and the client itself to the server. PS: The client does not necessarily refer to the browser. Sometimes you can use the Linux CURL command and the HTTP client test tool. Common request headers include Accept, accept-charset, accept-encoding, accept-language, Content-Type, Authorization, Cookie, user-agent, and so on.

Request body

When using POST, PUT, and other methods, the client is usually required to pass data to the server. This data is stored in the request body. The request header contains some information related to the request body. For example, Web applications today usually use Rest architecture, and the requested data format is json. You need to set content-Type: application/json.

The server’s permanent redirection response

The server responds to the browser with a 301 permanent redirect response so that the browser visits “www.google.com/” instead of “google.com/”.

Why does the server have to redirect rather than just send the content the user wants to see? One reason has to do with search engine rankings. If a page has two addresses, such as http://www.yy.com/ and http://yy.com/, search engines will treat them as two sites, resulting in fewer links per search and lower rankings. Search engines know what a 301 permanent redirect means, and will place all addresses with and without the WWW in the same ranking. There is also the cache friendliness of having different addresses. When a page has several names, it may appear in the cache several times.

Knowledge extension

1) The difference between 301 and 302.

Status code of 301 and 302 all redirection, said that after the browser to get the status code returned by the server will automatically jump to A new URL address, this address can be obtained from the response to the Location of the first (the effect of the user sees is he input address A moment into another address B) – this is that they have in common.

The difference between them is. 301 indicates that the resource at the old address A has been permanently removed (the resource is no longer accessible), and the search engine will swap the old url for the redirected url while fetching new content.

302 means that the resource at the old address A is still there (still accessible). This redirection is only A temporary jump from the old address A to address B. The search engine will grab the new content and save the old url. Good SEO302 in 301

2) Redirection reason:

  • 1, website adjustment (such as changing the structure of the web directory);

  • 2. The page is moved to a new address.

  • 3, web extension changes (such as the application needs to change.php. Html or. SHTML). In this case, if you do not do redirection, the old address in the user favorites or search engine database can only let visitors get a 404 page error message, the traffic is lost; Moreover some registered multiple domain name website, also need to access these domain name users automatically jump to the main site through redirection.

3) When do I jump to 301 or 302?

When a website or webpage temporarily moves to a new location within 24 to 48 hours, it is necessary to perform a 302 jump. The scenario where 301 jump is used is that the previous website needs to be removed for some reason and then accessed to the new address, which is permanent.

To be clear, the following are the general scenarios for using the 301 redirect:

  • 1. Do not want to renew the expired domain name (or found a more suitable domain name for the website), want to change the domain name.

  • 2, in the search engine search results appeared without the WWW domain name, and with the WWW domain name is not included, this time you can use 301 redirect to tell the search engine our target domain name is which one.

The server processes the request

After all these steps, we finally send our HTTP request to the server. In fact, the previous redirect has already arrived at the server. So, how does the server handle our request?

After receiving A TCP packet on a fixed port, the backend processes the TCP connection, parses the HTTP protocol, and encapsulates the PACKET into an HTTP Request object for upper-layer use.

Some of the larger sites will send your requests to a reverse proxy server, because when the site is very heavily visited and the site is getting slower and slower, one server is no longer enough. The same application is deployed on multiple servers and requests from a large number of users are distributed across multiple machines. In this case, the client does not directly access a web application server through HTTP. Instead, Nginx first requests the application server, and Nginx then requests the application server, and then returns the result to the client. Here, Nginx acts as a reverse proxy server. It also has the advantage that if one of the servers goes down, users will not be affected as long as other servers are running properly.

Through Nginx’s reverse proxy, we get to the Web server, where the server-side script processes our request, accesses our database, fetches what we need to fetch, etc. Of course, this process involves many complex operations of the back-end script

Intellectual development

1) What is a reverse proxy?

The web administrator can add an Nginx in the middle. The client requests Nginx, and Nginx requests the application server, and then returns the result to the client. In this case, Nginx is the reverse proxy server.

The server returns an HTTP response

Naturally, this part corresponds to what backend engineers think of as HTTP. The back end starts by receiving TCP packets on a fixed port, which corresponds to the socket in the programming language. It processes TCP connections, parses HTTP, and encapsulates HTTP Request objects based on packet formats for upper-layer use. This part of the work is usually done by Web servers. I have used Tomcat, Jetty, Netty and so on.

The HTTP response packet also consists of three parts: the status code, the response header, and the response packet.

Status code

The status line is described by the protocol version, numeric status code, and corresponding status. Each element is separated by a space.

Format: http-version status-code reason-phrase CRLF

For example: HTTP/1.1 200 OK \r\n

— Protocol version: http1.0 or another version

— State description: The state description gives a short text description of the status code. For example, a status code of 200 is described as OK

— Status code: The status code consists of three digits. The first digit defines the category of the response and has five possible values. The following

1xx: information status code: indicates that the server has received the client request and the client can continue to send the request.

  • 100 Continue

  • 101 Switching Protocols

2xx: success status code, which indicates that the server has successfully received and processed the request.

  • 200 OK Indicates that the client request is successful

  • 204 No Content Is successful, but does not return the body of any entity

  • 206 Partial Content A Range request was successfully executed

3xx: redirection status code, indicating that the server requires client redirection.

  • 301 Moved Permanently redirected Permanently. The Location header of the response packet should contain the new URL of the resource

  • 302 Found Temporary redirection. The URL in the Location header of the response packet is used to locate the resource temporarily

  • 303 See Other The requested resource has another URI. The client should use the GET method to obtain the requested resource

  • 304 Not Modified The server content is Not updated and can be read directly from the browser cache

  • 307 Temporary Redirect Temporary redirection. Same meaning as 302 Found. 302 Do not allow POST to be changed to GET, but this is not always the case. 307 More browsers may follow this standard, but it depends on the browser implementation

4xx: indicates the client error status code, indicating that the client request contains invalid content.

  • 400 Bad Request Indicates that the client Request has syntax errors and cannot be understood by the server

  • 401 Unauthonzed indicates that the request is unauthorized. This status code must be used with the WWW-Authenticate header field

  • 403 Forbidden Indicates that the server receives a request but refuses to provide the service. The reason for not providing the service is usually given in the response body

  • 404 Not Found The requested resource does Not exist, for example, an incorrect URL was entered

5xx: indicates the server error status code. It indicates that an unexpected error occurs because the server fails to properly process requests from clients.

  • 500 Internel Server Error Indicates that an unexpected Error occurs on the Server. As a result, the client request cannot be completed

  • 503 Service Unavailable Indicates that the server cannot process requests from clients. The server may recover after a period of time

Response headers

Response header: Consists of a keyword/value pair on each line. The keyword and value are separated by a colon (:). Typical response headers are:

The response message

Contains the specific information we need, such as cookies, HTML,image, request data returned from the back end, and so on. It should be noted that there is a space between the response body and the response header, indicating that the information in the response header ends in the space. The following figure shows the request body captured by Fiddler, in the red box:

8. HTML parsing

How does the browser render the page on the screen before it accepts the entire HTML document in its entirety? Different browsers may have different parsing processes. Here we only introduce the WebKit rendering process. The following figure is the corresponding WebKit rendering process, which includes:

Parse the HTML to build the DOM tree -> build the Render tree -> layout the Render tree -> Draw the Render tree

As the browser parses the HTML file, it loads “top-down” and parses and renders it as it loads. In the process of parsing, if external resources are requested, such as images, external linked CSS, iconFONT, etc., the request process is asynchronous and will not affect the loading of HTML documents.

During parsing, the browser first parses the HTML file to build the DOM tree, and then parses the CSS file to build the render tree. Once the render tree is built, the browser lays out the render tree and draws it to the screen. The process is complicated and involves two concepts: reflow(reflow) and repain(redraw).

Each element in the DOM node exists in the form of a box model, which requires the browser to calculate its position and size, a process called relow. Once the location, size, and other properties of the box model, such as color and font, are determined, the browser begins to draw the content, a process called repain. For example: display: None triggers backflow, while visibility:hidden only triggers redraw.

Pages will inevitably experience reflow and Repain when they first load. Reflow and Repain processes can be very performance draining, especially on mobile devices, and can ruin the user experience, sometimes causing pages to stagnate. So we should reduce reflow and repain as little as possible.

When a JS file is encountered during the document loading process, the HTML document will suspend the render (load parsing render synchronization) thread, and not only wait for the js file loading in the document, but also wait for the parsing execution to complete, before resuming the HTML document rendering thread. Because JS may modify the DOM, the most classic document.write, this means that all subsequent resource downloads may not be necessary until JS execution is complete, which is the root cause of JS blocking subsequent resource downloads. So in our usual code, js is at the end of the HTML document.

JS parsing is done by javascript parsing engines in browsers, such as Google’s V8. JS is single-threaded, that is, only one thing can be done at a time, and all tasks need to be queued so that the first task can finish before the next one can start. However, there are some tasks that are time-consuming, such as IO reads and writes, so you need a mechanism to do the synchronous and asynchronous tasks first.

The execution mechanism of JS can be regarded as a main thread plus a task queue. Synchronous tasks are tasks that are executed on the main thread, and asynchronous tasks are tasks that are executed on the task queue. All synchronization tasks are executed on the main thread, forming an execution stack. An asynchronous task will place an event in the task queue when it has a result. When a script runs, it first runs the execution stack in sequence, then extracts the events from the task queue and runs the tasks in the task queue. This process is repeated constantly, so it is also called Event loop.

Disconnect

TCP four waves:

First wave: The Client sends a FIN to stop data transmission from the Client to the Server, and the Client enters the FIN_WAIT_1 state.

Second wave: After receiving a FIN, the Server sends an ACK to the Client. The ACK sequence number is +1 (the same as that for SYN, one FIN occupies one sequence number). The Server enters CLOSE_WAIT state.

Third wave: The Server sends a FIN to disable data transfer from the Server to the Client, and the Server enters the LAST_ACK state.

Fourth wave: After receiving the FIN, the Client enters the TIME_WAIT state and sends an ACK to the Server to confirm that the FIN number is +1. The Server enters the CLOSED state and waves four times.

The figure above can be generalized:

Client: brother, I have no data to transfer, let’s close the link

Server: Ok, I got it. Let me see if I have any more to send

Server: I don’t have any either. Close it

Client: Ok

Intellectual development

1) Why is it three handshakes to establish a connection and four waves to close it?

This is because in LISTEN state, the server receives a SYN packet for establishing a connection and sends the ACK and SYN packets to the client. And close the connection, when I received the other side of the FIN message just said to each other can no longer send data but also receives the data, the board may not all data are sent to each other, so their can immediately close, also can send some data to each other, then send the FIN message now agreed to close the connection to the other side, therefore, Your ACK and FIN are usually sent separately.

Reference links:

Segmentfault.com/a/119000000… www.cnblogs.com/yuanzhiguo/… www.cnblogs.com/duhuo/p/517… zhuanlan.zhihu.com/p/78677852