The main steps are as follows:

  • 1. URL parsing
  • 2. DNS query
  • 3. Send an HTTP request to establish a TCP connection
  • 4. The server processes the request and responds
  • 5. The browser receives the response, parses it, and renders it to the page

1 URL parsing

URL(Universal Resource Locator) A Universal Resource Locator (URL) that indicates the address of a Resource. It consists of the following components:

    1. Transport protocol (e.g. HTTPS)
    1. The domain name
    1. Port (not displayed by default, followed by the domain name, starting with:, HTTP default 80, HTTPS default 443)
    1. Virtual directory
    1. The file name
    1. Parameter (from? Start, end at #, multiple arguments are concatenated with &)
    1. Anchor (after #)

For example, the Bilibili homepage www.bilibili.com/anime/?spm_…

The transport protocol is HTTPS, the domain name is www.bilibili.com, and then the virtual directory /anime/, followed by a spm_ID_FROM parameter

Parse the URL

  • A character encoding
  • Completion: For example, if you enter bilibili.com and press enter, it will become www.bilibili.com
  • HSTS: HSTS is used to force the client to access the page using HTTPS due to security risks. For example, bilibili.com becomes bilibili.com after you enter bilibili.com

Check the cache

  • There is the cache
    • No expiration, read cache returns cache
    • Expired, with an identifier to the server to ask for updates
      • No update, return 304, continue to use cache, read cache return cache
      • Update, override returns resource and cache id, returns 200, saves locally, loads page
  • No cache, request to the server, return the request result and cache id, store it locally, load the page

2 the DNS query

DNS: indicates the Domain Name System (DNS)

A domain name is a mapping to an IP address. IP addresses are hard to remember, while domain names are relatively easy to remember. However, datagrams cannot be transmitted by domain names because the IP address length is fixed (ipv4 32-bit ipv6 128-bit), but the domain name length is not fixed. Therefore, you need to convert domain names into IP addresses for communication between hosts.

DNS, the domain name system of the Internet, is designed as an online distributed database system and adopts the client server mode. DNS allows most of the names to be resolved locally, and only a small amount of the resolution needs to be communicated over the Internet. Therefore, DNS is very efficient. Because DNS is distributed, even if a single computer has a problem, it does not affect the normal operation of the whole DNS.

Domain name server program: complete domain name to IP address resolution program

Domain name server: the machine that runs the domain name server program

Domain name: consists of a sequence of labels separated by periods (.).

  • The label consists of letters and numbers, and the hyphen -.
  • No more than 63 characters per label (preferably no more than 12 for memory)
  • Case insensitive
  • The lowest level is written at the far left, and the highest level is written at the far right in ascending order
  • The length of a domain name consisting of multiple labels cannot exceed 255 characters
  • DNS does not specify how many levels of a domain name, does not specify what each level of a domain name represents

2.1 Domain name Level Classification

From high level to low level, from right to left, into top-level domain name, level 2 domain name, level 3 domain name, level 4 domain name… Also mentioned the DNS for the domain name in front of the series and each level domain specific meaning do not limit, so domain is relatively free, once a unit has a domain name, it can decide whether to divide the following child domain does not needs to be approved by the supervising authority, some common can refer to the following mind map.

The domain name system can be represented as a tree, the top is the root, the root is the top-level domain names below, top-level domain names here are their respective secondary domain name, secondary domain is level 3 domain names, tertiary domain and four domain name… It’s organized by organization, not by region

2.2 Domain Name Server

May, in accordance with the domain name system of tree structure, each level of the domain name is set to a corresponding domain name server), but this makes the number of domain name server too much, make the domain name, lower operation efficiency, and therefore put forward the way of divided area, a server management scope is called area, can be a KeZi tree corresponding to the domain name system, all nodes are connected in the area.

The DNS server is managed by area. The area must be smaller than or equal to the domain

According to the functions of DNS servers, DNS servers can be divided into:

  • Root DNS: The top-level DNS server that knows the domain names and IP addresses of all top-level DNS servers
  • Top-level domain name server: Manages all secondary domain names registered under this top-level domain name server
  • Permission domain name server: Domain name server responsible for a district
  • Local DNS server: when a host sends a DNS query request to a user, the local DNS server first sends the query request to the user. If the host to be queried belongs to the same LOCAL ISP, the local DNS server can immediately convert the host name to an IP address

2.3 DNS Query Methods

  • The query from the host to the local domain name service is usually recursive. If the local DNS server doesn’t know the IP address of the domain name being queried, the local DNS server says, it’s on me, I’ll tell you when I find it, and then the local DNS server goes to the other root DNS server and waits for the result. A recursive query returns either an IP address or an error saying it was not found
  • The local DNS server is used to query information from the root DNS serverIterative query
    • After receiving the iterative query request packet from the local DNS server, the root DNS server either provides the IP address to be queried or tells the local DNS server which TOP-LEVEL DNS server (IP address) to query.
    • After receiving a query request from the local DNS server, the TOP-LEVEL DNS server either directly provides the IP address to query or tells the local DNS server which permission DNS server (IP address) to query next.
    • Step by step, the local DNS server queries the IP address requested by the host and returns it to the host

    So it’s kind of like, you know, the local DNS server goes to the root DNS server, the root DNS server says I don’t know, you have to go to the top-level DNS server, the local DNS server goes to the top-level DNS server, the top-level DNS server says he doesn’t know, you have to go to… In this way, each step of the query will be done by the local DNS server.

Pay attention to

  • Host to the local domain name server query is generally recursive query, and the local domain name server how to check and it has no relationship
  • The local DNS server queries the root DNS server iteratively or recursively. Depending on the initial Settings, the two methods consume the same number of UDP packets

The cache

To improve the query efficiency, lighten the load of the root DNS server and reduce the number of DNS query packets on the Internet, DNS servers widely use cache to cache the recently queried domain names and the information records about where to obtain the domain name mapping. For example, if you want to query domain name y.abc.com and the local DNS server has the query record, you do not need to query the domain name again. Due to the timeliness of the domain name, a certain interval must be set to discard expired items.

The host also has a cache. At first, the host downloads the DNS server from the local domain name server, and then its own query is recorded to the local cache for maintenance, thus reducing the number of DNS queries

2.4 Summary of DNS Query Procedures

  • Querying the Browser cache
  • Check whether the IP address of the domain name exists in the cache of the host. If yes, do not perform DNS query
  • Checking the router cache
  • Perform recursive queries to the local domain name server
    • The local DNS server checks whether the IP address of the domain name exists in its cache and returns the IP address directly to the host
    • The local DNS server does not exist in the cache. The local DNS server iterates (or recursively) the query to the root DNS server and returns the result to the host (returns the IP address or does not exist).

3 a TCP connection

TCP has the following features:

  • TCP is a connection-oriented transport layer protocol. Before using TCP, a TCP connection must be established and released after data transmission is complete
  • Each TCP connection can have only two endpoints. TCP regards connections as the most basic abstraction. Each TCP connection is uniquely determined by two endpoints (sockets) at both ends of the communication. TCP connections ::={socket1,socket2}={(IP1:port),(IP2:port)}
  • TCP provides reliably delivered services. Data transmitted over a TCP connection is error-free, not lost, not duplicated, and arrives sequentially
  • TCP provides full duplex communication. TCP allows applications on both sides of the communication to send data at any time.
  • TCP is byte stream oriented. TCP sees the data handed over by an application as just a series of unstructured byte streams (a sequence of bytes flowing into or out of a process). TCP does not know the meaning of the byte streams being sent.

In the TCP/IP architecture, data from the sender is encapsulated at each layer, and data from the receiver is split at each layer.

  • Application layer HTTP data
  • Transport layer TCP header + HTTP data
  • Network layer IP header + TCP header + HTTP data
  • Network interface layer: Ethernet header + IP header + TCP header + HTTP data

3.1 Establishing a TCP Connection: Three-packet handshake

3.1.1 Related Fields

Before establishing a TCP connection, you need to know related fields. The header of a TCP packet contains a fixed 20 bytes plus an optional 4n bytes

  • Serial number: SEQ, also known as the segment number, is 4 bytes in the range of [0,2^ 32-1]. It is mod2^32 and returns to 0 after 2^32-1. It is the sequential number of each byte of the byte stream transmitted over a TCP connection
  • Acknowledgement number: ACK, consisting of 4 bytes. It is the sequence number of the first data byte in the expected next packet segment. If the acknowledgement number is N, it indicates that the data with the sequence number up to N-1 has been correctly received
  • ACKnowledgment ACK :(ACKnowledgment) is 1 bit, and the ACKnowledgment number is valid only when the ACK is set to 1. After the connection is established, all ACK packets are set to 1
  • SYN: (SYNchronization) A bit used to synchronize signals when a connection is established
    • SYN = 1, ACK = 0, indicating that this is a connection request packet segment
    • SYN = 1, ACK = 1: indicates that the peer accepts the connection

3.1.2 Establishment process

Assume that client A and server B are both in CLOSED state, then establish A TCP connection between AB as follows:

  • B creates the transfer control block TCB and prepares to receive the connection request from the customer process. Then, B is in LISTEN state and waits for the connection request from the customer
  • User A creates transport control block TCB, intends to establish A TCP connection with USER B, and sends A request packet to user B
    • SYN = 1 in header, select sequence number seq = x
    • This packet segment carries no data but consumes an ordinal number
    • After sending this packet segment, A enters the syn-sent state
  • After receiving the connection request packet, user B sends A confirmation message to user A if user B agrees to establish A connection
    • SYN and ACK are set to 1 in the packet segment
    • Ack = x + 1 (Acknowledge receipt of the packet whose serial number is A before X + 1 and hope to receive the next packet whose serial number is X + 1)
    • Also determine an initial serial number for yourself, seq=y
    • This message segment also carries no data, but consumes an ordinal number.
    • B enters the SYN-RCVD state after sending the acknowledgement packet
  • After A receives B’s confirmation, it also needs to give B its confirmation
    • Confirm that the ACK of the packet segment is set to 1
    • Acknowledgement id ACK = Y + 1 (Confirm the receipt of packets whose SERIAL number of B is earlier than Y + 1 and hope that the serial number of the next packet is Y + 1)
    • Serial number seq = x + 1 (A’s connection request message seq = x consumes one serial number)
    • Seq = x + 1 is not consumed if the packet does not carry data
    • After sending this confirmation, A enters the ESTABLISHED state
  • B also enters the ESTABLISHED state after receiving A’s confirmation

Why does A acknowledge B’s message and then send back an acknowledgement?

An error occurs in case an invalid connection request segment is suddenly sent to B.

Scenario Assumption Normal situation:

  • User A sends A connection request, but the packet is not sent to user B due to some reasons
  • If B does not receive it, the connection is not confirmed, and A will not receive B’s confirmation
  • A Resends the connection request
  • B received it and sent confirmation to A
  • After receiving B’s acknowledgement, A sends an acknowledgement to B, and the two establish A connection
  • The connection is established. Data transfer is complete. The connection is disconnected
  • The lost message of A requesting to establish A connection is sent to B
  • User B confirms the connection with User A
  • If user A does not reply to user B, the request is an error
  • The connection will not be established until B receives A’s confirmation

If A TCP connection is established only by A requesting B and B confirming it to A, the situation will become A lost request and A establishes A TCP connection between AB. A does not know about this, and B waits for A to send data, and holds the TCP connection for A, and B’s resources will be wasted

3.2 Release of TCP Connection — 4 waves

Terminate FIN: occupies 1 bit, used to release a connection. FIN = 1 Indicates that the packet sender has finished sending data and needs to release the transport connection

After the connection between client A and server B is ESTABLISHED, both client A and server B are in ESTABLISHED state. After data transmission, client A wants to disconnect the connection. The process is as follows:

  • A sends A connection release packet to B and stops sending data to B
    • If FIN is set to 1, the packet is a connection release packet
    • Seq is set to u, where u is the sequence number + 1 of the last byte sent before A
    • FIN packets consume a sequence number even if they do not carry data
    • A Enters the FIN-WaIT-1 state
  • User B receives the connection release packet from user A and confirms the connection release packet
    • ACK set to 1
    • Ack = u + 1 (seq = u for confirming receipt of A)
    • Seq = v, where v is the sequence number + 1 of the last byte before B
    • B Enters close-wait state
    • A has no data to send to B, but B can send some data to A. The TCP connection is half-closed
  • After receiving the acknowledgement from USER B, user A enters the FIN-WaIT-2 state and waits for user B to transmit the rest data before sending the connection release packet
  • After sending all data segments, USER B sends A connection release segment to user A
    • If FIN is set to 1, USER B and user A need to perform two operations on each other
    • If ACK is set to 1, the confirmation number is valid
    • Ack = u + 1 (A does not send another packet segment to B, so it is still the original confirmation number)
    • Seq = w (B sends some data after confirming A connection release message, the last byte + 1 = w)
    • B Enters the last-ACK state
  • User A acknowledges the connection release packet sent by user B
    • ACK= 1
    • ack = w + 1
    • seq = u + 1
    • User A enters the time-wait state and enters the CLOSED state after the TIME set by the TIME WAIT is 2MSL
  • B enters the CLOSED state after receiving the confirmation from A

It can be seen that B will enter the CLOSED state slightly earlier than A. In addition to the time-to-wait timer, TCP also designs a keepalive timer. In case the client machine fails, data is no longer transmitted but the connection is tied up. The server resets the keepalive timer (usually 2h) every time it receives data from the client. If no data is received within this period, the server sends probe packets to the client once every 75 seconds for 10 consecutive times. The server determines that the client is faulty and closes the connection. Therefore, the long-connected client must periodically send heartbeat packets to the server.

Why does A wait 2MSL after receiving and confirming the connection release message from B?

Maximum Segment Lifetime (MSL) : indicates the maximum Segment Lifetime. The value is set to 2 minutes in RFC793, but can be set to a smaller value based on actual conditions

  • In order for the last ACK sent by A to B to reach B, B cannot enter the CLOSED state if this acknowledgement is lost
    • The segment of the acknowledgement packet sent by USER A to USER B is lost, and user B in the last-ACK state cannot receive the acknowledgement packet from user A
    • After the MSL, USER B resends the FIN + ACK packet to user A
    • After another MSL, A will receive A retransmission from B, and A also knows that its confirmation is lost
    • A resends the confirmation to B, then repeats the story of yesterday, and the 2MSL timer is reset
  • Prevent invalid connection request packets. As mentioned above, after A sends the confirmation packet, all the packets generated during the connection duration disappear from the network after 2MSL. The next time a new connection is established, there will be no old connection request segment

4. HTTP request and response

Note that the HTTP request comes before the TCP connection.

After DNS gets the destination IP address, the browser starts constructing HTTP packets, including:

  • Request Header: Request method, target address, protocol followed, etc. The browser can only send GET and POST methods. The GET method is used to open the web page
  • Request body

After receiving an HTTP packet, the server encapsulates the packet and sends it. After receiving an HTTP packet, the server splits the packet and responds

HTTTP Request Type (Action)

  • Only GET can be used in HTTP/0.9
  • HTTP/1.0 defines three request methods: GET, POST, and HEAD
  • Five request methods have been added to HTTP/1.1: OPTIONS, PUT, DELETE, TRACE, and CONNECT

The characteristics of each method are as follows:

  • GET: sends a request to a specific resource
  • POST: Submits data to a specified resource for processing requests. POST data is contained in the request body. POST requests may result in the creation of new resources or the modification of existing resources
  • HEAD: Similar to a GET request, but returns a response with no specific content, which is used to retrieve the header
  • OPTIONS: Tests server performance
  • PUT: uploads the latest content to the specified resource location
  • DELETE: requests the server to DELETE the identified resource
  • TRACE: displays the request received by the server for testing
  • CONNECT: Allows the server to access other web pages instead of the host and return data

The server processes the request

  • After listening for an HTTP request, a child process is started to process the request
  • Parse HTTP requests (request method, domain name, path) and verify (whether virtual hosts are configured, whether virtual hosts receive this method, and the user’s permission to use this method 1)
  • If yes, 301 is returned. The browser resends the HTTP request based on the response
  • The URL rewrite
    • If the requested file exists, return it directly
    • Otherwise, the server rewrites the request to a REST-style URL as a rule and invokes the corresponding type of dynamic interpreter to handle the request based on the scripting decisions of the dynamic language

5 The browser receives the response, parses it, and renders the page

There are too many common status codes for HTTP response messages (100-101, 200-206, 300-307, 400-417, 500-505)

  • 200: OK Succeeded
  • 400: BadRequest Error request
  • 401: Unauthorized
  • 403: Forbidden Forbidden resource unavailable
  • 404: Not Found The resource in the specified location was Not Found
  • 500: Internal Server Error Indicates an Internal Server Error
  • 502: Bad Gateway Indicates an error Gateway. The server accesses the next server to complete the request and returns an invalid reply

After receiving the response resource, the browser analyzes the response resource and performs operations based on the status code of the response header. If the resource is compressed, decompress the resource, cache the resource, and parse the response content based on the MIME type of the resource.

After parsing, the content will be rendered to the page, rendering will be divided into HTML, Style, Script three parts, each browser kernel rendering process is similar.