“This is the 28th day of my participation in the November Gwen Challenge. See details of the event: The Last Gwen Challenge 2021”.

preface

What happens when we type the url into the browser? Have you ever wondered what magic happens behind it? Today, I’m going to reveal the story behind the browser layer by layer.

This article starts with how to generate HTTP messages, then shows how the DNS server looks up IP addresses for us, and finally shows how the protocol stack finally sends the messages. It’s a long article, and you’ll have to hold back.

Generate AN HTTP request message

1.1 parse the URL

The web site, actually I should call it the URL. Most web addresses start with “http://”, but there are many others, such as “ftp://”, “file://”, and so on.

This text represents the access method used by the browser. HTTP is used to access the Web server, and FTP is used to access the FTP server.

In addition to the protocol method to be specified at the beginning, the entire URL also contains the domain name of the server and the path name of the file to be accessed, as shown in the following figure:

www.lab.glasscom.com indicates the address of the server to be accessed, and the path /dir/file1. HTML indicates the file file1. HTML in the path to access the server.

In daily life, we sometimes visit the website address without specifying the specific file name to visit, only a simple domain name, generally in this case, the server will set a default access path, such as index.html or default.htm.

This is the first step in the browser’s work, parsing the URL.

1.2 Basic working principles of HTTP

By parsing the URL, we already know where the destination is. Next, the browser will access the Web server through HTTP protocol. HTTP protocol is a very important knowledge, and I will write a special column to explain it in detail later.

The HTTP protocol defines the message content and steps for interaction between a client and a server. As shown in the figure above, the client sends a request message to the server. The request can have different operations. HTTP represents different operations through methods:

After receiving the request, the Web server completes its own processing and stores the processing result in the response message. The response message is sent back to the client, and then the client reads the result for display.

1.3 HTTP Request Message Generation

HTTP request messages are formatted, so the browser will generate the request message in the specified format.

  • Request line: The first line of the request message is called the request line, which contains three important parameters, the request method, that tell the server what to do. URL: specifies the IP address and path of the server to be accessed, and the protocol version. HTTP has different versions. You need to specify the HTTP version number.
  • Request header: The request header is used to place some additional details, such as the data type, language, compression format, date, and so on supported by the client.
  • Message body: This is where the data you need to send is stored.

Let me give you a real example to find out.

For example, we visit www.baidu.com

The first line is the request line. It is a GET request. The access path is/and the protocol version is 1.1

From the first line down is the request header, and since there is no data to send, there is no request body.

1.4 Receiving a Response After requesting a Message

The format of the response message is roughly the same as the request message, except for the first line.

The first line of the response message contains the request protocol, status code, and response phrase, which indicate whether the execution result of the request is a success or an error.

2. How to query the IP address

2.1 Basic KNOWLEDGE of IP Addresses

Once the HTTP message is generated, we send it to your Web server via the operating system. One more important thing to do before sending a message through the operating system is to check the IP address corresponding to the domain name.

Local area networks in the Internet are based on TCP/IP to design. A network is formed by small subnets connected by routers to form a larger network. In a network, all devices are assigned an address, such as the place you live in, called “room XX”. This number is assigned to the entire subnet, and the room is assigned to the computers in the subnet. The whole is called an IP address. The message sent by the sender is first forwarded to the nearest router through the hub of the subnet, and then the router sends the message to the next router according to the destination address. The process is repeated and finally reaches the destination.

2.2 Why Do DOMAIN Names and IP Addresses Share

Let’s start with two questions for you to think about:

  1. Why use a domain name when we can identify a target address directly from an IP address?
  2. Can I use the domain name directly to determine the access object?

To answer the first question, an IP address is a string of numbers, but think about it. If you typed in an IP address every time you visited a website, you’d probably have a hard time remembering it. Using a name is much easier and more recognizable.

Said the second question again, use the domain name to directly determine the access object, to bypass IP, from actual operation efficiency, this is not feasible, IP address length is 4 bytes, domain name even the shortest need dozens of bytes, the longer the bytes, routers process data, the longer the time will be the speed of the routers are limited, The current reality is that routers are almost saturated, so direct access is not possible, so is there a good way?

So by having people use names, by having routers use IP addresses, who’s going to establish the relationship between domain names and IP addresses, the bridge is DNS.

2.3 How Do I Query the IP Address

We can look up the IP address through the DNS server. Our computer has a DNS client to send requests to the DNS server. We call it a DNS resolver. The operation of querying IP addresses through DNS is domain name resolution.

DNS server details

3.1 Basic WORKFLOW of the DNS Server

Its basic job is to receive the query message from the client and then return the response information based on the message content.

Typically, a client query message consists of three parts:

  • Domain name: the name of the server

  • Class: The value of Class is always IN for the Internet

  • Record type: indicates the type of the domain name. If the type is A, the domain name corresponds to an IP address. If the type is MX, the domain name corresponds to A mail server.

The DNS server searches for the corresponding record in the mapping table between the domain name and IP address and returns the corresponding record.

3.2 How to Quickly Search for a Domain name Based on the Domain name Structure

The current number of domain names is too large to be stored in one DNS server. Therefore, the information is distributed among multiple DNS servers. These DNS servers cooperate with each other to find the final result.

Domain names in DNS are separated by periods, such as www.lab.glasscom.com. According to the organizational structure of the company, COM stands for group, Glass com stands for business division, and LAB stands for group. The parts at one level are called domains.

Information about a domain is stored on the DNS server as a whole. One server can store information about multiple domains.

How do we find out which DNS server manages the information about the server we want to access? First, we can register the IP address of the DNS server that manages the lower-level domain with its upper-level DNS server, and then register the IP address of the upper-level DNS server with the upper-level DNS server, and so on. What are the benefits of this? If we want to query www.lab.glasscom.com, we can use the DNS server of the com domain to look down to the DNS server of the glasscom.com domain, all the way down, and finally we can look up the IP address of the desired domain name.

In real life, there is a server to save the root domain, what is the root domain, is higher than COM domain, generally not reflected in the domain name, but it is real, it manages the information of all subordinate DNS servers, root domain server IP address only 13 in the world. These addresses do not change, so all DNS servers store these 13 IP addresses.

Let’s see how we find the target DNS server.

The client will first access the nearest DNS server, and since the nearest DNS server does not store the IP address corresponding to the domain name we need, we need to search from the top down, through the root domain server, until we find the target DNS server, so as to obtain the IP address we need.

Generally speaking, if it is the domain name information that we often query, the DNS server itself has the cache function, it will record the domain name you have queried before, so that when you request the domain name information in the cache, the DNS server will directly return the response, saving the trouble from the root domain every time, reducing the query time.

4. Delegate the protocol stack to send messages

4.1 Data Sending and receiving Process

Once we get the desired IP address through the DNS server, we can get the protocol stack inside the operating system to send messages to the destination IP.

Sending and receiving data is accomplished by using the Socket library, as shown in the following figure:

Before sending and receiving data, both the client and the server have to establish the pipe. The key of this pipe is the data outlet of the pipe, which is called the socket.

So we need to create a socket before we can set up the pipe. The server creates a socket, the client creates a socket, and then connects to the server. When all the data has been sent, the connection to the pipe is disconnected and the communication wipe is over.

We can divide this process into four stages:

  • Create a socket

  • Connect the pipe to the server socket

  • Sending and receiving data

  • Disconnect the pipe and delete the socket

    4.2 Creating a socket

    How is a socket created? Is actually call Socket Socket components in the library, create good after, protocol stack returns a descriptor, program received the descriptor stored in memory, the descriptor is used to identify the different Socket, because the browser may exist multiple requests, you will create multiple Socket, so there must be a marker to identify.

    For example, when many people check in at the same time, in order to ensure that they stay in different rooms, each person will be given a room card as a unique identification, so that the attendant can find the corresponding person according to the room card.

    4.3 How do I Connect Pipes

    After the Socket is created, we need to connect with the server. Here, we call the CONNECT component in the Socket library to complete the call. The connect component needs to pass the descriptor, server IP address and port number of these three parameters.

    We already know the first two parameters. What does this port number do?

    Imagine that the IP address allows us to find the corresponding server, but the server may deploy multiple applications, such as deploying two Web services. We cannot identify the server based on the IP alone, so we need to add the port number to find the specific service. You might say, well, don’t we have a descriptor, and this is the only one? This is not possible because the server has no way of knowing this descriptor.

    4.4 Sending and Receiving Messages and Data is complete

    Sending a message is as simple as sending data into a socket and sending it to the socket of the other party. This process is also accomplished through the write program component of the Socket library.

    When the message returns, it is received through the READ component of the Socket library.

    When the server has sent the response message, it actively performs the disconnect operation by calling the Close component. When the client receives the data, it also calls CLOSE to disconnect.

    conclusion

    When the browser enters the URL, the browser will first parse the URL, and then we will generate HTTP request messages and introduce the basic concepts of HTTP protocol. Because we access through the domain name, we need to use DNS to get the IP address of the target object. Finally, we introduced the protocol stack (TCP IP) to actually send the message to the server, and complete the data reception.

    A website request behind the knowledge involved in many aspects, do know, know why, can really learn more valuable knowledge, pay attention to small eggs, take you in the ocean of technology roam, progress together, grow together.