The HTTP protocol is based on TCP connections. HTTP is a protocol that allows the browser to obtain resources from the server. It is the basis of the Web. Usually, the browser initiates requests to obtain different types of files, such as HTML files, CSS files, JavaScript files, images, and videos. In addition, HTTP is also the most widely used protocol in browsers, so it’s important to understand HTTP in depth to learn about browsers.

The browser initiates an HTTP request process

Enter a web address in your browser’s address bar: juejin.cn/user/157415… The browser then completes a sequence of actions, as shown below:

1. Build the request

First, the browser builds the request line information, and once it’s built, the browser is ready to make the web request.

GET /index.html HTTP1.1
Copy the code

2. Search the cache

Before actually making a web request, the browser looks in the browser cache to see if there is a file to request. Among them, browser caching is a technique for saving a copy of a resource locally for immediate use on the next request.

When the browser discovers that the requested resource already has a copy in the browser cache, it intercepts the request, returns a copy of the resource, and ends the request without going to the source server for a new download. The benefits of this are:

  • Alleviating server-side stress and improving performance (less time to acquire resources);
  • For web sites, caching is an important part of fast resource loading.

Of course, if the cache lookup fails, it enters the network request process.

3. Prepare IP addresses and ports

Before we look at network requests, we need to look at the relationship between HTTP and TCP. Because browser usageHTTP is used as an application-layer protocolIs used to encapsulate the request’s text information. And useTCP/IP sends it to the network as a transport layer protocol, so before HTTP work can begin, the browser needs to establish a connection with the server via TCP. That is to say,HTTP content is realized through the transmission data phase of TCP, you can better understand the relationship between the two with the following figure.Here’s a list of questions you can think about:

  • What is the first step in an HTTP web request? As shown in the figure above, the TCP connection is established with the server.
  • Do we have the information to establish a connection? We know that the first step in establishing a TCP connection is to prepare the IP address and port number.
  • How do I get the IP address and port number? It depends on what we have, we have a URL, can we use the URL to get IP and port information?

The packets are sent to the recipients via IP addresses. Because IP addresses are numeric identification, for example, the IP address of a website is 39.106.233.176, which is difficult to remember, but it is much easier to remember using the domain name of the website (time.geekbang.org), so a service has emerged based on this requirement, which is responsible for mapping domain names to IP addresses. The System that maps Domain names to IP addresses is called the Domain Name System (DNS).

So, along the way, you’ll see that in the first step the browser will ask DNS to return the IP of the domain name. Of course, the browser also provides DNS data caching service. If a domain name has been resolved, the browser will cache the result for the next query, which also reduces the network request.

Once you have the IP, the next step is to get the port number. In general, HTTP defaults to port 80 if the URL does not specify a port number.

4. Wait for the TCP queue

Now that you have the port and IP address ready, can you set up a TCP connection?

Again, the answer is “no”. Chrome has a mechanism that allows you to establish a maximum of six TCP connections under the same domain name. If 10 requests occur at the same time under the same domain name, four of the requests will be queued until the ongoing requests are completed.

Of course, if the number of current requests is less than 6, the next step is to establish a TCP connection.

5. Establish the TCP connection

When the queue is over, you can finally happily shake hands with the server, with the browser establishing a connection via TCP before the HTTP work can begin.

6. Send an HTTP request

Once the TCP connection is established, the browser can communicate with the server. The data in HTTP is transferred during this communication.First the browser sends a message to the serverThe request lineIt includesRequest method, Uniform Resource Identifier (URI), and HTTP version protocol.

Sending a request line tells the server what resources the browser needs. The most common request method is Get. For example, typing the domain name directly into the browser address bar tells the server to Get its resources.

Another common request method is POST, which is used to send some data to the server. For example, when you log in to a website, you need to send the user information to the server through POST. If the POST method is used, the browser also prepares data for the server, which is sent in the request body.

After the browser sends a request line command, it sends additional information in the form of a request header that tells the server the basics of the browser. For example, it contains the operating system used by the browser, the browser kernel and other information, as well as the domain name information of the current request, Cookie information of the browser, and so on.

The server side handles the HTTP request flow

After a long time, the HTTP request was finally delivered to the server. The server then prepares the content based on the browser’s request information.

1. Return the request

Once the server has finished processing, the data can be returned to the browser.First the server returns a response line, including the protocol version and status code.

However, not all requests can be handled by the server, so what about some messages that cannot be processed or processed incorrectly? The server tells the browser what it has done with the status code on the request line, for example:

  • The most commonly used status code is 200, indicating that the processing is successful.
  • If the page is not found, a 404 is returned.

There are many types of status codes, and I won’t go over them here. There are many online materials that you can search and learn by yourself.

Then, just as the browser sends the request header along with the request, the server sends the response header to the browser along with the response. The response header contains information about the server itself, such as when the server generated the returned data, the returned data type (JSON, HTML, streaming media, and so on), and the Cookie that the server wants to save on the client.

After sending the response header, the server can continue sending the data in the response body, which usually contains the actual content of the HTML.

This is how the server responds to the browser.

2. Disconnect the device

Normally, once the server returns the request data to the client, it closes the TCP connection. But if the browser or server adds the following header:

Connection:Keep-Alive 
Copy the code

The TCP connection will remain open after being sent, so the browser can continue sending requests over the same TCP connection. Maintaining a TCP connection saves the time required to establish a connection for the next request and speeds up resource loading. For example, the images embedded in a Web page are all from the same Web site, and if you initialize a persistent connection, you can reuse that connection to request other resources without having to re-establish a new TCP connection.

3. The redirection

At this point, the request process seems to be coming to an end, but there is one more thing you need to know. For example, when you open Geekbang.org in your browser, you will find that the final page is www.geekbang.org.

The two urls are different because a redirection operation is involved. You can also use curl to see what geekbang.org will return.

Enter the following command on the console:

curl -I geekbang.org
Copy the code

Note that the input parameter is -i, which is different from -i. -I means that only the data of the response header and the response row need to be retrieved, but the data of the response body need not be retrieved. Finally, the returned data is as follows:As you can see, the response line returns a status code 301, which tells the browser that I need to redirect to another url, and that url is contained in the Location field of the response header. Then the browser retrieves the address in the Location field. And use that address to navigate again, this is a complete redirection of the execution process. Which explains why geekbang.org was typed in, but what was openedwww.geekbang.org.

But don’t take the jump for granted. If you open 12306.cn, you will find that the site is not open. This is because 12306’s server doesn’t handle jumps, so you have to manually enter the full www.12306.cn to open the page.

q&a

1. Why do many sites open quickly the second time?

If the second page is opened quickly, the main reason is that some time-consuming data was cached during the first page loading.

So what data is cached? As you can see from the core request path above, the DNS cache and the page resource cache are two pieces of data that are cached by the browser. Among them, DNS cache is relatively simple, it is mainly in the browser local IP and domain name associated, not too much analysis here.

Let’s focus on the browser resource cache. Here’s how it works:First, how does the server get the browser to cache data?

As you can see from the first request above, when the server returns an HTTP response header to the browser, the browser uses the cache-Control field in the response header to set whether or not to Cache the resource. Usually, we also need to set a Cache expiration time for this resource, which is set by the max-age parameter in cache-control, such as 2000 seconds.

Cache-Control:Max-age=2000
Copy the code

This means that the cached resource will be returned to the browser if it is requested again if it has not expired.

But If the cache expires, browsers will continue to make web requests and put the following in the HTTP headers: if-none-match :” 4f80F – 13C-3a1XB12a “. When the server receives the request headers, it can judge whether the requested resources have been updated based on the if-none-match value.

  • If there is no update, the 304 status code is returned, which is equivalent to the server telling the browser, “This cache can continue to be used and I won’t send you the data again this time.”
  • If the resource is updated, the server returns the latest resource directly to the browser.

There are a lot of details about caching, but you can see HTTP caching for more details.

In short, many sites are able to get second visits in seconds because they cache a lot of their resources locally. The browser cache saves time by responding to requests directly with a local copy rather than generating actual network requests. DNS data is also cached by the browser, which eliminates DNS queries.

2. How is the login status maintained?

From the introduction above, you have seen how caching works. Let’s take a look at how login status is maintained.

  • The user opens the login page, enters the user name and password in the login box, and clicks ok. Clicking the button triggers the page script to generate user login information, which is then submitted to the server by calling the POST method.
  • After receiving the message from the browser, the server queries the background to verify that the user’s login information is correct. If so, it generates a string representing the user’s identity, writes the string to the set-cookie field in the response header, as shown below, and sends the response back to the browser.

Set-Cookie: UID=3431uad;

  • After receiving the response header from the server, the browser parses the response header. If the response header contains a set-cookie field, the browser saves this field locally. For example, keep UID=3431uad locally.
  • When the user visits again, the browser will make an HTTP request, but before making the request, the browser will read the saved Cookie data and write the data into the Cookie field in the request header (as shown below), and then the browser will send the request to the server.

Cookie: UID=3431uad;

  • After receiving the HTTP request header data, the server will search for the “Cookie” field information in the request header. When it finds the information containing UID= 3431UAD, the server queries the background, determines that the user is logged in, and then generates the page data containing the user information, and sends the generated data to the browser.
  • After the browser receives the page data containing the current user, it can correctly display the status information of the user login.

Ok, this process lets you know that the state of the browser page is achieved by using cookies. The Cookie process can be referred to the following figure:Simply put, if the response header sent by the server has a set-cookie field in it, the browser keeps the contents of that field locally. The next time a client sends a request to the server, the client automatically adds the Cookie value to the request header and then sends the request. After discovering the Cookie sent by the client, the server will check which client sent the connection request, and then compare the records on the server to obtain the status information of the user.

conclusion

As you can see from the figure, the HTTP request in the browser goes through the following eight stages from initiation to termination: build the request, find the cache, prepare the IP and port, wait for the TCP queue, establish the TCP connection, initiate the HTTP request, the server processes the request, the server returns the request, and disconnect the connection.

Then I solved two common problems with HTTP request paths, one involving Cache flow and the other how to use cookies for state management.

The resources

Working principle and practice of browser