HTTP / 0.9

HTTP/0.9 was proposed in 1991, mainly for academic communication, the requirements are very simple – to transfer HTML hypertext content between networks, so it is called hypertext Transfer protocol. Overall, its implementation is also very simple, using a request-response-based pattern, from the client to send the request, the server to return the data.

Complete request Flow

  • Because HTTP is based on TCP, the client needs to establish a TCP connection with the server based on the IP address and port. The connection establishment process is the TCP three-way handshake process.
  • After the connection is established, a GET request line is sent, such as GET /index.html to GET index.html.
  • After receiving the requested information, the server reads the corresponding HTML file and returns the data to the client as a stream of ASCII characters.
  • After the HTML document is transferred, disconnect the connection.

The characteristics of

  • The first is that there is only one request line, and no HTTP request header or body, because only one request line is required to fully express the client’s requirements.
  • The second is that the server does not return header information either, because the server does not need to tell the client much information, just return data.
  • The third is that the content of the returned file is transmitted as an ASCII character stream, since it is an HTML file, using ASCII bytecode is most appropriate.

HTTP / 1.0

HTTP/0.9 has a number of problems, such as the following:

  • Only HTML files are supported. JS, CSS, fonts, images, and videos cannot be transferred.
  • File transfer format is limited to ASCII, can not output files of other types of encoding;
  • There are only request lines and too little information is transmitted to the server;
  • Only respond to request data, no additional data can be transferred to the browser.

So it was no longer enough for the needs of the time, and then HTTP/1.0 came along, and it brought these:

  • New request headers and request bodies are added to transmit more information to the server, such as the following request header fields: Accept file type, accept-encoding Encoding, Accept-Charset character Encoding, accept-language

    Accept: text/html
Accept-Encoding: gzip, deflate, br
Accept-Charset: ISO-8859-1,utf-8
Accept-Language: zh-CN,zh
Copy the code

  • A new user-Agent field is added to the request header for the server to collect client information:

    The user-agent: Mozilla / 5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36Copy the code

  • Content-type tells the browser what Type of file the server is returning and what Encoding is used:

    Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Copy the code

  • Added the response line status code to inform the browser of the current request status, for example, 200 indicates that the request is successful:

    HTTP / 1.1 200 OKCopy the code

  • The new cache mechanism is used to cache downloaded resources, reducing the pressure on the server.

In terms of building the request flow, the biggest difference between HTTP/1.0 and HTTP/0.9 is the addition of many new fields in the request and response time for communication between the browser and the server.

HTTP / 1.1

Although HTTP/1.0 was already capable of transferring different types of files, it still had disadvantages, such as the following phases for each HTTP request:

  • Establish a TCP connection.
  • The HTTP request.
  • The HTTP response.
  • The TCP connection is disconnected.

HTTP/1.0 sends multiple requests for the same domain name:

You can see that each request requires the re-establishment of the TCP connection and disconnection operation, which undoubtedly increases the network overhead and delays the page display.

HTTP/1.1 adds a Connection field to the request header: to provide a persistent Connection to TCP ** :

Connection: keep-alive
Copy the code

Persistent connections are enabled by default. That is, the browser supports six TCP persistent connections for the same domain name by default. After persistent connections are enabled, multiple requests under the same domain name will be sent as follows:

The Host field is added in HTTP/1.1 to support virtual hosts

Host: bubuzou.com
Copy the code

Virtual host: A physical machine is bound to multiple virtual hosts. Each virtual host has its own domain name. These domain names share an IP address.

HTTP/1.1 supports dynamic content through the introduction of the Chunk transfer mechanism: the server will divide the data into several arbitrarily sized data blocks, each data block will be sent with the length of the previous data block, and finally use a zero-length block as a signal that the data is sent.

HTTP/1.1 also introduced client-side cookies and security mechanisms

HTTP/2

We know that HTTP/1.1 has made a number of optimizations for network efficiency, most notably in the following three ways:

  • Added persistent connections.
  • The browser maintains a maximum of six TCP persistent connections for each domain name.
  • Using CDN to achieve domain name sharding mechanism.

Problems with HTTP/1.1 remain

Although HTTP/1.1 adopts many strategies to optimize the loading speed of resources, and has achieved certain effects, HTTP/1.1 is not ideal for bandwidth utilization, which is also a core problem of HTTP/1.1.

Bandwidth is the maximum number of bytes that can be sent or received per second. We call the maximum number of bytes per second that can be sent upstream bandwidth and the maximum number of bytes per second that can be received downstream bandwidth.

HTTP/1.1 has poor bandwidth utilization because it is difficult to use up bandwidth. For example, we often say that 100M bandwidth, the actual download speed can reach 12.5m /S, while using HTTP/1.1, maybe when loading page resources can only use up to 2.5m /S, it is difficult to use up 12.5m.

This problem is mainly caused by three problems:

The first reason is the slow start of TCP

Once a TCP connection is established, it enters the state of sending data. At first, the TCP protocol uses a very slow speed to send data, and then speeds up until the speed of sending data reaches an ideal state. This process is called slow start. This process can be thought of as a car starting up, slowly at first, and accelerating faster as it picks up speed.

The reason why slow start will bring performance problems is that some key resource files commonly used in the page are not large, such as HTML files, CSS files and JavaScript files, usually these files after the TCP connection is established, the request will be initiated, but this process is slow start. So it took much longer than normal, delaying the precious first rendering of the page.

Second, multiple TCP connections are enabled at the same time, and these connections compete for fixed bandwidth

You can imagine that the system establishes multiple TCP connections at the same time. When the bandwidth is sufficient, the speed of sending or receiving each connection slowly increases. When bandwidth is low, these TCP connections slow down the speed of sending and receiving.

Some TCP connections download key resources, such as CSS files and JavaScript files, while some TCP connections download common resource files, such as pictures and videos. However, multiple TCP connections cannot negotiate which key resources can be downloaded first. This may affect the download speed of critical resources.

The third reason is HTTP/1.1 queue header blocking

We know that with persistent connections in HTTP/1.1, although it is possible to share a TCP pipe, only one request can be processed at a time in a pipe, and other requests can be blocked until the current one is completed. This means we can’t just send requests and receive content in one pipe.

This is a very serious problem, because there are many factors blocking requests, and are some uncertain factors, if some requests are blocked for 5 seconds, then the subsequent queued requests will be delayed for 5 seconds, in this waiting process, bandwidth, CPU are wasted.

HTTP/2 multiplexing

In order to solve the problems in HTTP/1.1, HTTP/2 adopted the most disruptive scheme: the multiplexing mechanism.

What is HTTP/2 multiplexing

HTTP/2 multiplexing mechanism in simple terms, the browser for the same domain resources, only establish a TCP connection channel, all requests for this domain name are completed in this channel;

In addition, data will no longer be transmitted in text format, but will be broken up into smaller streams and frames and encoded in binary format. In a TCP connection channel, any number of two-way data flows are supported. These data flows are parallel, out of order, and do not interfere with each other. The data transmitted in a data stream is a binary frame, which is the smallest unit of data transmission in HTTP/2. Frames in a stream are transmitted sequentially and in parallel, so there is no need to wait sequentially.

What problem was solved

Because only one TCP connection is used, the time consumed by slow TCP startup is reduced. In addition, because there is only one TCP connection, there is no problem of different TCP competing for network bandwidth.

After the request sent by the client passes through the binary frame division layer, it is no longer a complete HTTP request packet, but a bunch of out-of-order frames (that is, the frames of different streams are in disorder, but the frames of the same stream are transmitted in sequence), so the transmission is not in order, and there is no waiting, thus solving the HTTP head blocking problem.

How is it done

  • First, the browser prepares the request data, including the request line, the request information, and, if it is a POST method, the request body.
  • These data are processed by the binary frame layer and converted into frames with request ID numbers, which are sent to the server through the protocol stack. The request header information is stored in the header frame, and the request body data is stored in the Data frame.
  • After the server receives all the frames, it merges all the frames with the same ID into one complete request message.
  • The server then processes the request and sends the processed response line, response header, and response body to the binary framing layer, respectively.
  • Again, the binary framing layer converts the response data into frames with the request ID number that are sent through the protocol stack to the browser.
  • After the browser receives the response frame, it submits the frame’s data to the request based on the ID number.

Other HTTP/2 features

1. You can set the request priority

In the browser, some data is very important, such as key CSS or JS data. If this important data is pushed to the browser late, it will be a bad experience for the user.

Therefore, HTTP/2 supports setting the priority of requests so that the server receives a request with a higher priority and processes it first.

2. Server push

In HTTP/2, after the server parses to an HTML page, the server knows that the browser needs the resources referenced on the page, such as CSS and JS, so the server will actively push these resources to the browser, reducing the waiting time of the client.

3. Head compression

HTTP/2 uses the HPACK compression algorithm to compress the request and response headers, and although the effect is not obvious after a single request is compressed, if a page has 100 requests, each request is compressed by 20%, the speed increase is significant.

The principle of HPACK compression is actually two points:

  • It requires both the client and server to maintain and update an index list of previously seen header fields (that is, establish a shared compression context), and then use that list as a reference to effectively encode previously transmitted values. Reduce the size of each request by replacing existing fields in static or dynamic tables on each side with indexes during the actual transfer.
  • It allows the sent header fields to be encoded by static Huffman codes, reducing their respective transmission sizes.

HTTP/3

HTTP/2 is still tcp-based, so there are some issues.

The TCP header is blocked. Procedure

In HTTP/2, multiple requests run in a TCP connection. If packet loss occurs in a data flow, all requests in the TCP connection will be blocked. This is different from HTTP/1.1, where the browser establishes six TCP connections for each domain name. If one of the TCP connections is blocked, the other five connections can continue to transmit data.

TCP connection establishment delay

Before data is transmitted, a TCP three-way handshake is required, which costs 1.5 RTT. If HTTPS is used, a TLS connection is required and 1 or 2 RTTS are required.

Network latency, also known as Round Trip Time (RTT), is the total Round Trip Time of sending a data packet from the browser to the server, and then returning the data packet from the server to the browser.

In short, it takes 3-4 RTTS before transferring data. If the client is close to the server, the RTT is about 10ms, but if it is far away, it may be 100ms, so it takes about 300ms before the data is transferred, at which point you will feel slow.

TCP protocol fossilization

We know TCP suffers from queue blocking and connection establishment delay, but we cannot improve TCP for the following two reasons:

  • The intermediate equipment is rigid. Intermediate devices such as routers, switches, firewalls, and NAT rely on software that uses a lot of TCP features and is rarely updated once its functionality is set. If the TCP protocol is upgraded on the client, packets of the new protocol may not be understood when they pass through the device, causing data loss.
  • The operating system is another cause of TCP’s rigidity.

QUIC agreement

HTTP/3 is based on UDP, similar to the IMPLEMENTATION of TCP multi-channel data flow, transmission reliability and other functions, we call this function QUIC protocol.

  • It realizes the functions of flow control and transmission reliability similar to TCP. Although UDP does not provide reliable transmission, QUIC adds a layer on top of UDP to ensure reliable transmission of data. It provides packet retransmission, congestion control, and other features found in TCP.
  • Integrated TLS encryption function. QUIC currently uses TLS1.3, which has more advantages than the earlier version of TLS1.3, the most important of which is the reduction in the number of RTT spent on the handshake.
  • The multiplexing function in HTTP/2 is realized. Unlike TCP, QUIC enables multiple independent logical data flows over the same physical connection (see figure below). Realize the separate transmission of data stream, and solve the problem of TCP squadron head blocking.
  • Realize the quick handshake function. Since QUIC is udP-based, it can use 0-RTT or 1-RTT to establish connections, which means that QUIC can send and receive data as quickly as possible, which can greatly improve the speed of first page opening.

The challenge of HTTP / 3

  • First, as of now, HTTP/3 is not fully supported on the server or browser side. Chrome has supported Google’s version of QUIC for several years, but there are major differences between this version and the official QUIC.
  • Second, deploying HTTP/3 is also very problematic. Because the system kernel optimization of UDP is not nearly as good as TCP optimization, this is also an important reason to block QUIC.
  • Third, the rigidity of intermediate equipment. The UDP optimization of these devices is much lower than that of TCP. According to statistics, when QUIC is used, the packet loss rate is about 3% to 7%.