This article is taken from my public account [Sun Wukong, Don’t talk nonsense]

HTTP protocol is the basic protocol of the Internet, but also the necessary knowledge of web development. This paper mainly introduces the historical evolution and design of HTTP protocol.

HTTP is an application layer protocol based on TCP/IP. For an introduction to TCP, check out these two articles: Three handshakes, Improved Congestion.

development

The evolution of HTTP is very interesting. Let’s look at the process:

Look at the time of its release, and the rise of the Internet is the same point in time. Look at the design principles, does it look like the way we often see a software product take over the market?

  1. Start by acquiring users as easily as possible;
  2. When the scale of users came up, with the market discourse power, began to develop market norms;
  3. Timely correction of defects in specifications;
  4. Keep up with the pace of The Times, self iteration, development and forward;

So let’s try to see how it develops from this perspective

The HTTP 0.9

At this time of the agreement, the characteristics are: brief introduction. To what extent? HTTP 0.9 is a one-line protocol for requests. Here’s an example:

GET /index.html
Copy the code

After the TCP connection is established, the client sends a request to the server for webpage index.html.

The request is a single line, including the GET method and the path to the document to be requested. The response is a hypertext document with no header and no other metadata, just HTML. It couldn’t be simpler!

Simple! But it works! The Development of the Internet at this time is not so developed. Such a simple agreement has been able to meet the needs of people at that time.

From modest beginnings in 1991, HTTP has come into its own in the following years and has grown rapidly.

The HTTP 1.0

background

Between 1991 and 1995, both the HTML specification and a new type of software called the Web browser developed rapidly. At the same time, consumer-facing public Internet infrastructure is also emerging and growing rapidly.

As the demand for emerging Web applications increases and their use on the public Web explodes, many fundamental weaknesses of HTTP 0.9 are exposed:

  • Can it only provide replies in HTML format? We want a richer response…
  • Can it provide more metadata about requests and responses?
  • .

HTTP 0.9 has a large number of users because of its simplicity. Also because of its simplicity, the emerging Web developer community has been able to build on it (rather than overturning it and looking for other protocols) with experimental implementations that meet market needs.

detailed

Based on these experimental developments, the HTTP Working Group developed a set of best practices, which became the specification HTTP 1.0. It has also been said that HTTP 1.0 is not a formal specification or Internet standard. But it doesn’t matter. Let’s take a look at what classic HTTP 1.0 has in common:

  1. First, content can be sent in any format. This allows the Internet to transmit not only text, but also images, video and binary files. This laid the foundation for the great development of the Internet.

  2. Secondly, in addition to GET command, POST command and HEAD command are also introduced to enrich the means of interaction between browser and server.

  3. Again, the format of HTTP requests and replies has changed. In addition to the data section, each communication must include headers (HTTP headers) that describe some metadata.

  4. Other new features include Status Code, multi-character set support, multi-part Type, authorization, cache, and Content encoding.

A very classic chestnut:

// Request GET/HTTP/1.0 user-agent: Mozilla/5.0 (Macintosh; Accept: */* // HTTP/1.0 200 OK Content-type: text/plain Content-Length: 137582 Expires: Thu, 05 Dec 1997 16:00:00 GMT Last-Modified: Wed, 5 August 1996 15:55:28 GMT Server: Apache 0.84 < HTML > <body>Hello World</body> </ HTML >Copy the code

It’s a lot more complicated than HTTP 0.9.

The HTTP 1.0 shortcomings

The main disadvantage of HTTP/1.0 is that only one request can be sent per TCP connection. Once the data is sent, the connection is closed, and if additional resources are requested, a new connection must be created.

As you can see from the previous article, TCP connections are expensive to create because of the need for three handshakes between the client and server, and slow start. As a result, HTTP 1.0 has poor performance. The more external resources a web page loads, the more this problem becomes.

To solve this problem, some browsers use a nonstandard Connection field when making requests:


Connection: keep-alive
Copy the code

This field requires that the server not close the TCP connection so that other requests can be reused. The server also responds with this field:


Connection: keep-alive
Copy the code

A TCP connection is established that can be reused until the client or server actively closes the connection.

But!!!!!! This is not a standard field and may behave differently across implementations. Therefore, it is not a fundamental solution.

The HTTP 1.1

To address this glaring problem with HTTP 1.0, HTTP 1.1 was released just six months later. This version further refined the HTTP protocol and remains the most popular version today.

It doesn’t just solve the TCP connection problems of HTTP 1.0, it also adds more features: persistent connections, pipelining mechanisms, chunked transfer coding, and more. Let’s take a look:

Persistent connection

HTTP 1.1 changed the semantics of the HTTP protocol to use persistent connections by default. That is, TCP connections are not closed by default and can be reused by multiple requests without the Connection: keep-alive declaration. Its benefits and problems solved have been covered in HTTP 1.0 and are not covered here.

The client and server can close the connection if they find the other side inactive for a period of time. However, it is standard practice for the client to send Connection: close on its last request, explicitly asking the server to close the TCP Connection.

In other words, unless explicitly told (via the Connection: Close header), the server will leave the Connection open by default.

Pipeline mechanism

HTTP 1.1 also introduced pipelining, which allows clients to send multiple requests simultaneously over the same TCP connection.

For example, the client needs to request two resources. In the past, in the same TCP connection, first send A request, and then wait for the server to respond, and then send A request B. The diagram below:

The pipeline mechanism allows the browser to make both A and B requests. This speeds up the efficiency of the request.

However, the pipeline mechanism here is only for the requester side.

The server responds to request A in FIFO order, and then responds to request B. This is done because HTTP 1.x can only return responses strictly serially. In particular, HTTP 1.x does not allow multiple response data on a connection to arrive interlaced (multiplexing), so a response must be fully returned before the next response can begin transmission.

This configuration on the server side can have a significant impact on performance when encountering problems such as “queue head congestion”. This flaw will be fixed in HTTP 2.

The Content – Length field

Since a SINGLE TCP connection in the pipeline mechanism can transmit multiple replies, there is a need for a mechanism to distinguish which response the packet belongs to. This is what the Content-Length field is for, declaring the length of the response.


Content-Length: 2408
Copy the code

The code tells the browser that the response is 2408 bytes long, and that the following bytes belong to the next response.

Block transfer encoding

The conditions for using the Content-Length field on the server are as follows: The server must know the Length of the reply data before sending the reply.

For some time-consuming dynamic operations, this means that the server waits until all operations are complete before sending data, which is obviously inefficient. A better approach is to send a block of data as it is generated, using a stream instead of a buffer.

Therefore, HTTP1.1 states that you can skip the Content-Length field and use “chunked Transfer encoding” instead. Any request or response header with a Transfer-Encoding field indicates that the response will consist of an undetermined number of data blocks.


Transfer-Encoding: chunked
Copy the code

Each non-empty block of data is preceded by a hexadecimal number indicating the length of the block. Finally, a block of size 0 indicates that the data for this response has been sent.

Disadvantages of HTTP 1.1

Although HTTP 1.1 allows reuse of TCP connections, all data traffic within the same TCP connection is sequential. The server does not process another response until it has processed one. If the response at the front is particularly slow, there will be a queue of requests waiting. This is called “head-of-line blocking.”

Another potential problem: the proliferation of HTTP headers, because all HTTP headers are sent in plain text (without any compression), can lead to a high overhead. This can cause serious performance problems in some applications.

Both of these points are addressed in HTTP 2. Let’s move on.

SPDY protocol

In 2009, Google unveiled its own SPDY protocol to address the inefficiencies of HTTP 1.1. Once this protocol proved viable in Chrome, it was used as the basis for HTTP 2, where major features were inherited.

SPDY protocol is the predecessor of HTTP 2 protocol.

The SPDY protocol is the catalyst for HTTP 2, but SPDY is not HTTP 2. If described in Git terms, HTTP 2 is an offshoot of SPDY, and both have since evolved in their own right.

HTTP 2

HTTP 2 was born in 2005.

HTTP 2 aims to overcome the well-known performance limitations of the previous generation of standards, but it is also an extension of, rather than a replacement for, the previous 1.x standard. The semantics and functionality of HTTP remain unchanged, as do the core concepts of HTTP methods, status codes, URIs, header fields, and so on. None of these changes is on the table.

The main reason for incrementing a large version to 2 is that it changes the way data is exchanged between clients and servers: it adds a new layer of binary framing data. This layer is not compatible with previous HTTP 1.x servers and clients.

Binary frame layer

The heart of HTTP 2’s performance enhancements lies in the new binary framing layer, which defines how HTTP messages are encapsulated and transmitted between clients and servers.

See above, HTTP 2 adds a binary framing layer to the existing classical hierarchy. On the right, HTTP 1.x uses a newline as a separator for plain text, while HTTP 2 is a thoroughly binary protocol that splits all transmitted information into smaller messages and frames (header and data frames in the figure) and encodes them in binary format.

This directly leads to HTTP 1.x’s incompatibility with HTTP 2.

Streams, messages, and frames

The new binary framing mechanism changes the way data is exchanged between clients and servers. To illustrate this process, we need to understand several new concepts of HTTP 2.

  • Stream: a bidirectional byte stream on an established connection.
  • Message: A complete sequence of data frames corresponding to a logical message.
  • Frame: The smallest unit of HTTP 2 communication. Each frame contains the frame header and at least identifies the stream to which the current frame belongs.

All HTTP 2 traffic is done over a single connection that can host any number of two-way data streams. Accordingly, each data stream is sent as a message, which consists of one or more frames that can be sent out of order and then reassembled according to the stream identifier at the beginning of each frame.

A great deal of information can be condensed into a few simple sentences.

To understand HTTP 2, you must understand the basic concepts of flows, messages, and frames:

  • All communication is done over a TCP connection.
  • A stream is a virtual channel in a connection that can carry two-way messages. Each stream has a unique integer identifier (1, 2… N).
  • A message is a logical HTTP message, such as a request, response, and so on, that consists of one or more frames.
  • A frame is the smallest unit of communication and carries specific types of data, such as HTTP headers, loads, and so on.

In short, HTTP 2.0 reduces the basic unit of HTTP protocol communication to frames that correspond to messages in a logical flow. Accordingly, many streams can exchange messages in parallel over the same TCP connection.

Multiple requests and responses

In HTTP 2, clients and servers can break HTTP messages into discrete frames, send them out of order, and then reassemble them at the other end. There is no need for one-to-one correspondence as with HTTP 1.x, thus avoiding “queue head congestion”.

Look at the chestnuts below:

The figure contains multiple streams in transit on the same connection: the client is sending a DATA frame (Stream 5) to the server, while the server is sending a series of frames of Stream 1 and Stream 3 out of order to the client. At this point, you have 3 request/response parallel exchanges on one connection!

Breaking HTTP messages into separate frames, sending them interlaced, and then reassembling them at the other end is one of the most important enhancements to HTTP 2. It solves the problem of queue head blocking in HTTP 1.x and eliminates the need for multiple connections to process and send requests and responses in parallel. The result is faster applications, easier to develop, and cheaper to deploy.

Header compression

The HTTP protocol has no state, and all information must be attached to each request. Therefore, many fields of the request are repeated, such as Cookie and User Agent. The same content must be attached to each request, which will waste a lot of bandwidth and affect the speed.

HTTP 2 optimizes this by introducing header compression. On the one hand, the header information is compressed using gzip or COMPRESS and then sent. On the other hand, both the client and the server maintain a header table where all fields are stored, generating an index number, and then not sending the same field, but only the index number, which increases speed.

Server push

A powerful new feature added to HTTP 2 is the ability for the server to send multiple responses to a client request. In other words, in addition to the response to the initial request, the server can push additional resources to the client without the client explicitly requesting them.

This feature provides a perfect solution to most of the frustrations of inserting or embedding resources in the HTTP 1.x era.

HTTP 2 restrictions

Network information transmission process, need a lot of intermediate equipment. To successfully use HTTP 2, both ends need to support the HTTP 2 protocol. If either intermediate device does not support it, the connection will not succeed.

Optimization Suggestions

With that said, let’s take a look at specific optimization recommendations for the HTTP protocol.

In view of the HTTP 1.0

The optimization strategy for HTTP 1.0 is simple: Upgrade to HTTP 1.1. Over!

In view of the HTTP 1.1

  1. Reducing DNS queries
  2. Reduce HTTP requests: No request is as fast as no request!
  3. Using CDN: Put data geographically close to the client.
  4. Add an Expires header and set an ETag: Take advantage of the browser’s compression mechanism.
  5. Gzip Resources: All text resources should be compressed using Gzip.
  6. Avoid HTTP redirects: HTTP redirects have several good things to do.

For HTTP 2

There’s nothing to say about that. Expect devices to support HTTP 2 soon. Let’s use HTTP 2 as soon as possible.

extra

Recently, I saw that Google is working on the HTTP 3 series to convert TCP, the underlying HTTP protocol, into UDP. This is what Google should do as The Times move forward. Thumbs up!

Front-end performance optimization series:

(a) : start with the TCP three-way handshake

(two) : for the blocking of TCP transmission process

(3) : optimization of HTTP protocol

(IV) picture optimization

(v) : browser cache strategy

(vi) : How does the browser work?

(vii) : Webpack performance optimization