Today’s post is an unplanned update from “An HTTP protocol optimization” I saw during the day. In the article, the author describes this phenomenon:

On the premise that the number of MOBILE HTTP requests is about the same as that of Unicom, the network traffic brought by mobile HTTP response is 2.5 times that of Unicom. About 30 percent of mobile requests are not compressed, while Unicom is almost all compressed. HTTP sessions that are not compressed use the 1.0 protocol, whereas HTTP sessions that are compressed use the HTTP1.1 protocol.

This means that under the same Server configuration, 30% of the traffic from mobile operators goes through HTTP/1.0, and the HTTP Server used by the author does not enable GZip for HTTP/1.0 responses.

Why there is such a high percentage of HTTP/1.0 requests under mobile carrier networks is not discussed in this article, but it must be mobile. To get straight to another question, the title of this article says: Why does Nginx not compress HTTP/1.0 by default?

The author of that article doesn’t specify what HTTP Server he uses, I’ll just use Nginx. As you will see later, this problem is related to the HTTP protocol, which is encountered by all HTTP servers.

In the documentation on the Nginx website, there is a directive like this:

Syntax: gzip_http_version 1.0 | 1.1;

Default: gzip_http_version 1.1;

Context: http, server, location

Sets the minimum HTTP version of a request required to compress a response.

Obviously, this directive is used to set the lowest version of HTTP required for Nginx to enable GZip, and the default is HTTP/1.1. That is, Nginx does not compress HTTP/1.0 by default because of this directive, and changing its value to 1.0 will fix the problem.

For text files, the effect of GZip is very obvious. After enabling GZip, the traffic required for transmission will be reduced to about 1/4 ~ 1/3. Nginx can be configured to support such nice things. Why is it not enabled by default?

Nginx has an accept-encoding that meets the following criteria: Gzip, where The content-type of The response exists in The list of gzip_types) requests are on-the-fly Compression, The whole Compression process is done streaming in memory. That is, Nginx does not wait for the file GZip To complete before returning a response. Instead, Nginx compresses the response as it goes, which can significantly improve TTFB (Time To First Byte). The only problem with this is that when Nginx starts returning a response, it has no way of knowing how big the file to transfer will end up being, i.e. it can’t give the content-Length response header.

We also know that HTTP/1.1 supports TCP Persistent connections by default, and HTTP/1.0 can also enable Persistent connections by explicitly specifying Connection: keep-alive. HTTP runs on top of TCP connections, and naturally has the same characteristics as TCP, such as three-way handshake, slow start, etc. To improve HTTP performance, it is particularly important to enable persistent connections.

With these two points in mind, the client needs a mechanism to determine exactly where HTTP packets over A TCP persistent connection end. In HTTP/1.0, this mechanism was content-Length only. Thus, in the case described earlier in this article, HTTP Server can either not compress or do not enable persistent connections (for non-persistent connections, TCP breaks and the HTTP message ends), whereas Nginx defaults to the former.

So in HTTP/1.1, has this problem been solved? Of course! As I mentioned in a previous article, the chunked Transfer mechanism for HTTP/1.1’s transfer-Encoding: chunked is a perfect solution to this problem. For those interested, check out my article: Transfer-encoding in HTTP.

Long ago, we used to precompress static resources into.gz files to reduce Server CPU stress, so that HTTP Server didn’t need to compress the files immediately and wouldn’t encounter the problem mentioned in this article. As time goes on, almost no one does this anymore. For details on pre-compression, look at the ngx_HTTP_gzip_STATIC_module module of Nginx.

Theoretical knowledge first write here, finally use practice to verify:

First, do not enable the HTTP/1.0 GZip function of Nginx, use HTTP/1.0 request message test:

As you can see, although my request message indicates that GZip is acceptable, the content returned is still uncompressed; The server responds with Content-Length and Connection: keep-Alive, and the Connection is not broken. That is, for HTTP/1.0 requests, Nginx has abandoned GZip, the default policy, in order to enable persistent connections as much as possible.

Then, enable the HTTP/1.0 GZip function of Nginx and test with HTTP/1.0 request messages:

As you can see, this time the request message is exactly the same as last time, but the result is completely different: although the return content is compressed, the Connection is also broken, and the server returns Connection: close. The reason, as mentioned earlier, is that dynamic compression makes it impossible to know the length of the response in advance, and in HTTP/1.0 you had to rely on disconnection to let the client know that the response was over.

Finally, use HTTP/1.1 request message test:

As you can see, since the request message is HTTP/1.1, Nginx knows that the client supports HTTP/1.1 transfer-encoding: chunked, and the chunking resolves all problems: both compression and persistent connections are enabled.

So, for HTTP/1.0 requests, is it better for Nginx to forgo persistent connections or GZip?

In fact, since HTML documents or JSON interfaces are typically dynamically output in server-side languages such as PHP and Node.js, Nginx has no way of knowing their content-Length in advance, even without compression. Persistent connections were not enabled in HTTP/1.0 anyway. You might as well enable GZip to save traffic.

For static text files such as JS and CSS whose size can be known in advance, my suggestion is that important JS and CSS should be inlined in HTML for the first time access on the mobile terminal, and then stored in localStorage. Unimportant JS and CSS are imported through the external chain. Enable GZip and sacrifice keep-alive to reduce traffic.

For static image resources, since mainstream image formats have been highly compressed, there is no need to waste server CPU to enable GZip, which will not affect the keep-alive mechanism.

This article first wrote here, welcome to the original blog comments, exchanges. Browser GZip actually has a lot of interesting little story, first sell to keep in suspense, later free to write again.

The original post was published on 2015-12-15 00:54:33.

I am the blogger of “JerryQu’s Little Station (imququ.com)”. My independent blog has been suspended for a few years now, and I will bring some useful content to the nuggets community in the near future, as well as write some new articles in nuggets, please follow me.