preface

We know that both the CDN and the server interface (nginx configure GZIP to optimize the loading speed of site resources) will configure GZIP to compress the response data when the response is returned, so as to reduce the volume and speed up the network transmission efficiency. However, gZIP compression is only used for response data, which is generally sufficient for PC sites.

However, sometimes for mobile apps, gZIP compression of Response alone is not enough, because in request request, if the body is sometimes loaded with a lot of data, it will still have a large volume, thus consuming traffic. Mobile data is much more expensive than PC data. Therefore, for some apps that pay more attention to traffic consumption, it is best to use GZIP to compress data when requesting data, and then the server side in nginx or the gateway side of the program, Determine whether to gzip the requested data according to whether the request header content-encoding is gzip.

In field

I’m going to simulate this process by using Golang to simulate whether the client request and the server response should have a Gzip header. The code is very simple and already minimal. The client code is client.go, the server code is server.go, and no third-party libraries are required. (My Golang environment is 1.13.8)

The client

client.go:

package main

import (
	"bytes"
	"compress/gzip"
	"fmt"
	"io/ioutil"
	"net/http"
	"flag"
)

func main(a) {
	var requestUseGzip = flag.String("req_use_gzip"."1"."has request body use gzip")
	var responseUseGzip = flag.String("resp_use_gzip"."1"."has response body use gzip")
	flag.Parse()
	var requestBodyStr = `{"name":"zach ke","des":"woooooo boy", "response_use_gzip": "%s"}`
	data := []byte(fmt.Sprintf(requestBodyStr, *responseUseGzip))
	var httpRequest *http.Request
	var err error
	if *requestUseGzip == "1"{
		fmt.Println("request with gzip")
		var zBuf bytes.Buffer
		zw := gzip.NewWriter(&zBuf)
		if_, err := zw.Write(data); err ! =nil {
			fmt.Println("gzip is faild,err:", err)
		}
		zw.Close()
		httpRequest, err = http.NewRequest("POST"."http://localhost:9905/request", &zBuf)
		iferr ! =nil {
			fmt.Println("http request is failed, err: ", err)
		}
		httpRequest.Header.Set("Content-Encoding"."gzip")}else {
		fmt.Println("request without gzip")
		reader := bytes.NewReader(data)
		httpRequest, err = http.NewRequest("POST"."http://localhost:9905/request", reader)
		iferr ! =nil {
			fmt.Println("http request is failed, err: ", err)
		}
	}
	// Determine whether the return value should be gzip compressed
	if *responseUseGzip == "1" {
		httpRequest.Header.Set("Accept-Encoding"."gzip")}else{
		// Specify this as well, otherwise the content-Length header in response will not get the value
		httpRequest.Header.Set("Accept-Encoding"."deflate")
	}
	client := &http.Client{}
	httpResponse, err := client.Do(httpRequest)
	iferr ! =nil {
		fmt.Println("httpResponse is failed, err: ", err)
	}
	defer httpResponse.Body.Close()

	fmt.Println("respone content length=", httpResponse.ContentLength)
	if httpResponse.StatusCode == 200 {
		var respBody string
		switch httpResponse.Header.Get("Content-Encoding") {
		case "gzip":
			fmt.Println("response with gzip")
			reader, err := gzip.NewReader(httpResponse.Body)
			iferr ! =nil {
				fmt.Println("gzip get reader err ", err)
			}
			data, err = ioutil.ReadAll(reader)
			respBody = string(data)
		default:
			fmt.Println("response without gzip")
			bodyByte, _ := ioutil.ReadAll(httpResponse.Body)
			respBody = string(bodyByte)
		}
		fmt.Println("resp data=", respBody)
	}
}
Copy the code

As you can see, I use flag to control whether the request should be gzip compressed and whether the response should be gzip compressed. So there are four possibilities:

  1. Request and Response gzip are compressed
  2. Request is gzip compressed, but Response is not compressed
  3. Request is not compressed, and Response is gzip compressed
  4. Request is not compressed and response is not compressed

The service side

server.go:

package main

import (
	"compress/gzip"
	"fmt"
	"io/ioutil"
	"net/http"
	"bytes"
)

func handler(resp http.ResponseWriter, req *http.Request) {
	var bodyDataStr string
	// Whether gzip compression needs to be uncompressed
	fmt.Println("resquest content length=", req.ContentLength)
	if req.Header.Get("Content-Encoding") = ="gzip" {
		fmt.Println("request with gzip")
		body, err := gzip.NewReader(req.Body)
		iferr ! =nil {
			fmt.Println("unzip is failed, err:", err)
		}
		defer body.Close()
		data, err := ioutil.ReadAll(body)
		iferr ! =nil {
			fmt.Println("read all is failed.err:", err)
		}
		bodyDataStr = string(data)
	} else {
		fmt.Println("request without gzip")
		data, err := ioutil.ReadAll(req.Body)
		defer req.Body.Close()
		iferr ! =nil {
			fmt.Println("read resp is failed, err: ", err)
		}
		bodyDataStr = string(data)
	}
	fmt.Println("request json string=", bodyDataStr)

	respJson := []byte(`{code: "1",msg: "success"}`)

	// Determine whether the return value should be gzip compressed using the accept-encoding of the header
	if req.Header.Get("Accept-Encoding") = ="gzip" {
		fmt.Println("response with gzip")
		// Add the gzip header
		resp.Header().Set("Content-Encoding"."gzip")
		var zBuf bytes.Buffer
		zw := gzip.NewWriter(&zBuf)
		if_, err := zw.Write(respJson); err ! =nil {
			zw.Close()
			fmt.Println("gzip is faild,err:", err)
		}
		zw.Close()
		//fmt.Println("gzip content:", string(zBuf.Bytes()))
		resp.Write(zBuf.Bytes())
	}else{
		fmt.Println("response without gzip")
		// Return normally without compression
		resp.Write(respJson)
	}
}

func main(a) {
	fmt.Println("http://localhost:9905/request")
	http.HandleFunc("/request", handler)
	http.ListenAndServe(": 9905".nil)}Copy the code

To run

Run the server first:

go run server.go
Copy the code

Next, run the client program for each of the four cases and look at the output from the server

1. The compression

Client instructions:

~ >go run client.go
request with gzip
respone content length= 50
response with gzip
resp data= {code: "1",msg: "success"}
Copy the code

Server output:

resquest content length= 85
request with gzip
request json string= {"name":"zach ke","des":"woooooo boy", "response_use_gzip": "1"}
response with gzip
Copy the code

2. Request is compressed. Response is not compressed

Client instructions:

~ >go run client.go -resp_use_gzip=0 -req_use_gzip=1
request with gzip
respone content length= 26
response without gzip
resp data= {code: "1",msg: "success"}

Copy the code

Server output:

resquest content length= 85
request with gzip
request json string= {"name":"zach ke","des":"woooooo boy", "response_use_gzip": "0"}
response without gzip
Copy the code

3. Request is not compressed and response is compressed

Client instructions:

~ >go run client.go -resp_use_gzip=1 -req_use_gzip=0
request without gzip
respone content length= 50
response with gzip
resp data= {code: "1",msg: "success"}
Copy the code

Server output:

resquest content length= 64
request without gzip
request json string= {"name":"zach ke","des":"woooooo boy", "response_use_gzip": "1"}
response with gzip
Copy the code

4. None of them compress

Client instructions:

~ >go run client.go -resp_use_gzip=0 -req_use_gzip=0
request without gzip
respone content length= 26
response without gzip
resp data= {code: "1",msg: "success"}
Copy the code

Server output:

resquest content length= 64
request without gzip
request json string= {"name":"zach ke","des":"woooooo boy", "response_use_gzip": "0"}
response without gzip
Copy the code

It is easy to determine whether to use gzip compression by setting the request or response headers:

  1. If I set myselfContent-Encoding 为 gzip, indicating that your data is gzip compressed, the other party needs to perform gzip decompression
  2. If you want the data to be gzip-compressed, set the headerAccept-Encoding 为 gzip

Larger volume after gZIP compression

I wonder if you have noticed one of the above situations, that is, whether the data of request or response is compressed by gzip, the volume of bytes is larger than the original, but the volume of bytes that are not compressed is smaller. In order to be more intuitive, I have listed the following table:

– Request body byte Response body byte
All compressed 85 50
Compression only request 85 26
Only compressed response 64 50
Don’t compress 64 26

This is true. In fact, when gzip is compressed, the original data is smaller, but the compressed volume is larger. This is why nginx has a gzip_min_length parameter to control the minimum number of bytes to compress. See gzip configuration gzip_min_length to determine whether the response should be gzip compressed

Next, let’s make both the request body and the response body larger

respJson := []byte(`{code: "1",msg: "success", des: "12asdfsafsafsagaegewgsdfsdfsdfsdfgghhhhhhhhhhsfgegrfsdffffffffffadffwefwefefwafrrrrrrrrrrrrrwgsdfsdfsdfsdfgghhhhhhhhhhs fgegrfsdffffffffffadffwefwefefwafrrrrrrrrrwgsdfsdfsdfsdfgghhhhhhhhhhsfgegrfsdffffffffffadffwefwefefwafrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr"}`)Copy the code
var requestBodyStr = `{"name":"zach ke","des":"woooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo  boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo  boywoooooo boywoooooo boy", "response_use_gzip": "%s"}`Copy the code

It can be seen that the compressed volume is:

request with gzip respone content length= 111 response with gzip resp data= {code: "1",msg: "success", des: "12asdfsafsafsagaegewgsdfsdfsdfsdfgghhhhhhhhhhsfgegrfsdffffffffffadffwefwefefwafrrrrrrrrrrrrrwgsdfsdfsdfsdfgghhhhhhhhhhs fgegrfsdffffffffffadffwefwefefwafrrrrrrrrrwgsdfsdfsdfsdfgghhhhhhhh hhsfgegrfsdffffffffffadffwefwefefwafrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrr"}Copy the code
resquest content length= 89 request with gzip request json string= {"name":"zach ke","des":"woooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo  boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boy", "response_use_gzip": "1"} response with gzipCopy the code

The request volume is 89, and the response volume is 111.

If there is no compression, then:

request without gzip respone content length= 343 response without gzip resp data= {code: "1",msg: "success", des: "12asdfsafsafsagaegewgsdfsdfsdfsdfgghhhhhhhhhhsfgegrfsdffffffffffadffwefwefefwafrrrrrrrrrrrrrwgsdfsdfsdfsdfgghhhhhhhhhhs fgegrfsdffffffffffadffwefwefefwafrrrrrrrrrwgsdfsdfsdfsdfgghhhhhhhh hhsfgegrfsdffffffffffadffwefwefefwafrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrr"}Copy the code
resquest content length= 383 request without gzip request json string= {"name":"zach ke","des":"woooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo  boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boywoooooo boy", "response_use_gzip": "0"} response without gzipCopy the code

The request volume is 383 and the response volume is 343. You can see that the compression rate is pretty good.

conclusion

Basically, the more data you have, the more traffic you can save with GZIP, and the faster the network transfers, so why not? (In fact, you can also set the compression ratio and compression algorithm, so I’ll just use the default configuration for demonstration purposes.)

However, if gzip is used for request requests, performance will be affected. The larger the amount of data, the longer the compression and decompression time will be consumed, so the general PC site is not needed, only some apps that are sensitive to traffic consumption can consider using this method.

See my personal site: kebingzao.com/ for more good articles