HTTP header

This chapter mainly explains the structure of the HTTP header and the usage of each field in the header.

1. HTTP header

Hackr.jp User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; The rv: 13.0) Gecko / 2010010 Accept: text/HTML and application/XHTML + XML, application/XML. Q = 0.9 * / *; q=0 Accept-Language: ja,en-us; Q = 0.7, en. Q =0.3 accept-encoding: gzip, deflate DNT: 1 Connection: keep-alive if-modified-since: Fri, 31 Aug 2007 02:02:20 GMT If-None-Match:"45bae1-16a-46d776ac"
Cache-Control: max-age=0
Copy the code

HTTP/1.1 304 Not Modified
Date: Thu, 07 Jun 2012 07:21:36 GMT
Server: Apache
Connection: close
Etag: "45bae1-16a-46d776ac"
Copy the code

Header field name: field valueCopy the code

The content-type: text/HTMLCopy the code

Keep-alive: timeout=15, Max =100Copy the code

Cache-Control: private, max-age=0, no-cache
Copy the code

Cache-Control: public
Copy the code

Cache-Control: private
Copy the code

Cache-Control: no-cache
Copy the code

Cache-Control: no-store
Copy the code

Cache-control: smax-age=604800 (unit: second)Copy the code

Cache-control: max-age=604800 (unit: seconds)Copy the code

Cache-control: min-fresh=60 (unit: second)Copy the code

Cache-control: max-stale=3600 (unit: second)Copy the code

Cache-Control: only-if-cached
Copy the code

Cache-Control: must-revalidate
Copy the code

Cache-Control: proxy-revalidate
Copy the code

Cache-Control: no-transform
Copy the code

Connection: the name of the header field that will not be forwardedCopy the code

Connection: close
Copy the code

Connection: keep-alive
Copy the code

Date: Tue, 03 Jul 2012 04:40:59 GMT
Copy the code

Pragma: no-cache
Copy the code

Cache-Control: no-cache
Copy the code

Transfer-Encoding: chunked
Copy the code

Warning: [Warning code][Warning host: port number][Warning content] ([date and time])Copy the code

Accept: text/html,application/xhtml+xml,application/xml; Q = 0.9 * / *; Q = 0.8Copy the code

Accept-Charset: iso-8859-5,unicode-1-1; Q = 0.8Copy the code

Accept-Encoding: gzip,deflate

Copy the code

Accept-Language: zh-cn,zh; Q = 0.7, en - us, en. Q = 0.3Copy the code

Authorization: Basic dWVub3NlbjpwYXNzd29yZA==

Copy the code

Expect: 100-continue

Copy the code

From: [email protected]

Copy the code

This field is used to tell the server the E-mail address of the user using the user agent.

2.5.8 Host

Host: www.hackr.jp

Copy the code

Most of the time a server deployed on multiple sites, corresponds to the multiple domain name, when the DNS to the server, we need to know is which website, user access to assign to the corresponding port to the corresponding procedures for processing, so the Host user access is used to inform the server Host name and port number.

The Host header is the only header field that must be included in the request.

If the server does not have a host name, send a null value.

2.5.9 If – Match

A request header field in the form of if-xxx can be called a conditional request. Once the server receives a conditional request, it will only execute the request if it determines that the condition is true.

If-Match: "123456"

Copy the code

This header field is one of the strings that tells the server the value of the entity tag (ETag) used to match the resource. The server compares the field value of if-match with the ETag value of the resource and executes the request only If the two values are consistent. Otherwise, the feed Failed response is returned with status code 412.

When * is used, the server matches any ETag value.

2.5.10 If – Modified – Since

This header field specifies a point in time after which the request will be answered if the resource has been modified.

If-Modified-Since: Thu, 15 Apr 2004 00:00:00 GMT

Copy the code

If the resource has Not been Modified since this point in time, a response with the 304 Not Modified status code is returned.

2.5.11 If – None – Match

This field has the opposite effect to if-match. The request is answered only if the value of this field does not match the resource’s ETag.

2.5.12 If – Range

If the value of this field matches the value of the ETag or the date and time of the update, it is treated as a range request; If there is no match, all resources are returned.

2.5.13 If Unmodified — Since

If-Unmodified-Since: Thu, 03 Jul 2012 00:00:00 GMT

Copy the code

The header field, contrary to if-Modified-since, tells the server that the specified resource is processing the request only If there is no change after the specified date, otherwise it returns the 412 Precondition Failed state code.

2.5.14 Max – Forward

When a request is sent through the TRACE method or the OPTIONS method that contains the header field max-forward, this field specifies the maximum number of servers that can pass through as a decimal integer.

Max-Forward: 10

Copy the code

2.5.15 Proxy – Authorization

Proxy-Authorization: Basic dGlwObjkNLAGFfY5

Copy the code

This field informs the server of the information required for authentication between the client and the proxy server.

2.5.16 Range

Range: bytes=5001-10000

Copy the code

This field is suitable for scope requests that fetch only part of the resource, and it tells the server the specified scope of the server resource that it needs.

The server that receives a request with this field returns a 206 Partial Content response, or a 200 OK response with all resources if it cannot process the range request.

2.5.17 Referer

Referer: http://www.hackr.jp/index.htm

Copy the code

This header field tells the server the URI of the original resource requested.

According to this field, the server can know from which address the current request was redirected. For more information on Referer, read Ruan Yifeng’s blog: HTTP Referer Tutorial.

2.5.18 TE

TE: gzip, deflate. Q = 0.5Copy the code

This field tells the server the Transfer Encoding and relative priority of the response that the client can process, similar to transfer-encoding.

2.5.19 the user-agent

User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; The rv: 13.0) Gecko / 20100101 Firefox / 13.0.1Copy the code

This field sends information about the browser and user agent name that created the request to the server. When a crawler makes a request, it is possible to add the author’s address to this field.

It is worth mentioning that some websites will implement anti-crawler strategy, they will verify whether the user-Agent field is from the browser, if not, they will reject the crawler request; So if you want to continue to get the information you want, you will use this field to forge the crawler into a browser.

2.6 Response header Field

The response header field is the field used in the response packet returned by the server to the client, which is used to supplement the additional information of the response, server information, and additional requirements for the client.

2.6.1 Accept – Ranges

Accept-Ranges: bytes

Copy the code

This field is used to tell the client whether the server can handle a range request to specify a resource for a portion of the server side. Bytes when the server is able to handle client range requests, and None when the server is unable to handle client range requests.

2.6.2 Age

ETag: 600

Copy the code

This header field tells the client how long ago the source server created the response, and the value of the field is in seconds.

If the server that creates the response is a cache server, the Age value refers to the time between the second initiation of authentication and the completion of authentication.

2.6.3 ETag

ETag: "82e22293907ce725faf67773957acd12"

Copy the code

The header field, ETag, tells the client about the entity’s identity. It is a way to uniquely mark a resource as a string. The server assigns an ETag value to each resource.

When the resource is updated, the ETag value is also updated. When ETag values are completed, there is no uniform algorithm rule, only assigned by the server.

The URI that accesses a resource generally does not change, but when the resource itself changes, the ETag corresponding to the resource is updated.

Strong ETag and weak ETag

ETag values are strong and weak. Strong ETag values will change no matter how slight the resource changes, while weak ETag values will change only when the resource fundamentally changes and produces differences.

2.6.4 Location

Location: http://www.usagidesign.jp/sample.html

Copy the code

This header field can direct the client receiving the response to a resource different from the location of its request URI.

Basically this field is used in conjunction with the 3XX redirection status code.

2.6.5 Proxy – Authenticate

Proxy-Authenticate: Basic realm="Usagidesign Auth"

Copy the code

This header field sends the authentication information required by the proxy server to the client.

2.6.6 Retry After

Retry - After: 120Copy the code

This header field tells the client how long it will take to initiate the request again. This field can be specified as a specific time or the number of seconds after the response was created.

2.6.7 Server

This header field tells the client about the HTTP server application installed on the current server.

Server: Apache / 2.2.17 (Unix)Copy the code

2.6.8 than

Vary: Accept-Language

Copy the code

This header field controls the cache, and the source server communicates commands to the proxy server about how the local cache is used.

In particular, this header field specifies an additional header field, such as accept-encoding above. If the value of the specified field has not changed, the cache resource is used; If the value of a specified field changes, you need to request the latest resource from the source server.

This field can specify user-Agent to prevent the mobile browser from accessing the PC page cache.

When this field is specified as *, all requests are treated as unique and cached.

2.6.9 WWW – Authenticate

WWW-Authenticate: Basic realm="Usagidesign Auth"

Copy the code

This header field is used for HTTP access authentication.

2.7 Entity head field

The entity header field is the header used by the entity part contained in the request message and response message, and is used to supplement the update time and entity-related information of the content.

2.7.1 Allow

Allow: GET, HEADCopy the code

This header field notifies the client of HTTP methods supported by the server. When the server receives an unsupported HTTP method, the server returns the 405 Not Allowed status code.

2.7.2 the Content – Encoding

The Content - Encoding: gzipCopy the code

This header field tells the client the content encoding format to use for the body of the entity.

For details, see 2.5.3 Accept-encoding.

2.7.3 the Content – Language

The Content - Language: useful - CNCopy the code

This header field is used to inform the client of the natural language used by the entity body.

2.7.4 Content – Length

The Content - Length: 15000Copy the code

This header field indicates the size (in bytes) of the entity body.

You cannot use the content-Length header field if you use the content-encoding transfer for the entity body.

2.7.5 Content – the Location

The Content - the Location: http://www.hackr.jp/index-ja.htmlCopy the code

This header field gives the URI corresponding to the packet body. When the page Content returned is different from the actual object requested, the header content-location field specifies the URI.

2.7.6 Content – MD5

Content-MD5: OGFkZDUwNGVhNGY3N2MxMDIwZmQ4NTBmY2IyTY==

Copy the code

The client hashes the received packet body using the MD5 algorithm and compares it with the content-MD5 value to verify the integrity of the received packet body.

The calculation method of the hash algorithm is mainly used to prevent the transmission content from being tampered, because the results obtained after the calculation of the same input through the hash algorithm are the same, and the results obtained after the calculation of different input through the hash algorithm are basically not repeated. This is why many sites that provide resources for download also provide HASH values for resources.

But content-MD5 is unreliable because if the Content of the transport body can be modified, the value of content-MD5 may also have been modified to be unreliable. To ensure that resources are not tampered with, it is best to use HTTPS.

2.7.7 Content – Range

Content-Range: bytes 5001-10000/10000

Copy the code

For a scope request, this header field tells the client which parts of the entity of the response returned comply with the scope request, in bytes.

2.7.8 Content-Type

Content-Type: text/html; charset=UTF-8

Copy the code

This header field specifies the media type within the entity body.

2.7.9 Expires

Expires: Wed, 04 Jul 2012 08:26:05 GMT

Copy the code

This header field tells the client the date when the resource will expire.

If the header field cache-control has a max-age instruction, the max-age instruction is processed first.

2.7.10 Last-Modified

Last-Modified: Wed, 23 May 2012 09:59:55 GMT

Copy the code

This header field indicates when the resource was finally modified.

2.8 is the header field of the Cookie service

The working mechanism of Cookie is user identification and state management.

Header field for cookie service:

2.8.1 Set – cookies

When the server is ready to manage the state of the client, various messages are notified.

The table below lists the field values for set-cookie.

The following details the individual properties of cookies.

• Expires: This property specifies the expiration date when the browser can send a Cookie. When the Expires attribute is omitted, it is only valid for the duration of the browser Session. This is usually limited to before the browser is closed.

  It is also important to note that once a Cookie is sent from the server side to the client, there is no way to explicitly delete the Cookie on the server side. Deletion can only be achieved by overwriting existing cookies.

• Path: This property is used to restrict the file directory to which cookies are sent. But there are other ways to get around this limitation, so don’t expect too much from it.

• Domain: The domain name can match the end. For example, if domain is example.com, www.example.com and www2.example.com can also access cookies. Therefore, it is more secure not to specify the domain name, because by default only the domain name of the currently responding server can access the Cookie.

• Secure: This property restricts the Web page from sending cookies only when an HTTPS secure connection is made. When this property is omitted, the Cookie is accessible to the specified domain name of both HTTP and HTTPS.

• HttpOnly: This property prevents JavaScript from reading cookies.

2.8.2 cookies

Cookie: status =enable

Copy the code

The header field Cookie tells the server that when the client wants HTTP state management support, it includes the Cookie obtained from the server in the request.

2.9 Other header Fields

HTTP header fields are self-extensible, so there are some non-standard header fields in Web server and browser applications.

Here are some of the more common non-standard header fields:

• X-Frame-Options
• X-XSS-Protection
• DNT
• P3P

2.9.1 X – Frame – the Options

X-Frame-Options: DENY

Copy the code

This header field belongs to the HTTP response header and is used to control the display of Web content within the Frame tag of other Web sites. Its main purpose is to prevent clickjacking attacks.

Frame labels include iframe, Frame, and Object

This field can specify three types of values:

• DENY: Refuses to be nested within a Frame, even if the page is under the same domain name
• SAMEORIGIN: This page can only be embedded in the Frame of the same domain page
• ALLOW-FROM uri: indicates that a page can be nested within a specified source

2.9.2 X – XSS – Protection

This field belongs to the response header. It is a policy against cross-domain scripting attacks and is used to enable or disable the browser XSS defense mechanism.

The field can be set to two values:

• 0: Sets XSS filtering to invalid state
• 1: XSS filtering is set to a valid state

2.9.3 DNT

DNT: 1.Copy the code

The header field belongs to the HTTP request header. DNT is short for Do Not Track, indicating that personal information is refused to be collected.

0 means consent to be tracked and 1 means refusal to be tracked.

2.9.4 P3P

This header field belongs to the response header, which makes personal privacy on the site understandable only by using P3P technology.