Photo: By Charles Loyer

A, sequence

Hi, everyone, I am Chengxiang Ink shadow!

HTTP occupies an important position in network knowledge. The most basic HTTP protocol is the request and response message, and the message is composed of Header and entity. Most HTTP protocols rely on headers for different HTTP requests/responses.

This series of “Practical HTTP” puts aside the conventional Header explanatory expression, starting from the actual problem, to analyze the use of HTTP protocol, in the end to solve the problem? Explain how it is designed and how it is implemented.

HTTP is a stateless “loose protocol” that does not record the status of different requests, and because it contains two sides (client and server), most of its content is just a suggestion, which can be ignored.

“It says here that the MSRP is $2…”

“Oh, no advice!

This text is the fifth in this series, with the first four portals:

HTTP caching mechanism
HTTP content entity encoding compression mechanism
HTTP transport encoding compression mechanism
HTTP Cookie details

Today we’ll look at HTTP range requests again. Scope requests are mainly requests or uploads for larger files that can be operated on only a segment of it.

A more common scenario is resumable/download breakpoints, in the case of poor network conditions, can be disconnected and only continue to retrieve part of the content. For example, if you download software from the Internet, 95% of the time, the network is down, if the scope request is not supported, you are forced to start the download again. However, if there are scope requests, only the last 5% of resources need to be downloaded to avoid re-downloading.

Another scenario is multi-threaded download. For large files, multiple threads are opened, and each thread downloads a certain section of the file. Finally, after the download is completed, a complete file is assembled locally, which can make more effective use of resources.

While these are two common scenarios, let’s look at the technical details of HTTP protocol support for scope requests.

HTTP range requests

2.1 Whether range requests are supported

HTTP itself is a stateless “loose” protocol, and after many iterations, range requests are only supported over HTTP/1.1 (RFC2616). So if one end of the client or server is below HTTP/1.1, we should not use scope requests.

HTTP/1.1 explicitly declares a response header access-ranges to indicate whether range requests are supported, and it has only an optional bytes argument.

For example, given an MP4 response header, you can see that it is marked with Accept-Ranges:bytes, indicating that the current resource supports range requests.

2.2 Using range requests

If it is determined that both ends support scope requests, we can use it when requesting resources.

All files are ultimately bytes stored on disk or in memory, which can be divided into bytes for files to be operated on. This only requires HTTP support to request resources in the range n to n+x of the file.

HTTP/1.1 defines a Ranges request header to specify the range of request entities. It ranges from 0 to Content-length, using -split.

For example, you have downloaded 1000 bytes of resource content. If you want to continue downloading the subsequent resource content, simply add Ranges:bytes=1000- to the HTTP request header.

Range also has several different ways of limiting the Range, which can be flexibly customized as needed:

500-1000: Specifies the start and end ranges, generally used for multithreaded downloads.

2. 500- : Specifies the start range and continues until the end. This is more suitable for resumable, or online playback, etc.

3. -500: No start interval, meaning only that the last 500 bytes of content entities are required.

4. 100-300,1000-3000: Specify multiple ranges. There are few scenarios used in this way, so it’s good to know.

HTTP protocol is a bilateral negotiation protocol. Since the request header has been determined to use Ranges, the content-Ragne response header is also needed to mark the entity Content range of the response in the response header.

The content-range format is also clear, marking it first in bytes and then the Range and total length of the Content entities currently being passed.

Content-Range: bytes 100-999/1000
Copy the code

In this example, content entities in the range of 100 to 999 are passed, and the total size of the resource file is 1000 bytes. In addition, the HTTP response status code is 206 Partial Content.

The HTTP 206 Partial Content response code indicates that the request was successful and that the body contains the requested data Range specified in the Range header of the request.

About the interpretation of the 206 status code can refer to: https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Status/206

So a normal flow would look like this:

Note that content-Length is included in the response header of each HTTP transaction, except that it contains the Length of the Content entity of the current scope request response, rather than the full Length of the resource.

Now that I’ve basically covered the proper flow for HTTP range requests, let’s look at some special cases.

2.3 Resource Change

When we download a large resource in some download tools, we may occasionally pause and then download again, and it may start again.

It looks like the HTTP scope request is invalid, but it doesn’t have to be, most likely because the requested resource has changed during the request.

If you download, download the source of the resource file has changed, but the URL has not changed, as the file size may change (this is very easy to find), and in extreme cases even if there is no length changes, you continue to download again, probably eventually after the download is complete, unable to download the contents of the joining together into the file we need.

If we need to download a resource from the server, be sure to guard against possible changes to the resource. As mentioned earlier in HTTP caching, HTTP can be used to indicate whether the current resource has changed by ETag or last-Modified.

ETag: An authentication token fingerprint of the current file that identifies the uniqueness of the file.
Last-modified: indicates the time when the current file was Last Modified.

In THE HTTP Range request, these two fields can also be used to distinguish the segmented request resources, whether there is any modification, just need to put it in the request header if-range request header. If-range uses either the ETag or last-modified parameter.

If both operations are performed on the same resource file, the 206 status code is returned to start subsequent operations; otherwise, the 200 status code is returned, indicating that the file has been changed and must be downloaded from the beginning.

Note that if-range must be used together with Range; otherwise, it will be ignored by the server.

As an additional point, if the client requests the wrong Range in the header, the 416 status code will be returned.

HTTP 416 Range Not Satisfiable Indicates that the server cannot process the requested data Range. The most common case is when the requested data Range is outside the scope of the file, that is, the value at the head of the Range, while syntactically fine, is semantically meaningless.

About 416 status code, can refer to: https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Status/416

Examples of scope requests

3.1 Playing an adaptation with Chrome

Many of the technical points described above describe a single request fragment, so let’s use a practical example to illustrate the specifics of a scope request.

In this example, I found a play address for the video and played it directly in Chrome. After normal playback, then casually drag the video progress, then no operation to let it automatically play for a period of time, to look at HTTP transaction packets.

A brief description of the situation. When playing naturally, it will first send a request to the URL of the resource and return the response code of 200. It can be determined that the current resource supports Accept-ranges, and then send a Range request using Range. Each time a drag progress is made, a new range request is sent and the range is calculated according to the drag progress. There is no modification of the resource, so there is no re-request for download.

Instead of taking screenshots of HTTP transactions one by one, it abstracts the flow, as shown below:

As you can see, a resource download actually involves many requests, and we need to see it from a global perspective.

Iv. Summary of scope request

At this point we have covered the entire flow of HTTP range requests.

To recap the key points:

1. HTTP range requests must be supported by HTTP/1.1 or higher. If a certain segment of the two ends is earlier than this version, the request is not supported.

2. Determine whether range requests are supported by accept-ranges in the response header.

3. Specify the Range of bytes for the content entity of the request by adding Range to the request header.

4. In the response header, use Content-range to identify the Range of returned Content entities, and use Content-Length to identify the Range of returned Content entities.

5. During the request, if-range distinguishes whether the resource file has changed, with its value coming from ETag or last-modifies. If the resource file is changed, the download process will be restarted.

Add a flow chart to make it even clearer.

All the key technical points of this HTTP range request have been explained. Range requests are used in scenarios such as breakpoint continuations, multithreaded downloads, etc. Most CDN resources support range requests. It is up to your imagination to use range requests in these scenarios.

If you have any more ideas, please leave a comment.

Public number background reply growth “growth”, will get the learning materials I prepared, can also reply “add group”, learning progress together; You can also reply to “questions” and ask me questions.

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Graphic: HTTP range request, assist breakpoint continuation, multithreaded download core principle

A, sequence