This is the third day of my participation in the November Gwen Challenge. Check out the details: the last Gwen Challenge 2021

One, foreword

The Internet connection is slow and unstable, and may be disconnected due to network faults.

When a client downloads a large object, the probability of upload and download failure due to network disconnection becomes significant.

When a client requests a GET object, it sets the Range header to tell the interface service where it needs to start to output the object’s data.

To determine whether breakpoint downloading is supported, refer to the document 14.35.1 Byte Ranges

Accept-ranges = bytes
boolean support = urlConnection.getHeaderField("Accept-Ranges").equals("bytes");
System.out.println("Partial content retrieval support = " + (support ? "Yes" : "No));
Copy the code

Such as:

Donald @ Donald - pro: ~ $curl -i - range 0 to 9 http://localhost:8080/file/chunk/download HTTP / 1.1 206 Accept - Ranges: bytes Content-Disposition: inline; filename=pom.xml Content-Range: bytes 0-9/13485 Content-Length: 10 Date: Mon, 01 Nov 2021 09:53:25 GMTCopy the code

Make a direct judgment on the HEAD, e.g.

The HeadObject interface is used to get meta information about an Object. File contents are not returned using this interface.

HEAD /ObjectName HTTP/1.1
Host: BucketName.oss-cn-hangzhou.aliyuncs.com
Date: GMT Date
Authorization: SignatureValue
Copy the code

Note that the corresponding HTTP status code is:

  • 206 Partial Content:HTTP RangeThe request is successful
  • 416 Requested Range Not Satisfiable status:HTTP RangeRequest out of bounds
  • 200 OK: Range requests are not supported

The summary is as follows:

  1. HTTPScope request: requiredHTTP / 1.1If a certain segment of the two ends is earlier than this version, it is considered not supported.
  2. Through the response headerAccept-RangesTo determine whether range requests are supported.
  3. By adding in the request headerRangeThis request header specifies the byte range of the requested content entity.
  4. In the response header, passContent-RangeTo identify the range of Content entities currently returned, and use Content-Length to identify the Length of the range of Content entities currently returned.
  5. During the request process, you can passIf-RangeTo distinguish whether the resource file has changed, its value comes fromETagorLast-Modifled. If the resource file is changed, the download process will be restarted.




Two, production actual combat

Development also relies on evidence, setting boundaries, to control the overall picture.

There are ready-made documents, see ali Cloud documentation:

  • Range: bytes=0-499: indicates the content in the range of 0 to 499 bytes.
  • Range: bytes=500-999: indicates the content in the 500-999 byte range.
  • Range: bytes=-500: indicates the last 500 bytes.
  • Range: bytes=500-: represents the contents from the 500th byte to the end of the file.
  • Range: bytes=0-: indicates the complete file content from the first byte to the last byte.

HTTP Range:

  • ifHTTP RangeThe request is valid. The response returns 206 and is contained in the response headerContent-Range
  • ifHTTP RangeThe request is invalid or the specified range is not in the valid rangeRangeDoes not take effect, the response returns a value of 200 and passes the wholeObjectThe content.

The following is an example of an invalid HTTP Range request and error description: Assume that the size of the Object resource is 1000 bytes and the Range ranges from 0 to 999

  • Range: byte=0-499: format error,byteShould bebytes.
  • Range: bytes=0-1000: at the end of the byte1000Out of the valid interval.
  • Range: bytes=1000-2000: Indicates that the specified range exceeds the validity range.
  • Range: bytes=1000-: The first byte exceeds the validity range.
  • Range: bytes=-2000: Indicates that the specified range exceeds the validity range.

Here are some chestnuts:

#Normal range downloadDonald @ Donald - pro: ~ $curl -i - range 0 to 9 http://localhost:8080/file/chunk/download HTTP / 1.1 206 Accept - Ranges: bytes Content-Disposition: inline; filename=Screen_Recording_20211101-162729_Settings.mp4 Content-Range: bytes 0-9 Content-Type: application/force-download; charset=UTF-8 Content-Length: 16241985 Date: Wed, 03 Nov 2021 09:50:50 GMTCopy the code

Server – Business development

Here’s SpringBoot as chestnut:

  1. External supportrangedownload
  2. Underlying storage: Useceph


  1. ControllerAs follows:
@Slf4j
@RestController
public class Controller {
    @Autowired
    private FileService fileService;
    
    /** * download file ** external **@paramFileId indicates the fileId *@param token token
     * @paramAccountId accountId *@paramThe response response * /
    @GetMapping("/oceanfile/download")
    public void downloadOceanfile(@RequestParam String fileId,
                                  @RequestHeader(value = "Range") String range,
                                  HttpServletResponse response) {

        this.fileService.downloadFile(fileId, response, range); }}Copy the code
  1. ServiceAs follows:
@Slf4j
@Service
public class FileService {
    @Autowired
    private CephUtils cephUtils;
    
    /** * Download files directly ** Tips: Support breakpoint download *@paramFileId indicates the fileId *@paramThe response to return *@param* / the scope of the range
    public void downloadFile(String fileId, HttpServletResponse response, String range) {
        // Get file information based on fileId
        FileInfo fileInfo = getFileInfo(fileId);

        String bucketName = fileInfo.getBucketName();
        String relativePath = fileInfo.getRelativePath();

        // Process range information
        RangeDTO rangeInfo = executeRangeInfo(range, fileInfo.getFileSize());

        // rangeInfo = null to download the entire file directly
        if (Objects.isNull(rangeInfo)) {
        
            cephUtils.downloadFile(response, bucketName, relativePath);
            return;
        }
        // Download some files
        cephUtils.downloadFileWithRange(response, bucketName, relativePath, rangeInfo);
    }

    private RangeDTO executeRangeInfo(String range, Long fileSize) {

        if(StringUtils.isEmpty(range) || ! range.contains("bytes=") | |! range.contains("-")) {

            return null;
        }

        long startByte = 0;
        long endByte = fileSize - 1;

        range = range.substring(range.lastIndexOf("=") + 1).trim();
        String[] ranges = range.split("-");

        if (ranges.length <= 0 || ranges.length > 2) {

            return null;
        }

        try {
            if (ranges.length == 1) {
                if (range.startsWith("-")) {

                    For example, bytes=-1024 Indicates the data from the start byte to the 1024th byte
                    endByte = Long.parseLong(ranges[0]);
                } else if (range.endsWith("-")) {

                    //2. For example, bytes=1024- the 1024th byte to the last byte of data
                    startByte = Long.parseLong(ranges[0]); }}else {
                //3. For example, bytes=1024-2048 Indicates the 1024th to 2048 bytes of data
                startByte = Long.parseLong(ranges[0]);
                endByte = Long.parseLong(ranges[1]); }}catch (NumberFormatException e) {
            startByte = 0;
            endByte = fileSize - 1;
        }
        
        if (startByte >= fileSize) {
            
            log.error("range error, startByte >= fileSize. " +
                    "startByte: {}, fileSize: {}", startByte, fileSize);
            return null;
        }

        return newRangeDTO(startByte, endByte); }}Copy the code


Client – Business development

Here, Java is the chestnut.

Copy the code