File download, understand these 9 scenarios is enough

⚠️ this article for nuggets community first contract article, not authorized to forbid reprint

After the article was published, Po brother received a lot of messages from his friends, thanking them for their encouragement and support. At the bottom of the article, diggefriend @MyMisty Rain is not In Jiangnan and @Rainx posted the following comments:

Since digs the friend to have the request, even the title also helps a treasure elder brother to think well, that we come to the whole article, summarizes the file to download the scene.

In our work, there are usually nine scenarios for file downloading, each of which uses a different technology, and there are many details that we need to pay extra attention to. Today, I will take you to summarize these 9 scenarios, so that you can easily deal with a variety of download scenarios. After reading this article, you will know the following:

Blobs are often used when working with files on the browser side. For example, local image preview, image compression, large file block upload and file download. Blob is used in browser-side file download scenarios, such as the A tag download, showSaveFilePicker API download, Zip download, etc., so it is necessary to master the knowledge of Blob before studying the specific application. This will help us understand the sample code better.

First, basic knowledge

1.1 understand the Blob

Binary Large Object (Blob) indicates a Large Object of Binary type. In a database management system, binary data is stored as a collection of single individuals. A Blob is usually a video, sound, or multimedia file. In JavaScript, an object of type Blob represents an immutable, raw, file-like object. Its data can be read in text or binary format, and can be converted to a ReadableStream for data manipulation.

Blob objects consist of an optional string type (usually MIME type) and blobParts:

In JavaScript you can create Blob objects using the Blob constructor,Blob constructorHere’s the syntax for:

const aBlob = new Blob(blobParts, options);
Copy the code

The related parameters are described as follows:

BlobParts: This is an array of objects consisting of ArrayBuffer, ArrayBufferView, Blob, DOMString, etc. DOMStrings will be encoded as UTF-8.
Options: An optional object containing the following two properties:
- Type — The default is""Which represents the MIME type of the array contents to be put into the BLOB.
- The default is “endings”"transparent", which specifies to include a line terminator\nHow the string is written to. It is one of two values:"native", which means the line terminator will be changed to a newline suitable for the host operating system file system, or"transparent", which means that the ending character saved in the BLOB is kept unchanged.

1.2 Understanding the Blob URL

Blob URL/Object URL is a pseudo-protocol that allows Blob and File objects to be used as URL sources for images, download binary data links, and so on. In the browser, we use the url.createObjecturl method to create the Blob URL. This method takes a Blob object and creates a unique URL for it in the form of Blob :

/

.

blob:http://localhost:3000/53acc2b6-f47b-450f-a390-bf0665e04e59
Copy the code

The browser internally stores a URL → Blob mapping for each URL generated through url.createObjECturl. As a result, such urls are shorter, but can access bloBs. The generated URL is valid only as long as the current document is open. It allows references to blobs in , , but if you access a Blob URL that no longer exists, you will receive a 404 error from the browser.

The Blob URL above looks good, but it actually has a side effect. Although the URL → Blob mapping is stored, the Blob itself still resides in memory and the browser cannot free it. The mapping is automatically cleared when the document is uninstalled, so the Blob object is subsequently released. However, if the application has a long life, the Blob will not be released by the browser for a short time. So, if you create a Blob URL, even if the Blob is no longer needed, it will still be in memory.

To address this, you can call the url.revokeobjecturl (URL) method to remove the reference from the internal map, allowing the Blob to be removed (if there are no other references) and freeing up memory.

Now that you know about blobs and Blob urls, if you want to learn more about blobs, read the blobs you Don’t know. Let’s start with the client file download scenario.

With the continuous development of Web technology, the browser is more and more powerful. Many online Web design tools have emerged over the years, such as online Photoshop, online poster designer, or online Custom Form Designer. These Web designers allow the user to save the generated file locally once the design is complete, and some of them use the Web API provided by the browser for client-side file download. The following baoge first to introduce the client download, the most common a tag download scheme.

2. A tag download

html

<h3>A tag download example</h3>
<div>
  <img src=".. /images/body.png" />
  <img src=".. /images/eyes.png" />
  <img src=".. /images/mouth.png" />
</div>
<img id="mergedPic" src="http://via.placeholder.com/256" />
<button onclick="merge()">Image synthesis</button>
<button onclick="download()">Images are downloaded</button>
Copy the code

In the above code, we refer to the following three materials using the IMG tag:

When the user clicks the Image Composition button, the resultant image is displayed in the img#mergedPic container. After the image is successfully synthesized, the user can download the synthesized image to the local by clicking the image download button. The corresponding operation process is shown in the figure below:

As can be seen from the figure above, the overall operation process is relatively simple. Next, let’s look at the implementation logic for image compositing and image downloading.

For image compositing, Apog uses a third-party library called Merge-Images on Github. Using the mergeImages(images, [Options]) method provided by the library, we can easily achieve the function of image composition. When this method is called, a Promise object is returned, and when the asynchronous operation is complete, the synthesized image is returned as Data URLs.

const mergePicEle = document.querySelector("#mergedPic");
const images = ["/body.png"."/eyes.png"."/mouth.png"].map(
  (path) = > ".. /images" + path
);
let imgDataUrl = null;

async function merge() {
  imgDataUrl = await mergeImages(images);
  mergePicEle.src = imgDataUrl;
}
Copy the code

The image download function is with the help of dataUrlToBlob and saveFile these two functions to achieve. These are used to convert Data URLs => Blob and save files, respectively. The specific code is shown below:

function dataUrlToBlob(base64, mimeType) {
  let bytes = window.atob(base64.split(",") [1]);
  let ab = new ArrayBuffer(bytes.length);
  let ia = new Uint8Array(ab);
  for (let i = 0; i < bytes.length; i++) {
    ia[i] = bytes.charCodeAt(i);
  }
  return new Blob([ab], { type: mimeType });
}

// Save the file
function saveFile(blob, filename) {
  const a = document.createElement("a");
  a.download = filename;
  a.href = URL.createObjectURL(blob);
  a.click();
  URL.revokeObjectURL(a.href)
}
Copy the code

Since the topic of this article is file downloads, let’s focus on the saveFile function. Within this function, we use the HTMLAnchorElement. Download attribute, the attribute value indicates the name of the download file. If the name is not a valid file name for the operating system, the browser will adjust it. In addition, the function of this attribute is to indicate that the linked resource will be downloaded rather than displayed in the browser.

Note that the Download property has compatibility issues. For example, IE 11 and later versions do not support the download property, as shown in the figure below:

(Photo: caniuse.com/download)

When the download property of the a element is set, we call the url. createObjectURL method to create the ObjectURL and assign the returned URL to the A element’s href property. The download of the file is then triggered by calling the click method on the A element, and finally the url.revokeobjecturl method is called once to remove the reference from the internal map, allowing the Blob to be deleted (if there are no other references) and freeing memory.

With that out of the way for a tag downloads, let’s show you how to implement file downloads using the new Web API, showSaveFilePicker.

A tag download example: A-tag

Github.com/semlinker/f…

ShowSaveFilePicker API download

The showSaveFilePicker API is a method defined in the Window interface that, when called, displays a file selector that allows the user to select a save path. The signature of the method is as follows:

let FileSystemFileHandle = Window.showSaveFilePicker(options);
Copy the code

The showSaveFilePicker method supports an optional argument for an object type that can contain the following properties:

excludeAcceptAllOption: Indicates the Boolean type. The default value isfalse. By default, the selector should include an option not to apply any file type filters (as shown in the followingtypesOption enabled). Set this option totruemeanstypesOptions are not available.
types: Array type, which represents the list of file types allowed to be saved. Each item in the array is a configuration object that contains the following properties:
- Description (Optional): Describes the types of files that can be saved.
- accept: is an object whosekey 是 MIMEType whose value is a list of file extensions.

The showSaveFilePicker method is called, and a FileSystemFileHandle object is returned. With this object, you can call methods on this object to manipulate files. Such as call the createWritable method on the object, returns FileSystemWritableFileStream object, can the data written to the file. The specific use mode is as follows:

async function saveFile(blob, filename) {
  try {
    const handle = await window.showSaveFilePicker({
      suggestedName: filename,
      types: [{description: "PNG file".accept: {
            "image/png": [".png",}}, {description: "Jpeg file".accept: {
            "image/jpeg": [".jpeg"],},},],});const writable = await handle.createWritable();
    await writable.write(blob);
    await writable.close();
    return handle;
  } catch (err) {
     console.error(err.name, err.message); }}function download() {
  if(! imgDataUrl) { alert("Please compose the picture first.");
    return;
  }
  const imgBlob = dataUrlToBlob(imgDataUrl, "image/png");
  saveFile(imgBlob, "face.png");
}
Copy the code

When you use the updated saveFile function above to save the composite image, the following saveFile selector is displayed:

As you can see from the figure above, the showSaveFilePicker API allows you to select a directory for downloading files, select a format for saving files, and change the name of the files to be stored, compared to the A tag download method. The showSaveFilePicker API is not fully compatible with the showSaveFilePicker API.

(Photo credit: caniuse.com/?search=sho…

ShowSaveFilePicker is a method defined in the File System Access API. Methods such as showOpenFilePicker and showDirectoryPicker are also available. If you want to use these apis in a real project, consider using GoogleChromeLabs’ open source browser-fs-access library, which makes it easier to use the File System access API on supported platforms. and are automatically degraded for unsupported platforms.

The browser-fs-access library may be unfamiliar to you, but if you use filesaver.js, you should be familiar with it. Next, let’s introduce how to use filesaver. js library to implement client file download.

ShowSaveFilePicker API download example: save-file-picker

Github.com/semlinker/f…

You can download FileSaver files

Filesaver.js is a solution for saving files on the client and is ideal for Web applications that generate files on the client. It is an HTML5 version of the saveAs() FileSaver implementation that supports most major browsers, with compatibility shown below:

(Photo credit: github.com/eligrey/Fil…

With the introduction of the filesaver. js library, we can use the saveAs method it provides to save files. The signature of the method is as follows:

FileSaver saveAs(
 Blob/File/Url, 
 optional DOMString filename, 
 optional Object { autoBom }
)
Copy the code

The saveAs method supports three parameters, with the first parameter indicating that it supports the three types of Blob/File/Url, the second parameter indicating the File name (optional), and the third parameter indicating the configuration object (optional). Set {autoBom: true} if you want FlieSaver. Js to automatically provide Unicode text encoding hints (see: byte order markers).

After looking at the saveAs method, let’s take a look at three specific examples of its use:

1. Save the text

let blob = new Blob(["Everybody is good, I am a treasure elder brother!"] and {type: "text/plain; charset=utf-8" });
saveAs(blob, "hello.txt");
Copy the code

2. Save online resources

saveAs("https://httpbin.org/image"."image.jpg");
Copy the code

If the URL to be downloaded is in the same domain as the current site, use a[download] mode to download the file. Otherwise, a synchronous HEAD request is used to determine whether the CORS mechanism is supported, and if so, the data is downloaded and the file is downloaded using the Blob URL. If the CORS mechanism is not supported, a[download] mode will be attempted.

The standard W3C File API Blob interface is not available in all browsers. For this problem, you can consider using blob.js to solve compatibility issues.

(Photo credit: caniuse.com/?search=blo…

3. Save the canvas contents

let canvas = document.getElementById("my-canvas");
canvas.toBlob(function(blob) {
  saveAs(blob, "abao.png");
});
Copy the code

Note that the Canvas.toblob () method is not available in all browsers. For this problem, consider using Canvas-toblob. js to solve compatibility issues.

(Photo credit: caniuse.com/?search=toB…

With the saveAs method used as an example, let’s update the download method in the previous example:

function download() {
  if(! imgDataUrl) { alert("Please compose the picture first.");
    return;
  }
  const imgBlob = dataUrlToBlob(imgDataUrl, "image/png");
  saveAs(imgBlob, "face.png");
}
Copy the code

Obviously, with the saveAs method, downloading the composite image is easy. If you are interested in how filesaver. js works, you can take a look at how the 15.5K FileSaver works. This article. The previous scenarios are all about downloading a single file directly, but you can also download multiple files simultaneously on the client, compress the downloaded files into a Zip package and download them locally.

FileSaver Example: file-saver

Github.com/semlinker/f…

Five, Zip download

In this article, He explains how to use the API provided by the JSZip library to compress all files in the directory to be uploaded into a ZIP file, and then upload the resulting ZIP file to the server. Similarly, using the JSZip library, you can download multiple files simultaneously on the client, compress the downloaded files into a Zip package, and download them locally. The corresponding operation process is shown in the figure below:

In the Gif above, Abo demonstrates the process of packing the three images into a Zip file and downloading it locally. Next, let’s show you how to do this using the JSZip library.

html

<h3>Zip download example</h3>
<div>
  <img src=".. /images/body.png" />
  <img src=".. /images/eyes.png" />
  <img src=".. /images/mouth.png" />
</div>
<button onclick="download()">Pack to download</button>
Copy the code

const images = ["body.png"."eyes.png"."mouth.png"];
const imageUrls = images.map((name) = > ".. /images/" + name);

async function download() {
  let zip = new JSZip();
  Promise.all(imageUrls.map(getFileContent)).then((contents) = > {
    contents.forEach((content, i) = > {
      zip.file(images[i], content);
    });
    zip.generateAsync({ type: "blob" }).then(function (blob) {
      saveAs(blob, "material.zip");
    });
  });
}

// Download the file content from the specified URL
function getFileContent(fileUrl) {
  return new JSZip.external.Promise(function (resolve, reject) {
    // Get the contents of the file by calling the getBinaryContent method provided by the jszip-utils library
    JSZipUtils.getBinaryContent(fileUrl, function (err, data) {
      if (err) {
        reject(err);
      } else{ resolve(data); }}); }); }Copy the code

In the above code, the Download function is called when the user clicks the package download button. Inside this function, the JSZip constructor is called to create the JSZip object, then the promise. all function is used to ensure that all files are downloaded, and then the file(name, data [,options]) method is called. Add the downloaded file to the JSZip object you created earlier. GenerateAsync function is used to generate the ZIP file and save the zip file using saveAs provided by Filesaver. js.

Zip Example: Zip

Github.com/semlinker/f…

Vi. Download in the form of attachments

In the case of server download, attachment download is a common scenario. In this scenario, we set the Content-Disposition response header to indicate whether the Content of the response should be displayed inline or downloaded and saved locally as an attachment.

Content-Disposition: inline
Content-Disposition: attachment
Content-Disposition: attachment; filename="mouth.png"
Copy the code

In the case of an HTTP form, content-Disposition can also be used as a header in a multipart body:

Content-Disposition: form-data
Content-Disposition: form-data; name="fieldName"
Content-Disposition: form-data; name="fieldName"; filename="filename.jpg"
Copy the code

The first parameter is always a fixed form-data; The additional arguments are case insensitive and have a parameter value. The parameter name and the parameter value are joined by an equal sign (=) and the parameter value is enclosed in double quotation marks. Use semicolon (;) between arguments Space.

Now that we know what Content-Disposition does, let’s take a look at how to implement the attachment download function. Koa is an easy-to-use Web framework characterized by elegance, simplicity, lightweight, and high degree of freedom. So we chose it to build the file service and use @KOA/Router middleware to handle routing:

// attachment/file-server.js
const fs = require("fs");
const path = require("path");
const Koa = require("koa");
const Router = require("@koa/router");

const app = new Koa();
const router = new Router();
const PORT = 3000;
const STATIC_PATH = path.join(__dirname, "./static/");

// http://localhost:3000/file? filename=mouth.png
router.get("/file".async (ctx, next) => {
  const { filename } = ctx.query;
  const filePath = STATIC_PATH + filename;
  const fStats = fs.statSync(filePath);
  ctx.set({
    "Content-Type": "application/octet-stream"."Content-Disposition": `attachment; filename=${filename}`."Content-Length": fStats.size,
  });
  ctx.body = fs.createReadStream(filePath);
});

// Register the middleware
app.use(async (ctx, next) => {
  try {
    await next();
  } catch (error) {
    // ENOENT (no file or directory) : This is usually caused by a file operation, which indicates that no files or directories can be found on the given path
    ctx.status = error.code === "ENOENT" ? 404 : 500;
    ctx.body = error.code === "ENOENT" ? "File does not exist" : "Server wandering"; }}); app.use(router.routes()).use(router.allowedMethods()); app.listen(PORT,() = > {
  console.log('The application has been started: http://localhost:${PORT}/ `);
});
Copy the code

The above code is stored in file-server.js in the Attachment directory, which also has a static subdirectory for static resources. Currently, the static directory contains the following three PNG files:

├── class.htm ├─ class.htm ├─ class.htm ├─ class.htm ├─ class.htm ├─ class.htm ├─ class.htm ├─ class.htmCopy the code

After you have successfully started the file server by running the node file-server.js command, you can download the files in the static directory with the correct URL. Such as open http://localhost:3000/file? in your browser Filename =mouth.png, and you’ll start downloading the mouth.png file. If the specified file does not exist, it returns that the file does not exist.

The Koa kernel is simple, and extensions are implemented through middleware. For example, common routing, CORS, static resource processing and other functions are implemented through middleware. Therefore, to master the FRAMEWORK of Koa, the core is to master its middleware mechanism. If you want to learn more about Koa, read how to Better understand middleware and the Onion Model.

When writing HTML pages, for some simple images, it is usually chosen to embed the image content directly in the page, so as to reduce unnecessary network requests, but the image data is binary data, how to embed? Most modern browsers support a feature called Data URLs, which allows you to use Base64 to encode binary Data from an image or other file and embed it as a text string inside a Web page. So files can also be transferred in Base64 format, and we’ll show you how to download Base64 images.

An example of an attachment is attachment

Github.com/semlinker/f…

7. Base64 download

Base64 is a representation of binary data based on 64 printable characters. Since 2⁶ = 64, every 6 bits is a unit corresponding to a printable character. Three bytes have 24 bits, corresponding to four Base64 cells, that is, three bytes can be represented by four printable characters. The corresponding transformation process is shown in the figure below:

Base64 is commonly used in textual data situations to represent, transmit, and store binary data, including MIME E-mail and complex data in XML. In MIME E-mail, Base64 can be used to encode binary byte sequence data into text composed of a sequence of ASCII characters. When used, base64 is specified in the transport encoding. The characters used include 26 upper and lower Latin letters, 10 digits, plus sign (+), and slash (/), a total of 64 characters, and the equal sign (=) is used as a suffix.

So much for Base64, if you want to learn more about Base64, you can read this article about Base64 encoding. Let’s look at the implementation code:

7.1 Front-end Code

html

In the following HTML code, we use the SELECT element to let the user select the image to download. The image displayed in the img#imgPreview element changes as the user switches between different images.

<h3>Base64 download example</h3>
<img id="imgPreview" src="./static/body.png" />
<select id="picSelect">
   <option value="body">body.png</option>
   <option value="eyes">eyes.png</option>
   <option value="mouth">mouth.png</option>
</select>
<button onclick="download()">download</button>
Copy the code

const picSelectEle = document.querySelector("#picSelect");
const imgPreviewEle = document.querySelector("#imgPreview");

picSelectEle.addEventListener("change".(event) = > {
  imgPreviewEle.src = "./static/" + picSelectEle.value + ".png";
});

const request = axios.create({
  baseURL: "http://localhost:3000".timeout: 60000});async function download() {
  const response = await request.get("/file", {
    params: {
      filename: picSelectEle.value + ".png",}});if (response && response.data && response.data.code === 1) {
    const fileData = response.data.data;
    const { name, type, content } = fileData;
    constimgBlob = base64ToBlob(content, type); saveAs(imgBlob, name); }}Copy the code

The Download function in the above code is called when the user selects the image to download and clicks the download button. Inside this function, we use the GET method of the AXIos instance to make an HTTP request to get the specified image. Since the base64 image is returned, we need to convert the Base64 string to a blob object before calling the saveAs method provided by FileSaver. This is done by using the following base64ToBlob functions: The implementation of this function is as follows:

function base64ToBlob(base64, mimeType) {
  let bytes = window.atob(base64);
  let ab = new ArrayBuffer(bytes.length);
  let ia = new Uint8Array(ab);
  for (let i = 0; i < bytes.length; i++) {
    ia[i] = bytes.charCodeAt(i);
  }
  return new Blob([ab], { type: mimeType });
}
Copy the code

7.2 Server Code

// base64/file-server.js
const fs = require("fs");
const path = require("path");
const mime = require("mime");
const Koa = require("koa");
const cors = require("@koa/cors");
const Router = require("@koa/router");

const app = new Koa();
const router = new Router();
const PORT = 3000;
const STATIC_PATH = path.join(__dirname, "./static/");

router.get("/file".async (ctx, next) => {
  const { filename } = ctx.query;
  const filePath = STATIC_PATH + filename;
  const fileBuffer = fs.readFileSync(filePath);
  ctx.body = {
    code: 1.data: {
      name: filename,
      type: mime.getType(filename),
      content: fileBuffer.toString("base64"),}}; });// Register the middleware
app.use(async (ctx, next) => {
  try {
    await next();
  } catch (error) {
    ctx.body = {
      code: 0.msg: "Server wandering"}; }}); app.use(cors()); app.use(router.routes()).use(router.allowedMethods()); app.listen(PORT,() = > {
  console.log('The application has been started: http://localhost:${PORT}/ `);
});
Copy the code

In the above code, the Base64 encoding of the image is defined in the routing handler corresponding to the /file route. When the server receives a file download request from the client, such as GET /file? Filename =body.png when HTTP/1.1, the filename parameter is retrieved from the ctx.query object. This parameter indicates the name of the file. After obtaining the file name, we can concatenate the absolute path of the file and then read the contents of the file through the node.js fs.readFileSync method, which returns a Buffer. After successfully reading the contents of the file, we proceed to Base64 encoding the contents of the file by calling the toString method of the Buffer object, and the downloaded image is returned to the client in Base64 format.

Base64 format download example: base64

Github.com/semlinker/f…

8. Chunked download

Block transfer encoding is primarily used in scenarios where a large amount of data is being transmitted, but the length of the response is not available until the request has been processed. For example, when you need to generate a large HTML table with data from a database query, or when you need to transfer a large number of images.

To use chunked Transfer Encoding, you need to configure the transfer-encoding field in the response header and set it to either chunked or gzip, chunked:

Transfer-Encoding: chunked
Transfer-Encoding: gzip, chunked
Copy the code

The value of the transfer-encoding field in the response header is chunked, indicating that data is sent in a series of chunks. Note that the transfer-encoding and Content-Length fields are mutually exclusive. That is, they cannot be displayed at the same time in the response packet. Let’s look at the encoding rules for block transfers:

Each block contains two parts: block length and data block.
The block length is expressed as a hexadecimal number\r\nAt the end.
The data block followed by the block length is also used\r\nEnd, but the data is not included\r\n;
A termination block is a regular partition that indicates the end of a block. The difference is that its length is zero, which is zero0\r\n\r\n.

After understanding the coding rules of block transfer, let’s see how to use block transfer coding to achieve file download.

8.1 Front-end Code

html5

<h3>Chunked download example</h3>
<button onclick="download()">download</button>
Copy the code

const chunkedUrl = "http://localhost:3000/file? filename=file.txt";

function download() {
  return fetch(chunkedUrl)
    .then(processChunkedResponse)
    .then(onChunkedResponseComplete)
    .catch(onChunkedResponseError);
}

function processChunkedResponse(response) {
  let text = "";
  let reader = response.body.getReader();
  let decoder = new TextDecoder();

  return readChunk();

  function readChunk() {
    return reader.read().then(appendChunks);
  }

  function appendChunks(result) {
    let chunk = decoder.decode(result.value || new Uint8Array(), {
      stream: !result.done,
    });
    console.log("Data received:", chunk);
    console.log("This time has been successfully received.", chunk.length, "bytes");
    text += chunk;
    console.log("All received so far.", text.length, "bytes\n");
    if (result.done) {
      return text;
    } else {
      returnreadChunk(); }}}function onChunkedResponseComplete(result) {
  let blob = new Blob([result], {
    type: "text/plain; charset=utf-8"}); saveAs(blob,"hello.txt");
}

function onChunkedResponseError(err) {
  console.error(err);
}
Copy the code

The Download function in the above code is called when the user clicks the download button. Inside the function, we will use the Fetch API to perform the download operation. Because the data is sent as a series of chunks on the server side, we receive it as a stream on the browser side. ReadableStream: ReadableStream.getreader (); readableStream.getreader (); ReadableStream: ReadableStream.getreader (); readableStream.getreader ();

Because the contents of the file.txt file are plain text and the value of result.value is Uint8Array data, we use TextDecoder when processing the returned chunk of data. A decoder supports only one particular text encoding, such as UTF-8, ISO-8859-2, KOI8, CP1261, GBK, and so on.

If the received chunk is non-terminating and the result.done value is false, then the readChunk method is called to read the chunk. When the termination block is received, the block data transmission is complete. At this point, the result.done property returns true. Thus will automatically call onChunkedResponseComplete function, inside this function, we after decoding the text as a parameter to create a Blob object. After that, you continue to download files using the saveAs method provided by the FileSaver library.

Here we use the Wireshark network packet analysis tool, captured a packet. The details are shown in the figure below:

The HTTP chunked response contains Data chunk and End of chunked encoding. Next, let’s look at the server side code.

8.2 Server Code

const fs = require("fs");
const path = require("path");
const Koa = require("koa");
const cors = require("@koa/cors");
const Router = require("@koa/router");

const app = new Koa();
const router = new Router();
const PORT = 3000;

router.get("/file".async (ctx, next) => {
  const { filename } = ctx.query;
  const filePath = path.join(__dirname, filename);
  ctx.set({
    "Content-Type": "text/plain; charset=utf-8"}); ctx.body = fs.createReadStream(filePath); });// Register the middleware
app.use(async (ctx, next) => {
  try {
    await next();
  } catch (error) {
    // ENOENT (no file or directory) : This is usually caused by a file operation, which indicates that no files or directories can be found on the given path
    ctx.status = error.code === "ENOENT" ? 404 : 500;
    ctx.body = error.code === "ENOENT" ? "File does not exist" : "Server wandering"; }}); app.use(cors()); app.use(router.routes()).use(router.allowedMethods()); app.listen(PORT,() = > {
  console.log('The application has been started: http://localhost:${PORT}/ `);
});
Copy the code

In the /file routing processor, we first get the filename from the cx. Query, then concatenate the absolute path to the file, and then create a readable stream using the fs.createreadStream method provided by the Node.js platform. Finally, the created readable stream is assigned to the ctx.body property to return the image data to the client.

Now that we know that we can block data using transfer-encoding, is there any way to get files in a specified range? For this problem, we can use the HTTP protocol for scope requests. Next, we’ll show how to use HTTP scope requests to download data for a specified scope.

Chunked download example: chunked

Github.com/semlinker/f…

Nine, scope download

HTTP protocol-scoped requests allow the server to send only a portion of the HTTP message to the client. Range requests are useful when transferring large media files, or when used in conjunction with breakpoint continuation for file downloads. If the Accept-Ranges head exists in the response (and it does not have a value of “None”), then the server supports range requests.

In a Range header, you can request more than one part at a time, and the server returns it as a multipart file. If the server returns a range response, use the 206 Partial Content status code. If the requested Range is Not valid, the server will return the 416 Range Not Satisfiable status code, indicating that the client has an error. The server is allowed to ignore the Range header and return the entire file with a status code of 200.

The Range of grammar:

Range: <unit>=<range-start>-
Range: <unit>=<range-start>-<range-end>
Range: <unit>=<range-start>-<range-end>, <range-start>-<range-end>
Range: <unit>=<range-start>-<range-end>, <range-start>-<range-end>, <range-start>-<range-end>
Copy the code

unit: The unit used for a range request, usually bytes.
<range-start>: An integer representing the start value of a range in a specific unit.
<range-end>: An integer representing the end of the range in a specific unit.This value is optional, and if it does not exist, the range extends all the way to the end of the document.

Now that we’ve seen the Range syntax, let’s look at a practical example:

#Scope of a single
$ curl http://i.imgur.com/z4d4kWk.jpg -i -H "Range: bytes=0-1023"
#Multiple range
$ curl http://www.example.com -i -H "Range: bytes=0-50, 100-150"
Copy the code

9.1 Front-end Code

html

<h3>Scope download example</h3>
<button onclick="download()">download</button>
Copy the code

async function download() {
  try {
    let rangeContent = await getBinaryContent(
      "http://localhost:3000/file.txt".0.100."text"
    );
    const blob = new Blob([rangeContent], {
      type: "text/plain; charset=utf-8"}); saveAs(blob,"hello.txt");
  } catch (error) {
    console.error(error); }}function getBinaryContent(url, start, end, responseType = "arraybuffer") {
  return new Promise((resolve, reject) = > {
    try {
      let xhr = new XMLHttpRequest();
      xhr.open("GET", url, true);
      xhr.setRequestHeader("range".`bytes=${start}-${end}`);
      xhr.responseType = responseType;
      xhr.onload = function () {
        resolve(xhr.response);
      };
        xhr.send();
    } catch (err) {
        reject(new Error(err)); }}); }Copy the code

The Download function is called when the user clicks the download button. The range request is made inside this function by calling the getBinaryContent function. The corresponding HTTP request packet is as follows:

GET /file.txt HTTP / 1.1
Host: localhost:3000
Connection: keep-alive
User-Agent: Mozilla / 5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36Accept: * / *Accept-Encoding: identity
Accept-Language: zh-CN,zh; Q = 0.9, en. Q = 0.8, id; Q = 0.7Range: bytes=0-100
Copy the code

When the server receives the request in this range, it will return the corresponding HTTP response packet:

HTTP / 1.1 206 Partial Content
vary: Origin
Access-Control-Allow-Origin: null
Accept-Ranges: bytes
Last-Modified: Fri, 09 Jul 2021 00:17:00 GMT
Cache-Control: max-age=0
Content-Type: text/plain; charset=utf-8
Date: Sat, 10 Jul 2021 02:19:39 GMT
Connection: keep-alive
Content-Range: bytes 0-100/2590
Content-Length: 101
Copy the code

From the above HTTP response packets, we see the 206 status code and accept-ranges header introduced earlier. In addition, from the Content-range header, we know the total size of the file. After successfully obtaining the response body of the scope request, we can call the Blob constructor to create the corresponding Blob object, using the saveAs method provided by the FileSaver library to download the file.

9.2 Server Code

const Koa = require("koa");
const cors = require("@koa/cors");
const serve = require("koa-static");
const range = require("koa-range");

const PORT = 3000;
const app = new Koa();

// Register the middleware
app.use(cors());
app.use(range);
app.use(serve("."));

app.listen(PORT, () = > {
  console.log('The application has been started: http://localhost:${PORT}/ `);
});
Copy the code

The code on the server side is relatively simple, and the range request is implemented through the KOA-Range middleware. Because space is limited, brother a bao will not expand the introduction. Interested partners, you can read the source code of the middleware. In fact, scope request can also be applied to large file download scenarios, if the file server supports scope request, when downloading large files, the client can consider using large file block download scheme.

Scope download example: range

Github.com/semlinker/f…

10. Large files are downloaded in blocks

Believe some friend already know large file upload solution, when uploading large files, in order to improve the efficiency of the upload, we usually use the Blob. The slice method to cut large files according to the specified size, and then open in a multithreaded region upload, after all parts are uploaded successfully, then block merger notification service.

Can we apply a similar idea to large file downloads? In fact, under the condition that the server supports Range request header, we can also implement the function of large file block download. The specific processing scheme is shown in the figure below:

Because how do you do concurrent large file downloads in JavaScript? In this article, The big file concurrent download scheme has been introduced in detail, so I won’t go into it here. Let’s just review the complete process for concurrent large file downloads:

The async-pool library is used to implement concurrency control in large file block downloads. The library provides two different versions of ES7 and ES6 implementation, the code is very concise and elegant. If you want to know how async-pool implements concurrency control, you can read how to implement concurrency Control in JavaScript. This article.

Example for downloading large files in blocks: big-file

Github.com/semlinker/f…

Xi. Summary

After reading this article, I hope you have some idea of the technology behind the nine scenarios for downloading files. In fact, in the process of transferring files, in order to improve the transmission efficiency, we can use gZIP, Deflate or BR compression algorithm to compress files. Because of the limited space, I won’t go into this, but if you’re interested, you can read this article about several ways to transfer large files over HTTP.

With a file download scenario, how can there be no file upload scenario? If you haven’t read file uploading yet, these 8 scenarios are enough to understand this article, I suggest you have a free time, can learn about together. Thank you again for your continued support, and if you want to know more about it, please leave a message to Po.

12. Reference Resources

MDN — showSaveFilePicker
MDN – Content – Disposition
The File System Access API: simplifying access to local files
Reading and writing files and directories with the browser-fs-access library
File upload, understand these 8 scenarios is enough
How to achieve large file concurrent download in JavaScript?

File download, understand these 9 scenarios is enough

First, basic knowledge

1.1 understand the Blob

1.2 Understanding the Blob URL

2. A tag download

ShowSaveFilePicker API download

You can download FileSaver files

Five, Zip download

Vi. Download in the form of attachments

7. Base64 download

7.1 Front-end Code

7.2 Server Code

8. Chunked download

8.1 Front-end Code

8.2 Server Code

Nine, scope download

9.1 Front-end Code

9.2 Server Code

10. Large files are downloaded in blocks

Xi. Summary

12. Reference Resources

Related Posts

How do I obtain the IP address assigned locally by the current network in a VUE project?

Vue3: Combine apI-Reactive functions

Front – end common algorithm (list)- next