Recently, the company’s project needs to deal with the situation of file uploading. There are some problems in local development, which will be solved after investigation. Therefore, I became interested in the principle of file uploading, so I studied the situation of file uploading handled by native NodeJS.

A few important concepts

Content-Type

Content-type The entity header is used to indicate the MEDIA Type of the MIME Type of the resource. In the response, the Content-Type header tells the client the Content Type of the actual returned Content. In a request (such as POST or PUT), the client tells the server what type of data is actually being sent. The Content-Type consists of three parts

  • Media-type Specifies the MIME type of the resource or data. For example, application/json
  • Charset character encoding standard. For example: utf-8
  • Boundary A boundary is required for multi-part entities, consisting of 1 to 70 characters from a set of characters.

Content-Disposition

In a regular HTTP reply, the Content-Disposition response header indicates whether the Content of the reply should be presented inline (i.e. as a web page or part of a page) or downloaded and saved locally as an attachment

In a multipart/form-data reply message body, the Content-Disposition header can be used in a sub-part of the multipart message body to give information about its corresponding field. Each subpart is separated by a delimiter defined in the Content-Type. The message body itself has no practical meaning.

Content-disposition headers were originally defined in the MIME standard, and HTTP forms and POST requests use only a subset of all their parameters. Only form-data and optional name and filename can be used in HTTP scenarios.

Actual Scenario Application

TXT file, the content of the test::: file.

We need to set the content-type of the message to the server to be multipart/form-data. In the form we can specify the encType value to be multipart/form-data

http .createServer(function(req, Res) {if (req.url == "/upload" &&req.method.tolowerCase () === "get") {// Display a form res.writehead (200, { "content-type": "text/html" }); res.end( '<form action="/upload" enctype="multipart/form-data" method="post">' + '<input type="file" name="upload" multiple="multiple" />' + '<input type="submit" value="Upload" />' + "</form>" ); } }) .listen(3000);Copy the code

We can access the form by accessing localhost:3000/upload in our browser.



Then nodeJS adds the logic to upload files

If (req.url == "/upload" && req.method.tolowerCase () === "get") {// Display a form res.writehead (200, { "content-type": "text/html" }); res.end( '<form action="/upload" enctype="multipart/form-data" method="post">' + // '<input type="text" name="description" value="multiple" />' + '<input type="file" name="upload" multiple="multiple" />' + '<input type="submit" value="Upload" />' + "</form>" ); } else if (req.url == "/upload" && req.method.toLowerCase() === "post") { if (req.headers["content-type"].indexOf("multipart/form-data") ! == -1) var body = '' req.on('data', (buffer) => { body += buffer }) req.on('end', () => { res.end(JSON.stringify({ contentType: req.headers['content-type'], content: body.toString() })); })}Copy the code

Click select File select temp. TXT and then click Upload. The page then displays the following

    {
        "contentType":"multipart/form-data; boundary=----WebKitFormBoundaryflF412CVstMQDH5m",
        "content":"------WebKitFormBoundaryflF412CVstMQDH5m\r\nContent-Disposition: form-data; name=\"upload\"; filename=\"temp.txt\"\r\nContent-Type: text/plain\r\n\r\ntest:::\r\n------WebKitFormBoundaryflF412CVstMQDH5m--\r\n"
    }
Copy the code

Req.headers [‘content-type’] has been printed out with the entity. You can see that the contentType has a boundary, which is the same value as the string inside the content. When we upload a file, we receive the value of content-Type in headers and we have boundary, boundary is used to separate form fields, so if you have multiple content in your form you have multiple values in your content. Like this

    ------WebKitFormBoundarymrD3BuxNQlp4oQ2R
    Content-Disposition: form-data; name="description"

    multiple
    ------WebKitFormBoundarymrD3BuxNQlp4oQ2R
    Content-Disposition: form-data; name="upload"; filename="temp.txt"
    Content-Type: text/plain

    test:::
    ------WebKitFormBoundarymrD3BuxNQlp4oQ2R--
Copy the code

Test ::: is the content of our file. Content-disposition contains our filename. So to implement our file upload function, we just need to take our file name and file Content based on boundary and Content-Disposition and write the file Content into that file name. Below is the complete code, cleaned up

const http = require("http"); const fs = require("fs"); const querystring = require("querystring"); Http. createServer(function(req, Res) {if (req.url == "/upload" &&req.method.tolowerCase () === "get") {// Display a form res.writehead (200, { "content-type": "text/html" }); res.end( '<form action="/upload" enctype="multipart/form-data" method="post">' + '<input type="file" name="upload" multiple="multiple" />' + '<input type="submit" value="Upload" />' + "</form>" ); } else if (req.url == "/upload" && req.method.toLowerCase() === "post") { if (req.headers["content-type"].indexOf("multipart/form-data") ! == -1) { parseFile(req, res) } } else { res.end("pelease upload file"); } }) .listen(3000); function parseFile(req, res) { req.setEncoding("binary"); let body = ""; // let fileName = ""; // filename // boundary const boundary = req.headers["content-type"].split("; ")[1] .replace("boundary=", ""); console.log('req.headers["content-type"]', req.headers["content-type"]); req.on("data", function(chunk) { body += chunk; }); req.on("end", function() { const file = querystring.parse(body, "\r\n", ":"); console.log('file:::', JSON.stringify(file)); var fileInfo = file["Content-Disposition"].split("; "); for (value in fileInfo) { if (fileInfo[value].indexOf("filename=") ! = -1) { console.log('fileInfo[value]', fileInfo[value]); fileName = fileInfo[value].substring(10, fileInfo[value].length - 1); if (fileName.indexOf("\\") ! = -1) { fileName = fileName.substring(fileName.lastIndexOf("\\") + 1); } console.log(" fileName: "+ fileName); } } const entireData = body.toString(); contentType = file["Content-Type"].substring(1); Const upperBoundary = entireData.indexof (contentType) + contentType.length; const shorterData = entireData.substring(upperBoundary); console.log('shorterData', shorterData); // Replace start space const binaryDataAlmost = shorterData.replace (/^\s\s*/, "").replace(/\s\s*$/, ""); console.log('binaryDataAlmost', binaryDataAlmost); // Remove the extra data at the end of the data, i.e. : "--"+ boundary + "--" const binaryData = binaryDataAlmost.substring( 0, binaryDataAlmost.indexOf("--" + boundary + "--") ); const bufferData = new Buffer.from(binaryData, "binary"); console.log("bufferData", bufferData); fs.writeFile(fileName, bufferData, function(err) { res.end("sucess"); }); }); }Copy the code

conclusion

The whole process of file uploading is easy to understand. When uploading a file, you only need to set the content-Type to multipart/form-data. Then the server gets the headers boundary and processes the file content according to this. The above is the nodeJS native file upload process, which is slightly complicated. Koa now handles file uploads conveniently in conjunction with KOA-body. The corresponding processing process will be supplemented later

reference

content-type

content-disposition