🌰 : There is a requirement that the front-end needs to upload a CSV file [encoding format is variable], and then read the contents of the file to process ✏️ : Because file encoding format, if the front-end processing directly, have the code of [spreadsheet software is different, there are differences between the encoding type is, you get even Korean 】, so need to look to deal with file use nodejs to receive, then the processed data is returned to the front to operation, ensure the accuracy of the data and unity.

Plugins needed

  • Co – busboy – www.npmjs.com/package/co-… [Processing multipart data]
  • Chardet – www.npmjs.com/package/cha… [Possible encoding types of detection data]
  • Iconv – lite – www.npmjs.com/package/ico… Convert data encoding type to UTF-8

How to get files into NodeJs?

Use it directly in the front endInput of type fileTag to get file data

  • Define the associated tags
<input id="file" type="file" name="file"></input>
<button id="btn">upload</button>
Copy the code
  • The file data is directly sent to the Node for processing
const oBtn = document.querySelector('#btn');
const oFile = document.querySelector('#file');

oBtn.onclick = function() {
  const file = oFile.files[0];
  const formData = new FormData();
  formData.append('some title', file);

  let xhr = new XMLHttpRequest();
  xhr.onload = function() {
    console.log('Upload successful');
  }

  xhr.open('post'.'/api/data'.true);
  
  // You do not need to set the content-type header, otherwise you will lose the boundary data and cause an error
  // xhr.setRequestHeader('Content-Type', 'multipart/form-data'); 
  
  xhr.send(formData);
}
Copy the code

On the Node side, the data is received and usedco-busboyThe data is processed by the plug-in

  • My Node end here is usingKoaTo start the service, other similar processing
// Listen for routes
router.post('/api/data', dealData);

-----------------------------------------------------------------

// To save the data to a directory
const parse = require('co-busboy');
const fs = require('fs');
const path = require('path');
const sendToWormhole = require('stream-wormhole');

async function dealData(ctx, next) {
    const saveFile = function(ctx) {
    	return new Promise(async (resolve, reject) => {
            const stream = await parse(ctx)();
            const fileName = stream.filename;
            const filePath = path.join(__dirname, 'csv', fileName);
            const ws = fs.createWriteStream(filePath);

            ws.on('finish'.() = > {
              console.log('Saved successfully: the path is${filePath}`); resolve({ filePath, fileName, stream }); }) stream.pipe(ws); })}const unlinkFile = filepath= > {
    	fs.unlink(filePath, (err) = > {
            console.log(`${filePath}The file has been deleted);
        });
    }
    
    try {
    	const { filePath, fileName, stream } = await saveFile(ctx);
    	// TODO:Processing file data
    } catch(e) {
    	// Consume the stream
    	await sendToWormhole(stream);
    } finally{ unlinkFile(filepath); }}Copy the code

Through the above operations, the front-end upload file can be saved to the specified place on the server. The next thing to do is to process the file data into a unified format and return it.

Two, how to file data into a unified format?

I wrote an article about this juejin.cn/post/686434…