This is the seventh day of my participation in the August More text Challenge. For details, see:August is more challenging
preface
- In another article JavaScript Advanced – Nodejs + KOA2 Implementation file Upload Large File slice Upload Breakpoint Continuation (server side), we have introduced the server side implementation of file upload and implemented the server side interface for several different forms of upload
- In the following article, we will step by step implement the upload functionality of the front-end (client). Without talking too much about HTML and CSS, I’ll focus on the JS code implementation.
- Let’s take a look at the renderings
I. Environment preparation and dependent libraries
- Axios V0.21.1: Used to invoke the server side interface to send server side requests
- Spark-md5 v3.0.1: used to generate hash codes based on file content
- Qs V6.9.6: Used to parse application/ X-wwW-form-urlencoded format from parameters to a= x&B = Y format
Ii. Project structure
Web project root directory
- Scripts Directory for storing JS scripts
- Axios.min.js Axios library (third party)
- Qs.js QS Library (third party)
- Spark-md5.min. js Spark-MD5 library (third party)
- Axios 2. Js Axios twice encapsulates library (custom),
- Upload.js File upload function code (custom)
- Directory of CSS style files
- Upload. CSS Page style
- The index.html file is uploaded to the HTML page
Iii. Functional realization
- Combined with the screenshot above, it will be divided into 5 modules to explain, and the upload control used by all modules is the INPUT control with the native HTML type of File.
- For the sake of page aesthetics, we hide the input and replace it with a normal button that fires the input click event when clicked.
- In addition, you can add some additional progress display, picture thumbnail display, file name display, etc.
- There is no more explanation about HTML and CSS, the following will be divided into modules to focus on THE JS part, each module is wrapped with closure functions, to avoid variable conflict
1. Axios secondary encapsulation
In each function module, we will be sending requests to the server using Axios, and we will need to do something special with Axios, namely secondary encapsulation
- Create axios objects to avoid configuration conflicts in different scenarios
- Set the baseURL
- Set the default content-type to multipart/form-data
- Judge content-type in transformRequest; if it is Application/X-wwW-form-urlencoded, qs library will be used to format parameters
// Axios.js Encapsulates axios twice
let request = axios.create();
request.defaults.baseURL = 'http://127.0.0.1:3000';
request.defaults.headers['Content-Type'] = 'mutipart/form-data';
request.defaults.transformRequest = (data, headers) = > {
let contentType = headers['Content-Type'];
if (contentType === 'application/x-www-form-urlencoded') return Qs.stringify(data);
return data;
}
request.interceptors.response.use(response= > {
return response.data;
});
Copy the code2. Upload a single file FROM-DATA. Select the file first and then upload it
- Simple step analysis:
- You should first get the page elements you need: upload control input, select button, upload button, thumbnail display, file display, progress bar display
- Bind the click event of the select button and trigger the Click event of the upload control input in the click event
- Bind the change event to the input of the upload control to get the selected file
- Bind the click event of the upload button, in which the parameters are combined and a POST request is sent calling the server API to implement file upload
- The key code for file upload is the parameter concatenation section before sending the request
- Here we use js’s built-in FromData class to transfer files as arguments
- new FormData().append(“file”, file);
- Code implementation
//upload.js upload a single file to form-data
(function () {
let upload1 = document.querySelector("#upload1"),
upload_inp = upload1.querySelector('.upload-inp'),
upload_select = upload1.querySelector('.upload-btn.select'),
upload_upload = upload1.querySelector('.upload-btn.upload'),
sel_files = upload1.querySelector('.files'),
file1 = upload1.querySelector('.abbr'),
cur_pro = upload1.querySelector('.cur-pro'),
pro_val = upload1.querySelector('.pro-val'),
progress = upload1.querySelector('.progress'),
_file;
upload_select.addEventListener('click'.() = > {
upload_inp.click();
});
upload_inp.addEventListener('change'.function () {
let file = this.files[0];
_file = file;
sel_files.innerHTML = file.name;
progress.style.display = 'inline-block';
pro_val.innerHTML = ' ';
})
upload_upload.addEventListener('click'.function () {
let formData = new FormData();
formData.append('file', _file);
formData.append('filename', _file.name);
request.post('/upload_single_file', formData, {
onUploadProgress: function (ev) {
let pro = ((ev.loaded / ev.total) * 100).toFixed(0) + The '%';
cur_pro.style.width = pro;
pro_val.innerHTML = pro;
}
}).then(res= > {
console.log(res);
file1.src = `http://${res.serverPath}`;
file1.style.display = 'block';
}).catch(err= > {
console.log(err); }); }); }) ();Copy the code3. If BASE64 is uploaded as a single file, only PNG or JPG image files smaller than 100K can be uploaded
- Simple step analysis:
- First, you should get the page elements you need: upload control input, select the upload button, display thumbnail, file name, and progress bar
- Binds the click event for the select and upload button and fires the Click event for the upload control input within the Click event
- Since the select and upload buttons are combined, the step of sending the request to the server is placed in the change event of the input
- Bind the change event of input, in which parameters are combined and a POST request is sent calling the server API to implement file upload
- In the change event in the file field, we need
- According to the obtained file information, verify that the file size cannot exceed 100K (the size of a file uploaded in base64 format cannot be too large)
- The verification file can only be in PNG or JPG format and has the file.type attribute
- Convert files to BASE64 format using js’s built-in FileReader class
- And in filereader’s onLoad function to get converted to BASE64 format content and send a POST request to achieve file upload
- In addition, because the content of BASE64 format involves a lot of characters, in order to avoid some special character problems, encodeURIComponent should be used for encoding before parameter passing
- Code implementation
//upload.js Upload a single base64 file
(function () {
let upload2 = document.querySelector("#upload2"),
upload_inp = upload2.querySelector('.upload-inp'),
upload_upload = upload2.querySelector('.upload-btn.upload'),
sel_files = upload2.querySelector('.files'),
file2 = upload2.querySelector('.abbr'),
progress = upload2.querySelector('.progress'),
cur_pro = upload2.querySelector('.cur-pro'),
pro_val = upload2.querySelector('.pro-val'),
_file;
upload_upload.addEventListener('click'.() = > {
upload_inp.click();
});
upload_inp.addEventListener('change'.function () {
progress.style.display = 'inline-block';
pro_val.innerHTML = ' ';
let file = this.files[0];
_file = file;
if (file.size > 100 * 1024) {
alert('Picture must be less than 100K');
return;
}
if (!/(jpg|jpeg|png)/.test(file.type)) {
alert('Upload only PNG, JPG or JPEG images');
return;
}
sel_files.innerHTML = file.name;
let fr = new FileReader();
fr.readAsDataURL(file);
fr.onload = ev= > {
file2.src = ev.target.result;
file2.style.display = 'block';
console.log(file.name);
request.post('/upload_base64', {
file: encodeURIComponent(ev.target.result),
filename: file.name
}, {
headers: {
"Content-Type": "application/x-www-form-urlencoded"
},
onUploadProgress: function (ev) {
let pro = ((ev.loaded / ev.total) * 100) + The '%';
pro_val.innerHTML = pro;
cur_pro.style.width = pro;
}
}).then(res= > {
console.log(res);
alert('Upload successful');
return;
}).catch(err= > {
console.log(err);
alert('Failed? ')}); }; })}) ();Copy the code4. Upload form-data to multiple files
- Simple step analysis:
- The steps for multi-file upload are similar to those for single-file form-data
- First, get the page element you need: the upload control input, and select the upload button
- Bind the click event of the select button and trigger the Click event of the upload control input in the click event
- Bind the change event of the input of the upload control, obtain the selected file in the event, and traverse the selected file combination parameters to send a POST request and invoke the server API to realize file upload
- Instead of a single file form-data upload, what we get is a list of files
- In the change event you need to iterate over all the selected files, combine the parameters and then loop the request to invoke the server API interface for multiple file uploads
- Since there is no fixed number of files to upload, you need to dynamically splice the file name and upload progress into HTML elements and display them on the page after traversing the selected files
- Code implementation
// upload.js Upload multiple files to form-data
(function () {
let upload3 = document.querySelector("#upload3"),
upload_inp = upload3.querySelector('.upload-inp'),
upload_upload = upload3.querySelector('.upload-btn.upload'),
sel_files = upload3.querySelector('.list');
upload_upload.addEventListener('click'.() = > {
upload_inp.click();
});
upload_inp.addEventListener('change'.function () {
let files = this.files;
sel_files.innerHTML = ' ';
[].forEach.call(files, (file, index) = > {
sel_files.innerHTML += `<div><span class="files" style="margin-right:8px; font-size:12px">${file.name}</span><span class="pro-val" id="myfile${index}"></span></div>`
let formData = new FormData();
formData.append('file', file);
formData.append('filename', file.name);
request.post('/upload_single_file', formData, {
onUploadProgress: function (ev) {
let pro = ((ev.loaded / ev.total) * 100).toFixed(0) + The '%';
document.querySelector(`#myfile${index}`).innerHTML = pro;
// sel_files.innerHTML += `<span class="files">${file.name}</span> <span class="pro-val" >${pro}</span>`
}
}).then(res= > {
console.log(res);
// alert(' upload succeeded ');
}).catch(err= > {
console.log(err); }); }); }); }) ();Copy the code5. Multi-file drag-and-drop upload form-data
- Simple step analysis:
- In this module, the focus of implementing drag upload is on **== drag ==**, and the other upload steps are the same as the previous multi-file upload.
- In addition to drag-and-drop uploads, we also need to keep the original click-and-upload
- So first of all we need to get the upload control input, div block as the drag and drop area and the area that triggers the click event
- It then encapsulates the main upload logic into a separate method called uploadFiles that takes an array or array-like parameter to be called when clicked or drag-and-dropped
- Click upload:
- Here we need to bind the drag field (div) to a click event and fire the input click event within that event
- Bind the change event to the input, call to get the list of selected files, and call the upload method uploadFiles to implement file upload
- Drag upload:
- Drag uploads require two events, dragover and drop, which are events that are triggered when a file is dragged onto or falls onto a div
- Note that some files (TXT, JPG, PNG, etc.) will be displayed directly on the web page by default when dragging a file onto the web page, so we need to prevent the default behavior of the browser in both cases: ev.preventdefault ()
- And then in the drop event when we drag the file onto the div and release the mouse, we call uploadFiles to upload the file
- There is a dataTransfer property files attribute in the drop event to get a list of the files to drag in, so you can get all the multiple files via ev.datatransfer. files
- Code implementation
//upload.js drag and drop to upload form-data
(function () {
let upload5 = document.querySelector("#upload5"),
upload_inp = upload5.querySelector('.upload-inp'),
upload_upload = upload5.querySelector('.upload-btn'),
sel_files = upload5.querySelector('.list');
const uploadFiles = function uploadFiles(files) {
sel_files.innerHTML = ' ';
[].forEach.call(files, (file, index) = > {
sel_files.innerHTML += `<div><span class="files" style="margin-right:8px; font-size:12px">${file.name}</span><span class="pro-val" id="myfile${index}"></span></div>`
let formData = new FormData();
formData.append('file', file);
formData.append('filename', file.name);
request.post('/upload_single_file', formData, {
onUploadProgress: function (ev) {
let pro = ((ev.loaded / ev.total) * 100).toFixed(0) + The '%';
document.querySelector(`#myfile${index}`).innerHTML = pro;
}
}).then(res= > {
console.log(res);
// alert(' upload succeeded ');
}).catch(err= > {
console.log(err);
});
});
}
upload5.addEventListener('dragover'.function (ev) {
ev.preventDefault();
});
upload5.addEventListener('drop'.(ev) = > {
ev.preventDefault();
uploadFiles(ev.dataTransfer.files);
});
upload_inp.addEventListener('change'.function () {
uploadFiles(this.files);
});
upload5.addEventListener('click'.(ev) = >{ upload_inp.click(); }); }) ();Copy the code6. Upload large file slices and continue to transfer form-data at breakpoint
The next is the last module of this article, is also the most complex and the most important module: large file slice upload and breakpoint continuation, to achieve the slice upload and breakpoint continuation logic is a little more complex than the previous several modules, here we still step by step analysis:
- Simple logic analysis
- Slice upload, as the name implies, is to divide a large file into several small files and upload them separately. After all the slices are uploaded, they are merged into one file. In this way, the slice upload of a large file can be realized
- The key to a slice upload is to merge all the slices after the upload.
- Which files need to be merged?
- Find the files that need to be merged, in what order? => First of all, in order to find out which files need to be merged quickly and conveniently, we need to create a separate temporary folder on the server side to save all slices when uploading slice files to the server => Second of all, in order to ensure that the merged files are consistent with the original files, When slicing, you need to add an index to each slice so that the merge can be done in the order of the index.
- If there is a problem in the uploading process of the slice, which leads to the interruption of the upload, it is necessary to make a judgment in the next upload in order not to repeat the upload, which is also called resumable transfer. If the file exists, it will skip directly. Then how to judge whether a file (or slice) exists?
- In this case, we need to use the == spark-MD5 == library, which generates a string of hash values based on the contents of the file. As long as the contents of the file are unchanged, the generated hash values are always the same. Therefore, we can use the hash value and index to name the slices of the file
- Section analysis:
- To slice a file, use the file’s **==size== property and ==slice==** method
- Method 1 (fixed number) : cut a file into a fixed number, such as 20, then use size/20 to calculate the size of each slice file, and then use slice to cut
- Method 2 (fixed size) : fix the size of each slice file, say 100K, and then use size/100 to calculate the number of slices that need to be divided, again using slice
- In this case, we will adopt the combination of method 1 and method 2 to slice: We first fix the size of each slice according to method 2, calculate the number of slices, and then specify a maximum number. If the calculated number exceeds the maximum number, we need to slice again according to method 1. If the number is not exceeded, slice by fixed size.
- Simple step analysis
After teasing out the slicing logic, here are the steps to take:
- First, we encapsulate a method retrieveHash that returns a promise instance. This method is mainly used to generate a hash value according to the file content, with the help of Spark-MD5 and FileReader
- Encapsulate a merge request method uploadComplete sent after all slices have been uploaded
- You need to define a counter outside of this method, which is called every time a slice is uploaded, and which is added by 1 each time the method is called
- When the counter is equal to the number of slices, all slices have been uploaded and a merge request can be sent to merge slices
- One thing to note here is that before sending a merge request, it is best to delay it for a few seconds to avoid unnecessary errors
- Get the file to be uploaded in the change event in the file field
- RetrieveHash, the method retrieveHash encapsulated above, is called, and then a request is sent to the server based on the hash value to retrieve the filelist (used for breakpoint continue judgment) that has been uploaded.
- Slice according to the slice logic analyzed above, and save the slice file information in an array
- The first step in iterating through the slice array is to determine whether the slice has been uploaded. That is, whether the slice file already exists in the filelist obtained above
- If so, call uploadComplete and let the counters accumulate
- If it does not exist, then the server slice upload interface is called to upload the file. At the same time, the uploadComplete method is still called to accumulate the counter after the upload is completed. Once the counter value is equal to the number of slices, the merge interface is automatically called to merge the file
- So far the large file slice upload and breakpoint continue to achieve.
- Code implementation
//upload.js large file slice upload, breakpoint continue upload
(function () {
let upload4 = document.querySelector("#upload4"),
upload_inp = upload4.querySelector('.upload-inp'),
upload_upload = upload4.querySelector('.upload-btn'),
sel_files = upload4.querySelector('.files'),
cur_pro = upload4.querySelector('.cur-pro'),
pro_val = upload4.querySelector('.pro-val'),
progress = upload4.querySelector('.progress');
const retriveHash = function retriveHash(file) {
return new Promise((resolve, reject) = > {
let spark = new SparkMD5.ArrayBuffer();
let fr = new FileReader();
fr.readAsArrayBuffer(file);
fr.onload = (ev) = > {
spark.append(ev.target.result);
let hash = spark.end();
let suffix = /\.([0-9a-zA-Z]+)$/.exec(file.name)[1];
resolve({
hash,
suffix
});
};
});
}
let complete = 0;
const uploadComplete = function uploadComplete(hash, count) {
complete++;
let progerss = (complete / count * 100).toFixed(2) + The '%';
cur_pro.style.width = progerss;
pro_val.innerHTML = progerss;
if (complete < count) return;
cur_pro.style.width = '100%';
pro_val.innerHTML = '100%';
setTimeout(() = > {
request.post('/upload_merge', {
hash,
count
}, {
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
}
}).then(res= > {
console.log(res);
// alert(' upload succeeded ');
}).catch(err= > {
console.log(err);
});
}, 3000);
}
upload_upload.addEventListener('click'.function () {
upload_inp.click();
});
upload_inp.addEventListener('change'.async function () {
let file = this.files[0];
progress.style.display = 'inline-block';
cur_pro.style.width = '0%';
pro_val.innerHTML = '0%';
let chunks = [];
let {
hash,
suffix
} = await retriveHash(file);
sel_files.innerHTML = `${hash}.${suffix}`;
let {
filelist
} = await request.get('/uploaded', {
params: {
hash
}
});
let maxSize = 100 * 1024; //100k
let count = Math.ceil(file.size / maxSize);
// Limit the number of slices to 20 and recalculate the size of each slice
if (count > 20) {
maxSize = file.size / 20;
count = 20;
}
let index = 0;
while (index < count) {
chunks.push({
file: file.slice(index * maxSize, (index + 1) * maxSize),
filename: `${hash}_${index+1}.${suffix}`
});
index++;
}
chunks.forEach((item, index) = > {
// If it has already been uploaded, it will not be uploaded again
if (filelist && filelist.length > 0 && filelist.includes(item.filename)) {
uploadComplete(hash, count);
return;
}
let formData = new FormData();
formData.append('file', item.file);
formData.append('filename', item.filename);
request.post('/upload_chunk', formData).then(res= > {
uploadComplete(hash, count);
// console.log(res);
// alert(' upload succeeded ');
}).catch(err= > {
console.log(err); }); }); }); }) ()Copy the codeconclusion
In this paper, we realized several functional points of end-file upload by module and scenario, and focused on analyzing the functional points of large file slice upload and breakpoint continuation. The functional code for each module is provided.