preface
Business to achieve a large file slice, breakpoint continued function, access to the public account material management, H5 configuration to share picture title, in order to facilitate access, so here to make a record.
This article will document how the front end and back end work together to achieve functional demo. (The code is based on business requirements, so it’s not intended for the general public.)
Front end: Vue Element – UI Axios
The backend: an egg
Large file upload
Train of thought
The front end
Front-end Large file sharding We believe there are many solutions available online, most of which use Slice to slice large files into the desired size. Then with the help of HTTP concurrency, to achieve in the shortest time to upload large files. Because of concurrency, the order of upload may be different, so each small slice needs to be marked.
The back-end
The server receives the slices and merges the corresponding slices. When to merge depends on whether the front end requests the merged interface.
The front part
Use element-UI here to make the UI nice.
File upload
<el-col :span="12"> <el-progress :percentage="uploadPercentage"></el-progress> </el-col> <el-col :span="24"> <input type="file" @change="handleFileChange" accept="image/gif, image/jpeg"> </el-col> <el-col :span="24" style="margin-top: 10px; Text-align :right"> <el-button type="success" size="mini" @click="submitUpload"> </el-button> </el-col> <el-button </el-button> <el-button type="info" size="mini" @click="cancelLoad"> @click="handleResume"> </el-button> export default {... HandleFileChange (e) {const [file] = e.target.files if (! file) { return } this.file = file } ... }Copy the codeUpload section
Then the more important part, the long pass needs to do two parts:
- Slice large files
- Send slices to the server
let SIZE = 10 * 1024 * 1024
// Click Upload
async submitUpload() {
// Large file slices
const fileChunkList = this.createFileChunk(this.file)
this.data = fileChunkList.map(({ file }, index) = > ({
chunk: file,
hash: `The ${this.radio} - ${index}`.// Each slice name is marked with filename - index (this.radio is the filename)
index,
percentage: 0 // Progress bar, explained later
}))
await this.uploadChunks() // Concurrent requests
}
// Large file slices
createFileChunk(file, size = SIZE) {
const fileChunkList = []
let cur = 0
while (cur < file.size) {
fileChunkList.push({ file: file.slice(cur, cur + size) })
cur += size
}
return fileChunkList
}
// Upload slices
async uploadChunks(uploadedList = []) {
const requestList = this.data
.filter(({ hash }) = >! uploadedList.includes(hash))// This function can be ignored
.map(({ chunk, hash }) = > {
const formData = new FormData()
formData.append(hash, chunk)
return { formData }
})
.map(({ formData }, index) = > {
// api.upload is a self-wrapped AXIos request, as explained later
return API.Upload({
data: formData,
f: this.createProgressHandler(this.data[index]), // The progress bar can be ignored here
requestList: this.requestList, // The number of HTTP requests in progress, again, can be ignored first
formData: true // Whether content-type is form-data})})await Promise.all(requestList)
}
Copy the codeWhen we click the upload button, we split the large file with createFileChunk method, where each slice is 10M (the last slice is not 10M), and then use map to add the necessary information for each slice. Chunk is a binary file. Hash is used to identify the file name and index, so that the back end knows how to merge in sequence. Percentage is the percentage bar. If no progress bar is required, you can ignore it.
After adding the necessary information to each slice, a concurrent request is followed, in the uploadChunks method,.filter(({hash}) =>! Uploadedlist.includes (hash) is a breakpoint upload function that can be ignored for now. Put the chunk hash into FormData(), and then use promise.all() to make concurrent requests.
Request to merge
. methods: { ... async mergeRequest() {const res = await API.MergeFile({
data: {
size: SIZE,
fileName: this.radio / / file name}})if (res) {
this.$message.success('Upload successful')}},... },watch: {
requestList() {
if (this.requestList.length === 0 && this.file) {
this.mergeRequest()
}
}
...
Copy the codeThe watch section, which listens to see if the number of requests becomes zero, is a function of subsequent breakpoint continuation and progress bar sections. If not necessary, you can put this.mergerequest () after the Promise of uploadChunks.
The backend part
Here I use the mode:file of the egg-multipart inside the egg.
// config/config.default.js
config.multipart = {
mode: 'file'.fileSize: '50mb'.fileExtensions: [
' ']};Copy the code// controller/picture.js.// Upload interface method
async updatePicture() {
const { ctx, app } = this
const { writeFormFile } = app.lib / / (1)
const file = ctx.request.files[0]
try {
await writeFormFile.UploadDateFile(file)
ctx.success(true) / / (2)
} catch (err) {
ctx.fail(err) / / (3)
} finally {
ctx.cleanupRequestFiles()
}
}
...
Copy the code(1) (2) (3) (1) (2) (3) You need to write it according to the project. Ctx.cleanuprequestfiles (), on the other hand, cleans the cached files within the project after each completion. This is important to avoid unwanted files accumulating.
// app/lib/writeFormFile.js
const fse = require('fs-extra')
const path = require('path')
const UploadDateFile = async file => {
let { filepath, fieldname } = file
let name = fieldname.replace(/\s\-\s[0-9]*$/.' ') // The hash file is named filename - index, so remove (-index)
const tempPath = fse.readFileSync(filepath) // Read temporary file information
const { serverPath, loadPath } = await GenerateFileNameAndPath(name)
try {
await fse.accessSync(loadPath, fse.constants.R_OK | fse.constants.W_OK)
} catch (err) {
console.error('error err, err)
await fse.mkdirSync(loadPath)
}
const newPath = await path.join(loadPath + '/' + fieldname)
// Write temporary files
await fse.writeFileSync(newPath, tempPath)
return {
path: serverPath,
fullPath: loadPath
}
}
// Generate file name and load path, rewrite according to your own situation
const GenerateFileNameAndPath = async (name) => {
// Load path combination
const rootPath = path.resolve(__dirname, '.. /.. / '.'public')
const serverPath = '/upload/images/stzz/temp/'
const loadPath = rootPath + serverPath + name
return {
serverPath,
loadPath
}
}
...
Copy the codeAt this point, the back-end interface for receiving shards is complete, followed by the front-end request merge.
// controller/picture.js. async mergePicture() {const { ctx, app } = this
const { writeFormFile } = app.lib
try {
const params = ctx.joi({ / / (1)
fileName: Joi.string().required(),
size: Joi.number().integer().required()
})
await writeFormFile.MergeFile(params.fileName, params.size)
ctx.success(true)}catch (err) {
ctx.fail(err)
}
}
...
Copy the codeSimilarly, (1) belongs to its own project. In the front part, you may have noticed that we passed in the SIZE parameter, because in the merge process, if there is an order error, at least the position of the write is unchanged.
.// app/lib/writeFormFile.js
// Merge files
const MergeFile = async (fieldname, size) => {
const rootTempPath = path.resolve(__dirname, '.. /.. / '.`public/upload/images/stzz/temp/${fieldname}`)
const chunkPaths = await fse.readdir(rootTempPath) // Read the file in the path
// Reorder
chunkPaths.sort((a, b) = > {
a = a.split(The '-')
b = b.split(The '-')
return a[a.length - 1] - b[b.length - 1]})const rootPath = path.resolve(__dirname, '.. /.. / '.`public/upload/images/stzz/${fieldname}`)
await Promise.all(
chunkPaths.map((chunkPath, index) = > {
return pipeStream(
path.resolve(rootTempPath, chunkPath),
fse.createWriteStream(rootPath, {
start: index * size,
end: (index + 1) * size
})
)
})
)
fse.rmdirSync(rootTempPath) // Delete the temporary folder after the write is successful}...Copy the codeAt this point, large file slicing is complete! Wait, we’ve been talking about the progress bar, so how do we display the progress bar
Display progress bar
Axios allows processing of the progress event onUploadProgress for uploads
options: {
onUploadProgress: function (progressEvent) {
f(progressEvent.loaded)
}
}
Copy the codeAnd before, we used f in the uploadChunks method, and this is where we cut off the uploadChunks
. .map(({ formData }, index) = > {
// api.upload is a self-wrapped AXIos request, as explained later
return API.Upload({
data: formData,
f: this.createProgressHandler(this.data[index]), // The progress bar can be ignored here
requestList: this.requestList, // The number of HTTP requests in progress can be ignored for now
formData: true // Whether content-type is form-data
})
})
createProgressHandler(item) {
return e= > {
item.percentage = e
}
}
...
Copy the codeAs mentioned above, each slice writes some information, including percentage attribute. This is used to monitor the progress of each slice request.
Then, the progress of each section is combined with the total number of files to obtain the upload progress of large files.
. computed: { uploadPercentage() {if (!this.file || !this.data.length) return 0
const loaded = this.data
.map(item= > item.percentage)
.reduce((acc, cur) = > acc + cur)
let num = this.file.size ? parseInt((loaded / this.file.size) * 100) : 0
return num
}
}
...
Copy the codeAt this point, the progress bar display is complete, so the next is the function of the resumable breakpoint implementation
Breakpoint continuingly
The principle of breakpoint continuation is to let the front end/back end remember the uploaded slices and skip the uploaded slices when the front end uplows the slices next time. For convenience, the return of uploaded slice information is implemented by the back end.
pause
To do this, you need axios to work with you.
CancelToken cancels the file being uploaded using the Axios cancelToken configuration item.
options: {
cancelToken: common.source.token
}
Copy the codeConfigure the source in the main.js entry and store it in vuex.
// main.js. import axiosfrom 'axios'
import store from './store'
const CancelToken = axios.CancelToken
const source = CancelToken.source()
const common = store.state.common
common.source = source
...
Copy the codeIn VUEX,source is:
// store/modules/common.js
const state = {
source: {
token: null.cancel: null}}Copy the codeConfigure the same source.token in axios options to cancel all normal upload requests. Next, when we click pause upload, we can cancel the request.
<el-button type="info" size="mini" @click="cancelLoad"> const CancelToken = axios.CancelToken const </el-button type="info" size="mini" @click="cancelLoad" source = CancelToken.source() ... CancelLoad () {this.source.cancel(' suspend request ') this.update_state_asyn ({// vuex action method source: Cancel: this.source.cancel, token: source.token}})}...Copy the codeCancel the request part is complete, then click reply to upload, how do you know where to reply?
<el-button type="info" size="mini" @click="handleResume"> </el-button>... Async handleResume() {const res = await api.verify ({// Check uploaded file slice data: {fileName: This. Radio // file name}}) if (res && res.shouldupload) {await this.uploadchunks (res.uploadedList)} else { }} Async uploadChunks(uploadedList = []) {const requestList = this.data.filter (({ hash }) => ! Uploadedlist.includes (hash)) // Filter unuploaded files... }Copy the codeAfter clicking Reply upload, how do you know when the upload is complete? The requestList variable has been mentioned several times in this article. Here’s an introduction to axios’ interception capabilities.
As we know, Axios has response interceptors. So we store the requested slice information into an array requestList
const instance = axios.create()
...
await this.interceptors(instance, requestList)
...
// Response intercepts, reserving useful parts
this.interceptors = function (instance, requestList) {
instance.interceptors.response.use(
res= > {
let { data, status } = res
if (status === 200 && data && data.code === 200) {
const xhrIndex = requestList.findIndex(item= > item === instance)
requestList.splice(xhrIndex, 1) // When the request is complete, remove it.
}
}
)
requestList.push(instance) // Retain the inttance of all slices
}
Copy the codeThe length of the ‘requestList’ array can be listened for using the object’s reference to determine if the upload is complete.
At this point, the breakpoint continuation is almost complete.
Obtain public number material management
When obtaining the public account material management information, it is necessary to configure some basic information and authentication in the public account.
Public account platform to improve information
On the development – Basic Settings page of the official website of the public platform, click the “Modify Configuration” button and fill in the server address (URL), Token and EncodingAESKey, where THE URL is the interface URL used by the developer to receive wechat messages and events.
Verify that the message is indeed from the wechat server
After the developer submits the information, the wechat server will send a GET request to the filled server address URL. The parameters of the GET request are shown in the following table:
The backend interface
Verify the interface
async verify() {
const { ctx, app, config } = this
const { wxToken } = config
const { WeChat } = app.lib
const weChat = new WeChat(wxToken.wechat_stzz)
weChat.auth(ctx)
}
Copy the code// app/lib
const sha1 = require("sha1"); // Introduce the encryption module
function WeChat(config) {
// Pass in the configuration file
this.config = config;
this.token = config.token;
this.appId = config.appId;
this.appScrect = config.appScrect;
}
// wechat authorization verification method
WeChat.prototype.auth = function (ctx) {
// Get the data sent by the wechat server
const signature = ctx.query.signature,
timestamp = ctx.query.timestamp,
nonce = ctx.query.nonce,
echostr = ctx.query.echostr
// Three parameters are sorted lexicographically by token, timestamp and nonce
const arr = [this.token, timestamp, nonce].sort().join(' ')
/ / sha1 encryption
const result = sha1(arr)
if (result === signature) {
ctx.body = echostr
} else {
ctx.send('mismatch')...}}Copy the codeAfter authentication is successful, more interface permissions are granted to meet more service requirements.
Material Management Interface
Every time the public number interface is called, access_token is needed for verification. It is impossible for us to obtain access_token every time we call the interface. Because the access_token has a time limit, we do not need to request invocation every time. Egg middleware is used in conjunction with Redis.
const axios = require('axios')
module.exports = (options, app) = > {
return async function (ctx, next) {
const { redis, config } = app
const { wxToken } = config
const stzz = wxToken.wechat_stzz
const token = await redis.get('db0').get('wechatToken')
const jspTicket = await redis.get('db0').get('wechatJspTicket')
// Get the token value in the header of the sent data
const method = ctx.method.toLowerCase()
try {
if (method === 'get') {
await next()
} else if(! token || ! jspTicket) {if(! token) {const access = await axios.get(`https://api.weixin.qq.com/cgi-bin/token?grant_type=client_credential&appid=${stzz.appId}&secret=${stzz.appSecret}`)
if (access && access.status === 200 && access.data.access_token) {
await redis.get('db0').setex('wechatToken'.1.5 * 3600, access.data.access_token)
} else {
throw 'Failed to obtain token'}}if(! jspTicket) {const to = await redis.get('db0').get('wechatToken')
const ticket = await axios.get(`https://api.weixin.qq.com/cgi-bin/ticket/getticket?access_token=${to}&type=jsapi`)
if (ticket && ticket.status === 200 && ticket.data.ticket) {
await redis.get('db0').setex('wechatJspTicket'.1.5 * 3600, ticket.data.ticket)
} else {
throw 'Failed to obtain jspTicket'
}
// console.log('ticket', ticket)
}
await next()
} else {
await next()
}
} catch (error) {
ctx.err(error)
}
}
}
Copy the codeAccess_token can be easily obtained using middleware and Redis.
The next part is simply getting the material, which is a bit easier.
async getMaterialList() {
const { app, ctx } = this
const { redis } = app
try {
const token = await redis.get('db0').get('wechatToken')
// console.log('token', token)
const list = await axios.post(`https://api.weixin.qq.com/cgi-bin/material/batchget_material?access_token=${token}`, {
type: 'news'.offset: 0.count: 4
})
ctx.success(list && list.data)
} catch (error) {
ctx.fail(error)
}
}
}
Copy the codeHere, to obtain the public number material management is completed. It’s time to prepare for the tedious verification and configuration of the H5.
H5 Share image title configuration
The platform part of the official account
Share picture title setting, need to improve the information in the public account. Such as:
- Bind the domain name. Fill in JS interface security domain name in function Setting of Public Account Setting.
- Whitelist
The front part
The introduction of JS file: res2.wx.qq.com/open/js/jwe… (HTTPS is supported).
It is important to note that if the HTTPS protocol is used, the imported resource must be HTTPS and not HTTP.
axios.post('${baseURL}api/weChat/h5Verify', {
url: window.location.href
})
.then(function (res) {
if (res && res.data && res.data.result) {
const result = res.data.result
wx.config({
debug: false.appId: result.appId,
timestamp: result.timestamp,
nonceStr: result.noncestr,
signature: result.signature,
jsApiList: [
'updateAppMessageShareData'.'updateTimelineShareData'
]
})
wx.ready(function () {
wx.updateAppMessageShareData({
title: '${this.ruleForm.title}'.// Share the title
desc: '${this.ruleForm.des}'.// Share the description
link: '${baseURL}api/public/upload/images/h5/${encodeURIComponent(this.fileList[0].name.split('.') [0] + '.html')} '.imgUrl: '${this.h5Host}${encodeURIComponent(this.fileList[0].name)}'.// Share ICONS
success: function () {// The setting succeeded
}
})
wx.updateTimelineShareData({
title: '${this.ruleForm.title}'.// Share the title
link: '${baseURL}api/public/upload/images/h5/${encodeURIComponent(this.fileList[0].name.split('.') [0] + '.html')} '.imgUrl: '${this.h5Host}${encodeURIComponent(this.fileList[0].name)}'.// Share ICONS
success: function () {
// The setting succeeded
}
})
}
})
.catch(function (error) {
console.log(error)
})
Copy the codeUse axios.post to send the current web page URL. When a link is shared for the second time, the page URL will automatically have multiple parameters such as: &f… . This is determined by H5 in order to determine whether it is the source of sharing.
The backend part
To generate the parameters required by wx.config, the back end needs to get the ACCESS_token and jspTicket of WX. Middleware shows up earlier, and this is just a small screenshot.
// middleware/wechat.js
if(! token) {const access = await axios.get(`https://api.weixin.qq.com/cgi-bin/token?grant_type=client_credential&appid=${stzz.appId}&secret=${stzz.appSecret}`)
if (access && access.status === 200 && access.data.access_token) {
await redis.get('db0').setex('wechatToken'.1.5 * 3600, access.data.access_token)
} else {
throw 'Failed to obtain token'}}if(! jspTicket) {const to = await redis.get('db0').get('wechatToken')
const ticket = await axios.get(`https://api.weixin.qq.com/cgi-bin/ticket/getticket?access_token=${to}&type=jsapi`)
if (ticket && ticket.status === 200 && ticket.data.ticket) {
await redis.get('db0').setex('wechatJspTicket'.1.5 * 3600, ticket.data.ticket)
} else {
throw 'Failed to obtain jspTicket'
}
// console.log('ticket', ticket)
}
Copy the codeJsp_ticket is also time-sensitive and lasts for 2 hours. Once we have the JSP_ticket, we can work with the URL to generate the parameters we need.
// controller/
async h5Verify() {
const { ctx, app, config } = this
const { wxToken } = config
const { WeChat } = app.lib
const { redis } = app
try {
const params = ctx.joi({
url: Joi.string().required()
})
const jspTicket = await redis.get('db0').get('wechatJspTicket')
const weChat = new WeChat(wxToken.wechat_stzz)
const res = weChat.getSignature(params.url, jspTicket)
ctx.success(res)
} catch (error) {
ctx.fail(error)
}
}
Copy the code// lib/WeChat.js. / generate signature function WeChat. Prototype. GetSignature =function (nowUrl, key) {
let noncestr = Math.random()
.toString(36)
.substr(2); // Random string
let timestamp = moment().unix() // Get the timestamp, numeric type
let jsapi_ticket = `jsapi_ticket=${key}&noncestr=${noncestr}×tamp=${timestamp}&url=${nowUrl}`
jsapi_ticket = sha1(jsapi_ticket)
return {
noncestr: noncestr,
timestamp: timestamp,
signature: jsapi_ticket,
appId: this.appId
}
}
...
Copy the codeHere’s the end of H5 sharing picture title.
Sharing is not easy, like words must not forget to point 💖!!
Pay attention to not point 💖 only is play hooligan, collect only also not point 💖 also is play hooligan.
End 👍 👍 👍.
Reference:
Bytedance Interviewer: Please implement a large file upload and resumable breakpoint