Wavesurfer is a tripartite library for interactive navigation Audio visualization using Web Audio API and Canvas. Making the address
Background:
Wavesurfer is usually used to render the sound track waveform. Wavesurfer is used in the author’s current project to realize the analysis and annotation of the audio intercept paragraph. In the process of implementation, it is found that the loading and rendering of large audio (waV files above 100M) will occupy too much browser memory, leading to the browser crash. In view of this phenomenon, we have investigated the solutions that can be found at present, such as:
- Based on wavesurfer.js ultra large audio progressive request implementation
Because the server partner is too busy to support this solution, it is not verified whether it is feasible. Finally, after reading part of wavesurfer source code, a similar piecewide loading method was used to solve the current problem.
The main process of audio segmentation loading is as follows:
- Initiate the first 100 bytes of a WAV resource request
- Concatenate the header information of the current WAV resource file from the first 100 bytes
- Request current WAV resources in order of a certain range of bytes, and each request segment is splined with the previously obtained header information, which is processed into a buffer that Wavesurfer can operate
- Wavesurfer generates waveform information for each small section of buffer obtained before processing
- When all bytes of WAV resource are requested and buffer is processed by Wavesurfer into waveform information, the waveform information of all request segments is splintered and sent to Wavesurfer for rendering. At the same time, the waveform information file is generated and uploaded to the server for saving. Next time to obtain the same WAV resources directly obtain waveform information file, avoid repeated decode
The following explains the previous process steps:
Before you do that you need to understand what WAV resources are (basics)
- Reference 1
- Reference 2
Why is the first 100 bytes requested
Note that the data in a WAV resource file is the actual data portion (hexadecimal) starting at 50H bits, so capturing 100 bytes of the resource can capture a complete header
Because the wavesurfer method needs to be called in step 3 to decode, it must be a complete audio file in this process, otherwise decode will fail.
In case of segment-loaded audio, except for the first section of the request result with the header information of the WAV resource, all subsequent sections of the request will not automatically carry the header information, that is, the incomplete audio file. So you need to get the current WAV resource header information splicing to all the request section after the formation of a “complete” audio file, so that wavesurfer normal decode generated current request section waveform information.
How do I get the first 100 bytes of header information
In reference 2, the article introduces that the corresponding ACSCII code of 48H ~ 4BH 64 61 74 71 is data and the real data part from 50H. In addition, the corresponding data information of data is always 64, 61, 74, 71 (the corresponding base 10 is 100, 97, 116, 97), so it can be used as the target point to obtain real data. The data information in front of data is the complete header information.
import axios from 'axios';
function getWavHeaderInfo (buffer) {
const firstBuffer = buffer;
// Create a data view to manipulate data
const dv = new DataView(firstBuffer);
// Retrieve data from the data view
const dvList = [];
for (let i = 0, len = dv.byteLength; i < len; i += 1) {
dvList.push(dv.getInt8(i));
}
// Locate the data field in the header
const findDataIndex = dvList.join().indexOf(', 100,97,116,97 ');
if (findDataIndex <= - 1) {
throw new Error('Parsing failed');
}
// All data information strings before the data field
const dataAheadString = dvList.join().slice(0, findDataIndex);
The 8 bits after the data field are the end of the header information
const headerEndIndex = dataAheadString.split(', ').length + 7;
// Capture all header information
const headerBuffer = firstBuffer.slice(0, headerEndIndex + 1);
return headerBuffer;
}
const requestOptions = {
url: 'Audio URL'.method: 'get'.responseType: 'arraybuffer'.// Request a WAV message format
headers: {
Range: `bytes=0-99`.// The first 100 bytes}}; axios(requestOptions).then(response= > {
const headerBuffer = getWavHeaderInfo(response.data);
})
Copy the codeConcatenate the request result for each segment with the header information already obtained
Previously, the header information of the current resource has been obtained through the first request, and the next step is to spline the header information of the remaining resources in sequence one by one, and then hand it to Wavesurfer for decode
(PS: For code brevity, the following code examples are written in Class.)
import axios from 'axios';
import _ from 'lodash';
class requestWav {
constructor() {
}
defaultOptions = {
responseType: 'arraybuffer'.rangeFirstSize: 100.rangeSize: 1024000 * 2.requestCount: 1.// Number of requests
loadRangeSucess: null.// The callback completed after each request
loadAllSucess: null.// A callback after all the loading is complete
}
// Merge the configuration
mergeOptions(options) {
this.options = Object.assign({}, this.defaultOptions, options);
}
loadBlocks(url, options) {
if(! url || ! _.isString(url)) {throw new Error('Argument [url] should be supplied a string');
}
this.mergeOptions(options);
this.url = url;
this.rangeList = [];
this.rangeLoadIndex = 0; // Index of the current request
this.rangeLoadedCount = 0; // Number of current requests
// Take a resource of 100 bytes to get the resource header information
this.rangeList.push({ start: 0.end: this.options.rangeFirstSize - 1 });
this.loadFirstRange();
}
// Actually initiate the request
requestRange(rangeArgument) {
const range = rangeArgument;
const { CancelToken } = axios;
const requestOptions = {
url: this.url,
method: 'get'.responseType: this.options.responseType,
headers: {
Range: `bytes=${range.start}-${range.end}`,},// Configure an extension that cancels requests
cancelToken: new CancelToken((c) = >{ range.cancel = c; })};return axios.request(requestOptions);
}
fileBlock(fileSize) {
// Calculate each size range queue that needs to be requested based on the size of the file in the header
let rangeStart = this.options.rangeFirstSize;
for (let i = 0; rangeStart < fileSize; i += 1) {
const rangeItem = {
start: rangeStart,
end: rangeStart + this.options.rangeSize - 1};this.rangeList.push(rangeItem);
rangeStart += this.options.rangeSize; }}// Request the first paragraph
loadFirstRange() {
this.requestRange(this.rangeList[this.rangeLoadIndex]).then((response) = > {
const fileRange = response.headers['content-range'].match(/\d+/g);
// Get the file size
const fileSize = parseInt(fileRange[2].10);
// Calculate the request block
this.fileBlock(fileSize);
// Place the result of the request into the data field in the request queue
this.rangeList[0].data = response.data;
// Get the resource header information (same method as step 2)
this.headerBuffer = this.getWavHeaderInfo(response.data);
// Handle the callback after each segment is loaded
this.afterRangeLoaded();
// Load the rest
this.loadOtherRanges();
}).catch((error) = > {
throw new Error(error);
});
this.rangeLoadIndex += 1;
}
afterRangeLoaded() {
this.rangeLoadedCount += 1;
// After each request, determine the current request index and all the numbers that should be requested
// Trigger loadRangeSucess and loadAllSucess callbacks
if (this.rangeLoadedCount > 1
&& this.options.loadRangeSucess
&& typeof this.options.loadRangeSucess === 'function'
) {
this.contactRangeData(this.options.loadRangeSucess);
}
if (this.rangeLoadedCount >= this.rangeList.length
&& this.options.loadAllSucess
&& typeof this.options.loadAllSucess === 'function'
) {
this.options.loadAllSucess();
this.rangeList = [];
}
}
loadOtherRanges() {
// loop the request range queue
if (this.rangeLoadIndex < this.rangeList.length) {
this.loadRange(this.rangeList[this.rangeLoadIndex]);
}
}
loadRange(rangeArgument) {
const range = rangeArgument;
this.requestRange(range).then((response) = > {
// Place the result of the request into the data field in the request queue
range.data = response.data;
this.afterRangeLoaded();
this.loadOtherRanges();
}).catch((error) = > {
throw new Error(error);
});
this.rangeLoadIndex += 1;
}
contactRangeData(callback) {
const blobIndex = this.rangeLoadIndex - 1;
if (!this.headerBuffer) {
return;
}
// Get the data from each data in the request queue, concatenate the existing header information, and save it as audio/wav blob file
const blob = new Blob(
[this.headerBuffer,
this.rangeList[blobIndex].data],
{ type: 'audio/wav'});const reader = new FileReader();
// Read the blob as buffer to the loadRangeSucess callback
reader.readAsArrayBuffer(blob);
reader.onload = (a)= > {
callback(reader.result, blobIndex);
};
reader.onerror = (a)= > {
throw new Error(reader.error);
};
}
destroyRequest() {
// Destruct request.
// If the current resource is not loaded and the URL of the resource is changed, cancel the request to avoid wasting requested resources
if (!this.rangeList) {
return;
}
this.rangeList.forEach((rang) = > {
if (rang.cancel) {
rang.cancel('Cancel audio download'); }}); }}export default new requestWav();
Copy the codeUntil the previous three steps have been completed, all the audio paragraphs have been obtained, and the “complete” audio has been assembled, and the corresponding buffer has been read. After that, the next chapter will continue to improve the fourth and fifth steps to obtain each section of buffer to generate waveform and finally assemble all waveform files.
Wavesurfer Processing large Audio file waveform Rendering (2)
This article is shared as a project to climb the pit record. I hope I can get friends to correct my mistakes.