background
The last article on Web Audio showed you how to parse Audio data and map the frequency spectrum of Audio. In this article, I will continue my exploration of Web Audio and show you how to collect microphone Audio data in the browser. And how to draw real-time dynamic audio map during the acquisition process.
Implementation approach
Again, let’s get to the general idea. Collection of audio, first of all, we through the navigator. MediaDevices. GetUserMedia method to get the microphone and voice recording. Navigator is an implementation of the browser’s NavigatorID standard interface that can be used to query information about the application running the current script. We won’t talk about this object here, just how to record audio with it.) getUserMedia Records sound to get the audio stream. We then convert the Audio stream to real-time Audio data using Web Audio related apis. Finally, real-time audio data will be drawn into a map by canvas, and the effect of dynamic real-time map will be realized through repeated this process. The overall process is as follows:
Obtain microphone permission to record sound
Browser security policy rules, the navigator. MediaDevices. GetUserMedia method can only effective under the HTTPS protocol or localhost domain name. So here we first use Node to build a minimal version of the local Web server, and then obtain the microphone permission and record sound.
// server.js
const http = require('http');
const path = require('path');
const fs = require('fs');
http.createServer((request, response) = > {
/ / request body
console.log(new Date(), ':'.JSON.stringify(request));
// The HTML file to load
const file = path.resolve(__dirname, './demo2.html');
// Check whether the file exists
fs.exists(file, exists= > {
if(! exists)console.log ('File does not exist, sand sculpture! ');
else {
response.writeHeader(200, { "Content-Type" : "text/html" });
response.end(fs.readFileSync(file, 'utf-8')); }}); }).listen(8090); // Listen on port 8090
/* * demo2.html * Get the microphone and record the sound */
let audioCtx = null; // Audio context
let source = null; / / audio source
let audioStream = null; // The audio stream generated by recording
let analyserNode = null; // A node for analyzing audio real-time data
let animationFrame = null; / / timer
function recordSound () {
navigator.mediaDevices
.getUserMedia({ 'audio': true })
.then(initAudioData)
.catch(e= > {
console.log('Something's wrong, sand sculpture :', e);
});
}
// Stop recording
function stopRecord () {
// Turn off the microphone
const tracks = audioStream.getAudioTracks();
for (let i = 0, len = tracks.length; i < len; i++) {
tracks[i].stop();
}
// Disconnect the audio node
analyserNode.disconnect();
source.disconnect();
analyserNode = null;
source = null;
// Clear the timer
clearInterval(animationFrame);
}
// Event binding
document.querySelector('#record').onclick = recordSound;
document.querySelector('#stop').onclick = stopRecord;
Copy the codeProcess Audio stream data using the Web Audio Api
After we get the audio stream, we create the audio source using the AudioContext AudioContext. Choose MediaStreamAudioSourceNode here, and it receives a MediaStream object to create the audio source. Then we insert an audio node between the audio source and destination, which is used to acquire and process the audio data, and then use the data to draw the waveform diagram. AnalyserNode is selected here. Of course, ScriptProcessorNode and AudioWorkletNode can also obtain and process real-time audio data. For details, please refer to the relevant Api.
// Audio data processing
function initAudioData (stream) {
audioStream = stream;
// Create audio context
audioCtx = new (window.AudioContext || window.webkitAudioContext)();
// Create an audio source
source = audioCtx.createMediaStreamSource(audioStream);
// Create an audio analysis node
analyserNode = audioCtx.createAnalyser();
// fftSize determines the amount of audio data that can be retrieved
analyserNode.fftSize = 4096;
// Connect the audio source to analyserNode
source.connect(analyserNode);
// analyserNode then connects to the loudspeaker to play
analyserNode.connect(audioCtx.destination);
// Simply use the timer to draw the waveform, or you can use requestAnimationFrame to repeatedly execute the drawing function at the rate of screen refresh
animateFrame = setInterval(drawWaver, 60);
}
Copy the codeReal-time acquisition of audio data, waveform drawing
After AnalyserNode is created, you can set the length to capture the audio time domain data in real time at the current point in time, and then repeatedly retrieve the data through a timer or requestAnimationFrame callback to perform the drawing function to achieve the dynamic waveform effect. Again, a Float32Array array object receives audio data (values in the range -1 to 1) and samples the data. In this case, we draw a graph with a maximum value and a minimum value for each 12 data items.
// Draw a graph
function drawWaver () {
const originData = new Float32Array(analyserNode.fftSize);
const positives = [];
const negatives = [];
// Get current real-time audio data
analyserNode.getFloatTimeDomainData(originData);
// Set a maximum value and a minimum value 4096/12 = 341.3333 for each 12 bits of data
for (let i = 0; i < 341; i++) {
let temp = originData.slice(i * 12, (i + 1) * 12);
positives.push(Math.max.apply(null, temp));
negatives.push(Math.min.apply(null, temp));
}
// Create canvas context
let canvas = document.querySelector('#canvas');
if (canvas.getContext) {
let ctx = canvas.getContext('2d');
canvas.width = positives.length * 4;
let x = 0;
let y = 100;
ctx.fillStyle = '#fa541c';
// Canvas height is 200, and the abscissa is 100px in the middle of the canvas. Draw positive data above the abscissa and negative data below it
for (let k = 0; k < positives.length; k++) {
// Each rectangle is 3px wide and 1px apart, so the total length of the shape is equal to length * 4
ctx.fillRect(x + 4 * k, y - (100 * positives[k]), 3.100 * positives[k]);
ctx.fillRect(x + 4 * k, 100.3.100 * Math.abs(negatives[k])); }}}Copy the codeSimple audio capture and real-time mapping are done. Finally, post the effect picture:
