First, supplementary points related to recording

1. Av_read_frame () often returns error code -35 when reading recordings from recording devices. What does this error code mean?

#define EAGAIN          35              // Resource temporarily unavailable
Copy the code
  • This error code representsThe resource is temporarily unreadable
  • This can be masked by the following code
else if (reslutCode == AVERROR(EAGAIN)){
qDebug() << "The recording equipment is not ready." << reslutCode;
continue;
}
Copy the code

2. Avformat_open_input () crashes when called on MAC without permissions.

Add the info.plist file under the project with the following contents:

<? The XML version = "1.0" encoding = "utf-8"? > <! DOCTYPE plist PUBLIC "- / / / / DTD plist Apple / 1.0 / EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd" > < plist Version = "1.0" > < dict > < key > NSMicrophoneUsageDescription < / key > < string > application user microphone < / string > <key>NSCameraUsageDescription</key> <string> Apply for user camera </string> </dict> </plist>Copy the code

Then import the info.plist file in.pro

QMAKE_INFO_PLIST = Info.plist
Copy the code
  • Finally,debugRun in mode

3. A tip: how to intuitively obtain the recording device on the MACSampling rate and sound track? (Understand can be)

  • Click:The apple icon in the upper left corner of the MAC → About the machine → System report → audio

4. How to use the date as the name of the recording output file? (Understand can be)

/ / file name
QString filename = FILEPATH;
filename += QDateTime::currentDateTime().toString("MM_dd____HH:mm:ss");
filename += ".pcm";
QFile file(filename);
Copy the code

5. If the child thread exits abnormally and the UI terminates from the recording state, how to detect the abnormal termination of the child thread?

if(! _audioThread) {// Click "Start recording"
 _audioThread = new AudioThread (this);
 _audioThread->start();
 connect(_audioThread, &AudioThread::finished, [this]() {
     _audioThread = nullptr;
     ui->startRecordAudioButton->setText("Start recording.");
 });
 ui->startRecordAudioButton->setText("End of the recording.");
}
Copy the code

Second, the PCM

1. How about using ffPlay to play PCM’s naming format? What are -ar, -ac, and -f? How to view the sample format of PCM?

  • The format is as follows:
ffplay -ar 44100 -ac 2 -f s16le in.pcm
Copy the code
  • -ar: the sampling rate
  • -ac: a track number
  • -f: Sampling format,s16le: signed 16 – bit little endian
  • View the sample format:ffmpeg -formats | grep PCM

2. What is SDL?

  • SDL is a Simple DirectMedia Layercross-platformC language multimedia development library.
  • Provides low-level access to audio, keyboard, mouse, game joystick, and graphics hardware
  • It is used in many video player software, emulators, and popular games

3. Import SDL2 header and library files

INCLUDEPATH += /usr/local/Cellar/sdl2/2.014._1/include

LIBS += -L /usr/local/Cellar/sdl2/2.014._1/lib -lSDL2
Copy the code

4. Initialize the SDL

if(SDL_Init(SDL_INIT_AUDIO) ! =0) {
qDebug() << "Error initializing SDL" << SDL_GetError();
return 1;
}
qDebug() << "SDL initialization succeeded";
SDL_Quit();
Copy the code

5. SDL has two main modes for playing audio, which are they?

  • 1. A program that actively pushes data to an audio device.
  • Pull: Audio device to actively pull data from a program

6. Use SDL to complete the playbackin.pcmThe core steps (ideas) of

  • inThe child threadThe main operations done in

① SDL_Init Initializes the SDL_INIT_AUDIO subsystem

② SDL_OpenAudio Opens the audio subsystem

③ Create the SDL_AudioSpec object and configure the audio parameters and the callback function of the device pull stream

④ file.open Open the file

⑤ SDL_PauseAudio(0) starts playing audio

⑥ Read PCM data and provide it to audio player

⑦ After all playback, close the file, close the device, clear the audio subsystem

  • inAudio player thread (will constantly call pull_audio_data to pull the stream)The main operations of
Void pull_audio_data(void *userdata, // Samples * format * channels / 8) int len)Copy the code

SDL_memset(stream, 0, len);

Call SDL_MixAudio(stream, buffer->data, buffer->pullLen, SDL_MIX_MAXVOLUME); Provides data for the audio subsystem

③ Adjust data and len in the buffer

7. Use SDL to finish playingin.pcmThe core code of

#include "playthread.h"

#include <SDL2/SDL.h>
#include <QDebug>
#include <QFile>

#define FILENAME "F:/in.pcm"
/ / sampling rate
#define SAMPLE_RATE 44100
// Sample format
#define SAMPLE_FORMAT AUDIO_S16LSB
// Sample size
#define SAMPLE_SIZE SDL_AUDIO_BITSIZE(SAMPLE_FORMAT)
/ / track number
#define CHANNELS 2
// The number of samples in the audio buffer
#define SAMPLES 1024
// How many bytes each sample takes
#define BYTES_PER_SAMPLE ((SAMPLE_SIZE * CHANNELS) >> 3)
// File buffer size
#define BUFFER_SIZE (SAMPLES * BYTES_PER_SAMPLE)

typedef struct {
 int len = 0;
 int pullLen = 0;
 Uint8 *data = nullptr;
} AudioBuffer;

PlayThread::PlayThread(QObject *parent) : QThread(parent) {
 connect(this, &PlayThread::finished,
         this, &PlayThread::deleteLater);

}

PlayThread::~PlayThread() {
 disconnect();
 requestInterruption();
 quit();
 wait();

 qDebug() << this << "Destructed.";
}

// Wait for audio device callback (callback multiple times)
void pull_audio_data(void *userdata,
                  // The stream needs to be populated with PCM data
                  Uint8 *stream,
                  Samples * format * Channels / 8
                  int len
                 ) {
 qDebug() << "pull_audio_data" << len;

 // Empty the stream (mute processing)
 SDL_memset(stream, 0, len);

 / / remove the AudioBuffer
 AudioBuffer *buffer = (AudioBuffer *) userdata;

 // File data is not ready yet
 if (buffer->len <= 0) return;

 // Select the minimum value of len and bufferLen (to ensure data security, prevent pointer out of bounds)
 buffer->pullLen = (len > buffer->len) ? buffer->len : len;

 // Populate the data
 SDL_MixAudio(stream,
              buffer->data,
              buffer->pullLen,
              SDL_MIX_MAXVOLUME);
 buffer->data += buffer->pullLen;
 buffer->len -= buffer->pullLen;
}

void PlayThread::run(a) {
 // Initialize the Audio subsystem
 if (SDL_Init(SDL_INIT_AUDIO)) {
     qDebug() << "SDL_Init error" << SDL_GetError();
     return;
 }

 // Audio parameters
 SDL_AudioSpec spec;
 / / sampling rate
 spec.freq = SAMPLE_RATE;
 // Sample format (s16LE)
 spec.format = SAMPLE_FORMAT;
 / / track number
 spec.channels = CHANNELS;
 // The number of samples in the audio buffer (this value must be a power of 2)
 spec.samples = SAMPLES;
 / / callback
 spec.callback = pull_audio_data;
 // The argument passed to the callback
 AudioBuffer buffer;
 spec.userdata = &buffer;

 // Turn on the device
 if (SDL_OpenAudio(&spec, nullptr)) {
     qDebug() << "SDL_OpenAudio error" << SDL_GetError();
     // Clear all subsystems
     SDL_Quit();
     return;
 }

 // Open the file
 QFile file(FILENAME);
 if(! file.open(QFile::ReadOnly)) { qDebug() <<"file open error" << FILENAME;
     // Shut down the device
     SDL_CloseAudio();
     // Clear all subsystems
     SDL_Quit();
     return;
 }

 // Start playing (0 is unpause)
 SDL_PauseAudio(0);

 // Store data read from file
 Uint8 data[BUFFER_SIZE];
 while(! isInterruptionRequested()) {// If the audio data read from the file is not complete, skip it
     if (buffer.len > 0) continue;

     buffer.len = file.read((char *) data, BUFFER_SIZE);

     // The file data has been read
     if (buffer.len <= 0) {
         // The number of samples left
         int samples = buffer.pullLen / BYTES_PER_SAMPLE;
         int ms = samples * 1000 / SAMPLE_RATE;
         SDL_Delay(ms);
         break;
     }

     // Read file data
     buffer.data = data;
 }

 // Close the file
 file.close();

 // Shut down the device
 SDL_CloseAudio();

 // Clear all subsystems
 SDL_Quit();
}
Copy the code