Solve AAC audio coding time stamp calculation problem

Original address original article, reprint please contact the author

Northwest wang xiang where is, southeast see a few back to the circle. Yesterday the wind blows no one will, tonight clear light as in previous years.

The theme

Audio is streaming data, unlike video, there is no concept of P frame and B frame. It’s like building a wall. Just go up. In general, the file generation step in AAC coding requires no computation time at all if the file is written using OutputStream. But for audio and video syncing or using Android’s built-in MediaMuxer to generate audio files, you need to calculate the time stamp of the audio frame.

reference

The calculation method and API involved in this article are in the Android environment. AudioRecord is used to record audio, MediaCodeC encodes audio in AAC format and MediaMuxer encapsulates audio files in AAC format.

methods

AAC codes have two ways of calculating timestamps. The first is to use the amount of PCM data to calculate; Second: calculate the duration of a frame under the corresponding parameter configuration of AAC coding, and then calculate it with the number of frames.

AAC encoding, MediaMuxer generated file pseudocode

MediaCodeC’s AAC coding process is not described here, but is replaced by pseudocode. This is mainly to show where the timestamp is set in the code:

// MediaCodeC gets the available input queue index = codec.dequeueInputBuffer (......) // When the available output queue is obtained, We fill inputBuffer = codec.getinputBuffer (index) // Fill inputBuffer with PCM data (ByteArray) InputBuffer. Put (byteAarray -- PCM data) codec.queueInputBuffer(index, 0, byteArray size, presentationTimeUs, 0)Copy the code

In the above pseudocode, presentationTimeUs is where we need to set the timestamp

After populating the PCM data, after getting MediaCodeC output, the data is written using MedaMuxer to generate an AAC file.

Path = Output path. Mp4 | Aac | or mp4 | mediaMuxer= mediaMuxer (path, MediaMuxer. OutputFormat. MUXER_OUTPUT_MPEG_4) MediaMuxer. AddTrack (audio track) MediaMuxer. Start () / / codec to output data available. The data is AAC audio data id = codec. DequeueOutputBuffer (bufferInfo, 10000).if(id >= 0){
    outputBuffer = codec.getOutputBuffer(id)
    mediaMuxer.writeSamplet(audioTrack, outputBuffer, bufferInfo)   
}

Copy the code

Note that when using MediaMuxer to generate AAC audio files, there is no need to add the AAC header information, just write it. When MediaMuxer writes to the file, the BufferInfo parameter contains the offset and timestamp of the frame.

For a more complete audio encoding code, see GitHub address AudioEncoder

Use PCM data volume to calculate

PCM is pure audio data that has not been compressed. I have written an article about the introduction to audio before, recording some common sense questions related to PCM. If you are interested, you can go to see them. PCM, as the most original audio data, can calculate the time according to the size. The formula is given first:

presentationTimeUs = 1000000L * (totalBytes / 2) / sampleRate
Copy the code

This is a mono PCM file timestamp calculation method configured with sampletRate sampling rate, sampling bits of 16 bits

Next, let’s analyze the calculation origin of the above formula: Assume that there is a PCM file, sampling rate is S, sampling bit is N –(generally, the selection of sampling bit is 4bit, 8bit, 16bit and 32bit), and the sound channel is mono. Then within 1s, the size of this PCM is:

Size = S * n * 1, unit: bitCopy the code

As we all know, 1 Byte = 8bit, 1 Short = 16bit. Then, the size of PCM per unit time is:

Byte = S * N * 1/8 Short = S * N * 1/16Copy the code

So, according to the above, the configuration parameters are PCM file with sampleRate, 16bit and 1 sound channel. When the total size of the encoder passed in reaches totalByte, the timestamp calculation method is as follows:

Currents (Subtle) = totalByte/(sampleRate * 16 * 1/8) = totalByte / 2 / sampleRate * 1000000LCopy the code

Of course, if ShortArray is selected to host PCM data, then the formula becomes:

Currents (Subtle) = totalShort/(sampleRate * 16 * 1/16) = totalShort/sampletRate * 1000000LCopy the code

Calculation using AAC frame time

When the encoder outputs data each time, it can be regarded as output a frame of AAC data. The number of frames in a 1 second AAC audio file is sampleRate / 1024 frames. Thus, the duration of a frame AAC is:

PerFrameTime (Subtle) = 1000000L/sampleRate / 1024Copy the code

The above