There are two kinds of Tools for AudioToolBox,

For processing audio data, playback is output, recording is input, etc.

The main use is AVAudioEngine,

Just a little bit lower down, the Audio Queues

Played using Audio Queues, that is, take Audio data, inject it into buffer, and commit the buffer to Audio Queues

Buffer is easy to understand,

It is also possible to submit audio data to a sound card one by one without using a buffer

The downside is that the sound card consumes data slowly and the memory prepares data quickly, which is determined by the computer hardware

With buffer, the sampled data is filled to a capacity capability and can be submitted together to the sound card for consumption

Memory and sound card performance, some balance

Large buffer, the sound card to consume a long time, time latency is larger

Small buffer, sound card to consume fast, small delay

There are file related, audio processing assistance, various services

Audio File Services, handling local Audio resource files

Audio File Stream Services,

That is, audio resource files on the network

The example of this article is to develop a local WAV player

Player flow, routine 3 steps

  • Read the local audio resource file, restore toUInt8Sample data of

The file in this example is in WAV format, uncompressed format,

The sampling rate is 16000, the bit depth is 16 bits pcM_s16le,

A 16-bit sampling requires two UInt8 data to express

  • Transform sampling data of UInt8 into AVAudioPCMBuffer

  • Finally, play

Give AVAudioPCMBuffer to AVAudioPlayerNode, start AVAudioEngine,

Let the AVAudioPlayerNode play and you’re done

The first step corresponds to the Parser in this article’s example code,

Step 2, Reader

Step 3, Streamer

This article focuses on the first step, restoring the audio file to sample data,

The rest is indexed on the previous blog, GitHub Repo

Use Audio File Services to restore

Get the file address, get the file

Use an AudioFileOpenURL to open a file and use resources using the file ID (AudioFileID)

And then go get the properties of the audio file,

How many package Packet using kAudioFilePropertyAudioDataPacketCount, you know,

Use kAudioFilePropertyDataFormat, to obtain a wav file information describing ASBD Audio Stream, Basic Description.

ASBD contains a package, how many bytes

Uncompressed file, calculation is relatively simple, each packet size is equal,

The sampling data distribution is linear,

Using AudioFileReadBytes, read the data one package at a time,

Read the header pointer to the data in the packet, allocate a block of memory,

Specify size, initialize the array, stuff it into Data, and there it is

Near the bottom of the point, allocate memory, complete initialization

Generally with Swift, not accessible

var playbackFile: AudioFileID? public internal(set) var dataFormatD: AVAudioFormat? public internal(set) var packetsX = [Data]() public func parse(url src: URL) throws { Utility.check(error: AudioFileOpenURL(src as CFURL, .readPermission, 0, &playbackFile), operation: "AudioFileOpenURL failed") guard let file = playbackFile else { return } var numPacketsToRead: UInt32 = 0 GetPropertyValue(val: &numPacketsToRead, file: file, prop: kAudioFilePropertyAudioDataPacketCount) var asbdFormat = AudioStreamBasicDescription() GetPropertyValue(val: &asbdFormat, file: file, prop: kAudioFilePropertyDataFormat) dataFormatD = AVAudioFormat(streamDescription: &asbdFormat) /// At this point we should definitely have a data format var bytesRead: UInt32 = 0 GetPropertyValue(val: &bytesRead, file: file, prop: kAudioFilePropertyAudioDataByteCount) guard let dataFormat = dataFormatD else { return } let format = dataFormat.streamDescription.pointee let bytesPerPacket = Int(format.mBytesPerPacket) for i in 0 .. < Int(numPacketsToRead) { var packetSize = UInt32(bytesPerPacket) let packetStart = Int64(i * bytesPerPacket) let dataPt: UnsafeMutableRawPointer = malloc(MemoryLayout<UInt8>.size * bytesPerPacket) AudioFileReadBytes(file, false, packetStart, &packetSize, dataPt) let startPt = dataPt.bindMemory(to: UInt8.self, capacity: bytesPerPacket) let buffer = UnsafeBufferPointer(start: startPt, count: bytesPerPacket) let array = Array(buffer) packetsX.append(Data(array)) } }Copy the code

AudioFileReadBytes reads one package at a time,

Equivalent to reading one UInt8 data at a time,

for i in 0 .. < Int(numPacketsToRead) { var array = [UInt8]() for j in 0.. <bytesPerPacket{ var one: UInt32 = 1 let packetStart = Int64(i * bytesPerPacket + j) let dataPt: UnsafeMutableRawPointer = malloc(MemoryLayout<UInt8>.size) AudioFileReadBytes(file, false, packetStart, &one, dataPt) let data = dataPt.assumingMemoryBound(to: UInt8.self).pointee array.append(data) } packetsX.append(Data(array)) }Copy the code

Using a slightly lower-level service is a matter of tossing the pointer around

Auxiliary methods for obtaining audio file information:

It’s the same routine,

With AudioFileGetPropertyInfo first, to get the information about the memory size, the size

Get the information with AudioFileGetProperty

And then the memory size, the initialization, the property is taken

func GetPropertyValue<T>(val value: inout T, file f: AudioFileID, prop propertyID: AudioFilePropertyID) {
    var propSize: UInt32 = 0
    guard AudioFileGetPropertyInfo(f, propertyID, &propSize, nil) == noErr else {
 
        return
    }
    
    guard AudioFileGetProperty(f, propertyID, &propSize, &value) == noErr else {
        
        return
    }
}
Copy the code

Use Audio File Stream Services to restore

The first step is to retrieve the data and hand it to Audio File Stream Services

Get the sourceURL and load the Data

    public var sourceURL: URL? {
        didSet {
            resetStream()

            if let src = sourceURL{
                do {
                    let data = try Data(contentsOf: src)
                    load(didReceiveData: data)
                } catch {
                    print(error)
                }
            }
        }
    }
Copy the code

Initialize, and you’ve created an audio stream,

Feed the data, just go two callback methods

  • ParserPropertyChangeCallback, attribute in the callback, get want properties

  • ParserPacketCallback, data callback, restores 16 bits of sampled data

Audio File Services (Audio File Services)

Audio File Stream Services is a bit of a detour

    public init() throws {
        let context = unsafeBitCast(self, to: UnsafeMutableRawPointer.self)
        guard AudioFileStreamOpen(context, ParserPropertyChangeCallback, ParserPacketCallback, kAudioFileWAVEType, &streamID) == noErr else {
            throw ParserError.streamCouldNotOpen
        }
    }
    

Copy the code

Pass the data read from the Audio File to the Audio File Stream Service

AudioFileStreamParseBytes UInt8 data, to establish good from the start the audio Stream

    // MARK: - Methods
    
    public func parse(data: Data) throws {
        os_log("%@ - %d", log: Parser.logger, type: .debug, #function, #line)
        
        let streamID = self.streamID!
        let count = data.count
        _ = try data.withUnsafeBytes { (bytes: UnsafePointer<UInt8>) in
            let result = AudioFileStreamParseBytes(streamID, UInt32(count), bytes, [])
            guard result == noErr else {
                os_log("Failed to parse bytes", log: Parser.logger, type: .error)
                throw ParserError.failedToParseBytes(result)
            }
        }
    }

Copy the code

When the data gets to the audio Stream, it gets called back

In the callback, 16-bit sampling data is restored, represented by two UInt8

func ParserPacketCallback(_ context: UnsafeMutableRawPointer, _ byteCount: UInt32, _ packetCount: UInt32, _ data: UnsafeRawPointer, _ packetDescriptions: Optional<UnsafeMutablePointer<AudioStreamPacketDescription>>) { let parser = Unmanaged<Parser>.fromOpaque(context).takeUnretainedValue() /// At this point we should definitely have a data format guard let dataFormat = parser.dataFormatD else { return } let format = dataFormat.streamDescription.pointee let bytesPerPacket = Int(format.mBytesPerPacket) for i in 0 .. < Int(packetCount) { let packetStart = i * bytesPerPacket let packetSize = bytesPerPacket let packetData = Data(bytes: data.advanced(by: packetStart), count: packetSize) parser.packetsX.append(packetData) } }Copy the code

In attribute callback, get information such as the AudioStreamBasicDescription audio files,

See GitHub Repo for details

Auxiliary methods for obtaining audio file information:

It’s the same routine,

First get the memory size of the property, then get the property

In contrast, the service has been changed and is now AudioFileStream


`func GetPropertyValue<T>(_ value: inout T, _ streamID: AudioFileStreamID, _ propertyID: AudioFileStreamPropertyID) {
    var propSize: UInt32 = 0
    guard AudioFileStreamGetPropertyInfo(streamID, propertyID, &propSize, nil) == noErr else {
        return
    }
    
    guard AudioFileStreamGetProperty(streamID, propertyID, &propSize, &value) == noErr else {
        return
    }
}
Copy the code

How to debug?

See the data

Print, print, print, print

OSStatus

Code that’s close to the bottom is prone to all kinds of errors,

Apple will give you an OSStatus, which is not very easy to read

This website is a shameosstatus

github repo