Play network audio, can first download good, get audio files, simple
Use AVAudioPlayer to play, over
Apple package under AVAudioPlayer handles local files for convenience
Just get a file url and play it
Simple mechanical understanding:
Facilitate the transmission of audio, generally using audio compression files, MP3 and so on. File pressure small volume, good transmission
Sound cards play PCM buffers
Apple helped develop a compressed format that was converted to an uncompressed raw file called PCM,
Also help the development of audio playback resource scheduling, from the PCM file to take out a section of the buffer, to the sound card consumption
(Actually there are not two steps, of course the process is parallel)
Now the manual
This article introduces the direct audio streaming media
When you receive an audio packet from the network, you play it.
Step four:
1, network audio files >> download to local audio data
Download the binary data of the audio file
Task of URLSession to retrieve network files
When you get a packet of Data, you process one
In this example, a packet, Data, corresponds to an audio packet, and corresponds to an audio buffer
This is the easy step,
Create a URLSessionDataTask and download it
It’s all in the network proxy method
Extension Downloader: URLSessionDataDelegate public func urlSession(_ session: urlSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) { totalBytesCount = response.expectedContentLength CompletionHandler (.allow)} public func urlSession(_ session: urlSession, dataTask: URLSessionDataTask, didReceive data: Data) {// Update, TotalBytesReceived += Int64(data.count) progress = Float(totalBytesReceived)/Float(totalBytesCount) // Data is handed to the delegate to be parsed as an audio packet delegate? .download(self, didReceiveData: data, progress: progress)} public func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { state = .completed delegate?.download(self, completedWithError: error) } }Copy the codeAudio basic understanding first:
Audio files are divided into encapsulation format (file format), and encoding format
The three levels of audio data are buffer, packet and frame
Packet, packet,
Audio packet, audio frame
Audio encoding format, generally divided into variable bit rate, and fixed bit rate
Fixed bit rate CBR, average sampling, corresponding to raw file, PCM (uncompressed file)
Variable bit rate VBR, corresponding to compressed files, such as MP3
Core Audio supports VBR, generally via the variable frame rate format VFR
VFR refers to: the volume of each packet is the same, and the number of frames in packet is different, and the audio data contained in frame is large or small
Description of data in Core Audio
Fixed rate described in ASBD AudioStreamBasicDescription
The description of ASBD refers to some configuration information, including channel number, sampling rate, bit depth…
VFR variable bit rate, with ASPD description, AudioStreamPacketDescription
VFR in compressed audio data corresponds to ASPD
Each Packet has its ASPD
ASPD contains information about the position of the packet mStartOffset, the number of frames in the packet, mVariableFramesInPacket
2, Audio data >> Audio Packet
Take Audio Queue Services, process the Audio binary data obtained in the previous step and parse it into Audio packet
2.1 Establish audio processing channel, register parse callback method
public init() throws {
letcontext = unsafeBitCast(self, to: UnsafeMutableRawPointer. Self) / / create a active audio file stream parser, Create a parser ID guard AudioFileStreamOpen (context, ParserPropertyChangeCallback ParserPacketCallback, kAudioFileMP3Type, &streamID) == noErrelse {
throw ParserError.streamCouldNotOpen
}
}
Copy the code2.2 Pass data in and start parsing
public func parse(data: Data) throws {
let streamID = self.streamID!
let count = data.count
_ = try data.withUnsafeBytes({ (rawBufferPointer) in
let bufferPointer = rawBufferPointer.bindMemory(to: UInt8.self)
if letAddress = bufferPointer. BaseAddress {/ / the audio data to the parser / / streamID, specify the parserlet result = AudioFileStreamParseBytes(streamID, UInt32(count), address, [])
guard result == noErr else {
throw ParserError.failedToParseBytes(result)
}
}
})
}
Copy the code2.3 Audio Information parsing
func ParserPropertyChangeCallback(_ context: UnsafeMutableRawPointer, _ streamID: AudioFileStreamID, _ propertyID: AudioFileStreamPropertyID, _ flags: UnsafeMutablePointer<AudioFileStreamPropertyFlags>) {
letParser = Unmanaged< parser >.fromopaque (context). TakeUnretainedValue ()casekAudioFileStreamProperty_DataFormat: / / get data format var format = AudioStreamBasicDescription () GetPropertyValue (& format, streamID, propertyID) parser.dataFormat = AVAudioFormat(streamDescription: &format)casekAudioFileStreamProperty_AudioDataPacketCount: GetPropertyValue(& Parser. packetCount, streamID, propertyID) default: // Audio stream file, number of packets in the separated audio data Func GetPropertyValue<T>(_ value: inout T, _ streamID:) GetPropertyValue<T>(_ value: inout T, _ streamID:) AudioFileStreamID, _ propertyID: AudioFileStreamPropertyID) { var propSize: UInt32 = 0 guard AudioFileStreamGetPropertyInfo(streamID, propertyID, &propSize, nil) == noErrelse {
return
}
guard AudioFileStreamGetProperty(streamID, propertyID, &propSize, &value) == noErr else {
return}}Copy the code2.4 Parse callbacks and process data
func ParserPacketCallback(_ context: UnsafeMutableRawPointer, _ byteCount: UInt32, _ packetCount: UInt32, _ data: UnsafeRawPointer, _ packetDescriptions: UnsafeMutablePointer < AudioStreamPacketDescription >) {/ / back to the self (parser)let parser = Unmanaged<Parser>.fromOpaque(context).takeUnretainedValue()
letpacketDescriptionsOrNil: UnsafeMutablePointer<AudioStreamPacketDescription>? = packetDescriptions // ASPD exists, is compressed audio package // uncompressed PCM, use ASBDletisCompressed = packetDescriptionsOrNil ! = nil guardlet dataFormat = parser.dataFormat else {
return} // parser.packets = self.packets = self.packetsif isCompressed {
for i in0.. < Int(packetCount) {// Compress audio data, each packet corresponds to an ASPD, one by onelet packetDescription = packetDescriptions[i]
let packetStart = Int(packetDescription.mStartOffset)
let packetSize = Int(packetDescription.mDataByteSize)
let packetData = Data(bytes: data.advanced(by: packetStart), count: packetSize)
parser.packets.append((packetData, packetDescription))
}
} else{// Original audio data PCM, file unified configuration, calculation is relatively simplelet format = dataFormat.streamDescription.pointee
let bytesPerPacket = Int(format.mBytesPerPacket)
for i in 0 ..< Int(packetCount) {
let packetStart = i * bytesPerPacket
let packetSize = bytesPerPacket
let packetData = Data(bytes: data.advanced(by: packetStart), count: packetSize)
parser.packets.append((packetData, nil))
}
}
}
Copy the code3, Audio packet >> audio buffer
public required init(parser: Parsing, readFormat: AVAudioFormat) throws {// Take audio data self.parser = parser guard from the previous parserlet dataFormat = parser.dataFormat else {
throw ReaderError.parserMissingDataFormat
}
let sourceFormat = dataFormat.streamDescription
let commonFormat = readFormat.streamDescription // Create audio Format converter converter // by specifying input Format, and output Format // input Format is parsed from the previous step, take output Format from paser, develop the specifiedlet result = AudioConverterNew(sourceFormat, commonFormat, &converter)
guard result == noErr else {
throw ReaderError.unableToCreateConverter(result)
}
self.readFormat = readFormat
}
Copy the codeDevelop the specified output format
public var readFormat: AVAudioFormat {
return AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 44100, channels: 2, interleaved: false)! } // Bit depth, using Float32 // sample rate 44100 Hz, standard CD sound quality // minute around the channelCopy the codeThe audio packet is parsed in the previous step and the audio buffer is read
public func read(_ frames: AVAudioFrameCount) throws -> AVAudioPCMBuffer {
let framesPerPacket = readFormat. StreamDescription. Pointee. MFramesPerPacket var packets = frames/framesPerPacket / / create a blank, specify the Format and capacity, Audio buffering AVAudioPCMBuffer Guardlet buffer = AVAudioPCMBuffer(pcmFormat: readFormat, frameCapacity: frames) else{throw ReaderError. FailedToCreatePCMBuffer} buffer. FrameLength = frames / / the parse the audio packets packet, convert AVAudioPCMBuffer, So AVAudioEngine plays try queue.sync {letcontext = unsafeBitCast(self, to: UnsafeMutableRawPointer. Self) / / set a good converter converter, using the callback method ReaderConverterCallback, Fill to create the data in the buffer buffer. MutableAudioBufferListletstatus = AudioConverterFillComplexBuffer(converter! , ReaderConverterCallback, context, &packets, buffer.mutableAudioBufferList, nil) guard status == noErrelse {
switch status {
case ReaderMissingSourceFormatError:
throw ReaderError.parserMissingDataFormat
case ReaderReachedEndOfDataError:
throw ReaderError.reachedEndOfFile
case ReaderNotEnoughDataError:
throw ReaderError.notEnoughData
default:
throw ReaderError.converterFailed(status)
}
}
}
return buffer
}
Copy the code- The use of AudioConverterFillComplexBuffer pose:
AudioConverterFillComplexBuffer (format converter, the callback function, custom parameters pointer, the number of packages pointer, receive the converted data pointer, receive ASPD pointer)
AudioConverterFillComplexBuffer(converter! , ReaderConverterCallback, context, &packets, buffer.mutableAudioBufferList, nil)Copy the code- AudioConverterFillComplexBuffer ReaderConverterCallback callback function, the use of the position:
Callback functions (format converter, number of packets pointer, pointer to receive converted data, pointer to receive ASPD, pointer to custom parameters)
That is passed to the AudioConverterFillComplexBuffer six parameters,
In addition to its callback argument itself, the other five arguments, its callback function is useful
Convert the buffer callback function to create a blank audio buffer and fill it with data
func ReaderConverterCallback(_ converter: AudioConverterRef, _ packetCount: UnsafeMutablePointer<UInt32>, _ ioData: UnsafeMutablePointer<AudioBufferList>, _ outPacketDescriptions: UnsafeMutablePointer<UnsafeMutablePointer<AudioStreamPacketDescription>? >? , _ context: UnsafeMutableRawPointer?) -> OSStatus {// restore self (reader)letreader = Unmanaged<Reader>.fromOpaque(context!) .takeunretainedValue () // Ensure that the input format is guardlet sourceFormat = reader.parser.dataFormat else {
returnReaderMissingSourceFormatError} / / this class Reader, which recorded the location of a play to currentPacket, / / play the position, // Play to the end of the package, according to the download parsing situation, divided into two situations // 1, download parsing is completed, play to the end of the package // 2, download is not completed, parsing is completed, play to the end of the package // (only two situations, because of the parsing time, Nowhere near as long as it took to download. Download completed = Parsing completed)let packetIndex = Int(reader.currentPacket)
let packets = reader.parser.packets
let isEndOfData = packetIndex >= packets.count - 1
if isEndOfData {
if reader.parser.isParsingComplete {
packetCount.pointee = 0
return ReaderReachedEndOfDataError
} else {
returnReaderNotEnoughDataError}} // The previous setting only processed one packet of audio data ata timelet packet = packets[packetIndex]
var data = packet.0
letDataCount = data. The count ioData. Pointee. MNumberBuffers = 1 / / audio data copies come over: Allocates memory, and then copy the address data ioData. Pointee. MBuffers. MData = UnsafeMutableRawPointer. The allocate (byteCount: dataCount, alignment: 0) _ = data.withUnsafeMutableBytes { (rawMutableBufferPointer)in
let bufferPointer = rawMutableBufferPointer.bindMemory(to: UInt8.self)
if letaddress = bufferPointer.baseAddress{ memcpy((ioData.pointee.mBuffers.mData? .assumingMemoryBound(to: UInt8.self))! , address, dataCount)}} ioData pointee. MBuffers. MDataByteSize = UInt32 (dataCount) / / processing compressed files to MP3, AAC ASPDlet sourceFormatDescription = sourceFormat.streamDescription.pointee
if sourceFormatDescription.mFormatID ! = kAudioFormatLinearPCM {ifoutPacketDescriptions? .pointee == nil { outPacketDescriptions? .pointee = UnsafeMutablePointer<AudioStreamPacketDescription>.allocate(capacity: 1) } outPacketDescriptions? .pointee? .pointee.mDataByteSize = UInt32(dataCount) outPacketDescriptions? .pointee? .pointee.mStartOffset = 0 outPacketDescriptions? .pointee? . Pointee. MVariableFramesInPacket = 0} packetCount. Pointee = 1 / / update the position of the broadcast to currentPacket reader. CurrentPacket = reader.currentPacket + 1return noErr;
}
Copy the code
4, using AVAudioEngine, playback and real-time sound processing
AVAudioEngine can do real-time sound processing and add effects with Effect units
4.1 play first
Set up AudioEngine, add nodes, connect nodes
func setupAudioEngine(){attachNodes() // attachNodes connectNodes() // prepare AudioEngine engine. Prepare () // AVAudioEngine data stream, Use push model // use timer, every 0.1 seconds or so, scheduling playback resourceslet interval = 1 / (readFormat.sampleRate / Double(readBufferSize))
let timer = Timer(timeInterval: interval / 2, repeats: true) {
[weak self] _ inguard self? .state ! = .stoppedelse {
return} // Allocate the buffer, schedule the playback resource self? .scheduleNextBuffer() self? .handleTimeUpdate() self? .notifyTimeUpdated() } RunLoop.current.add(timer,forMode:.common)} // Add the playback node open funcattachNodes() {engine. Attach (playerNode)} // Attach the playerNode to open funcconnectNodes() {
engine.connect(playerNode, to: engine.mainMixerNode, format: readFormat)
}
Copy the codeSchedule the playback resource and deliver the data (the audio buffer created in the previous step) to AudioEngine’s playerNode
func scheduleNextBuffer(){
guard let reader = reader else {
return} // Manage playback by status recording // playback status, is a switch guard! isFileSchedulingComplete || repeatselse {
return
}
do{// Create the audio bufferlet nextScheduledBuffer = try reader.read(readBufferSize) / / playerNode play consumed playerNode. ScheduleBuffer (nextScheduledBuffer)} catch ReaderError. ReachedEndOfFile { isFileSchedulingComplete =true
} catch { }
}
Copy the codeOpen play
public func play() {// Start guard! playerNode.isPlayingelse {
return
}
if! engine.isRunning {do{try engine.start()} catch {}} // To improve user experience, mute before playingletlastVolume = volumeRampTargetValue ?? Volume Volume = 0 // PlayerNode. play() // Play swellVolume(to: lastVolume) with normal volume after 250 ms // Update the playing state =.playing}Copy the codeAfter 4.2 sound
Add real-time pitch and playback speed effects
// Use AVAudioUnitTimePitch to adjust the playback speed and pitch effectlet timePitchNode = AVAudioUnitTimePitch()
override func attachNodes() {// Add playback node super.attachNodes() // Add sound node engine. Attach (timePitchNode)connectNodes() {
engine.connect(playerNode, to: timePitchNode, format: readFormat)
engine.connect(timePitchNode, to: engine.mainMixerNode, format: readFormat)
}
Copy the codeFill in the details
Duration 5
First get the number of packages, downloaded data, after parsing, add up
The first 2 minutes and 34 seconds of mp3 can be divided into 5925 packages
public var totalPacketCount: AVAudioPacketCount? {
guard let _ = dataFormat else {
returnParserPacketCallback = AVAudioPacketCount(packets.count); // ParserPacketCallback = AVAudioPacketCount(packets.count); Add data to packetsreturn max(AVAudioPacketCount(packetCount), AVAudioPacketCount(packets.count))
}
Copy the codeGet the total number of audio frames
public var totalFrameCount: AVAudioFrameCount? {
guard letframesPerPacket = dataFormat? .streamDescription.pointee.mFramesPerPacketelse {
return nil
}
guard let totalPacketCount = totalPacketCount else {
returnNil} // The total number of packets in the previous step X how many frames are in each packetreturn AVAudioFrameCount(totalPacketCount) * AVAudioFrameCount(framesPerPacket)
}
Copy the codeCalculate the audio duration
public var duration: TimeInterval? {
guard letsampleRate = dataFormat? .sampleRateelse {
return nil
}
guard let totalFrameCount = totalFrameCount else {
returnNil} // Total number of audio frames from the previous step/sampling ratereturn TimeInterval(totalFrameCount) / TimeInterval(sampleRate)
}
Copy the code6. Adjust the current position of playback
6.1 Audio Playback manager inside the Streamer
Public func seek(to time: TimeInterval) throws {// Use the Parser audio package and reader audio buffer to play guardlet parser = parser, let reader = reader else {
return} // Take the time to calculate the relative position of the audio frame; // Take the relative position of the audio framelet frameOffset = parser.frameOffset(forTime: time),
let packetOffset = parser.packetOffset(forFrame: frameOffset) else {
return} currentTimeOffset = time isFileSchedulingComplete =false// Record the current statuslet isPlaying = playerNode.isPlaying
letlastVolume = volumeRampTargetValue ?? Playernode.stop () volume = 0 // Update the location of reader resourcesdo {
try reader.seek(packetOffset)
} catch {
return} // Just record the current state, restoreifIsPlaying {playerNode.play()} // Update UI delegate? Streamer (self, updatedCurrentTime: time) swellVolume(to: lastVolume)}Copy the codeFigure out the frame offset for the current time
public func frameOffset(forTime time: TimeInterval) -> AVAudioFramePosition? {
guard let_ = dataFormat? .streamDescription.pointee,let frameCount = totalFrameCount,
let duration = duration else {
returnNil} // Take the current time/total audio duration, calculate the ratiolet ratio = time / duration
return AVAudioFramePosition(Double(frameCount) * ratio)
}
Copy the codeFigure out the packet position of the current frame
public func packetOffset(forFrame frame: AVAudioFramePosition) -> AVAudioPacketCount? {
guard letframesPerPacket = dataFormat? .streamDescription.pointee.mFramesPerPacketelse {
returnNil} // How many frames are there in a packetreturn AVAudioPacketCount(frame) / AVAudioPacketCount(framesPerPacket)
}
Copy the code6.2 Audio Resource scheduling in reader
Public func seek(_ packet: AVAudioPacketCount) throws {queue.sync {currentPacket = packet}}Copy the codeRecord the position currentPacket, so it works
ReaderConverterCallback in Step 3
/ /... // In this example, an audio packet corresponds to an audio bufferlet packet = packets[packetIndex]
var data = packet.0
// ...
_ = data.withUnsafeMutableBytes { (rawMutableBufferPointer) in // ...
}
// ...
Copy the code
7 UI user experience is improved. Manually drag and drop the scene of playing time
It is divided into three events: finger press down, finger drag, finger lift
/ / finger press, screen refresh play the progress of the proxy method @ IBAction func progressSliderTouchedDown (_ sender: UISlider) {isSeeking =true} / / finger drag, screen refresh broadcast pace agent method, the use of gestures corresponding UI @ IBAction func progressSliderValueChanged (_ sender: UISlider) {letCurrentTime = TimeInterval(progressSlider.value) currentTimelabel.text = currentTime.tommss ()} @ibAction func progressSliderTouchedUp(_ sender: UISlider) {seek(sender) isSeeking =false
}
Copy the codeThe related agent method updates the UI of the current event and progress bar based on the playback progress
If you’re dragging it, block it
func streamer(_ streamer: Streaming, updatedCurrentTime currentTime: TimeInterval) {
if! isSeeking { progressSlider.value = Float(currentTime) currentTimeLabel.text = currentTime.toMMSS() } }Copy the code8 Single track loop mode
Step 4 During playback, distribute playback resources using a timer
Manage the logic of the two methods below
(Scheduling audio buffer and changing state after playing)
let timer = Timer(timeInterval: interval / 2, repeats: true) {
[weak self] _ in/ /... self? .scheduleNextBuffer() self? .handleTimeUpdate() // ... }Copy the codeScheduling audio buffer,
func scheduleNextBuffer(){
guard let reader = reader else {
return} // If repeats are repeated, continue to play, regardless of the guard! isFileSchedulingComplete || repeatselse {
return} / /... The following is, the player node plays resources}Copy the codeHandle related states according to the playback situation
func handleTimeUpdate(){
guard let currentTime = currentTime, let duration = duration else {
return} // The current playback time, after the audio length, it is considered to have finished playing, to pauseifcurrentTime >= duration { try? Seek (to: 0) // If repeated, do not pauseif! repeats{ pause() } } }Copy the code