Audio session

AVAudioSession (audio session) participation. Audio session is the intermediary between the application and the operating system, scheduling system audio functions through semantic description.

Audio session classification

AVFoundation defines seven categories to describe the audio behavior used by applications, as shown in the table below.

classification role Whether to allow remixing Audio input Audio output
Ambient Games, productivity apps allow allow
Solo Ambient Games, productivity apps allow
Playback Audio and video players optional allow
Record Tape recorder, audio capture allow
Play And Record VoIP and voice chat optional allow allow
Audio Processing Offline session and processing
Multi-Route Advanced A/V applications using external hardware allow allow

There are a few things you need to think about when choosing the right category for your application. Is audio playback a core or secondary feature? Can application audio be mixed with background sound? Does the application need to capture audio input for recording or send audio over the network? Once you’ve identified the core audio for your application, it becomes easier to choose the appropriate category.

Configuring an Audio Session

AVAudioSession configuration in the application’s life cycle can be modified, but usually only to its configuration, generally in the boot method application (_ : configure didFinishLaunchingWithOptions:), as shown in the following code:

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions { AVAudioSession *session = [AVAudioSession sharedInstaance]; NSError *error; if (! [session setCategory:AVAudioSessionCategoryPlayback error:&error]) { NSLog(@"Category Error:%@",[error localizedDescription]); } if (! [session setActive:YES error:&error]) { NSLog(@"ACtivation Error:%@",[error localizedDescription]); } return YES; }Copy the code
Use AVAudioPlayer to play audio

AVAudioPlayer is the first choice of AV Foundation for Audio playback, or the first choice of the entire iOS Audio playback. It provides all the core functions in the Audio Queue Service and is suitable for local playback or scenarios that are not sensitive to latency. AVAudioPlayer is created with two types of interfaces, one is for local path files such as init(contentsOf: URL) and the other is for memory Data such as init(Data: Data). The initialization reference code is as follows:

NSURL *fileURL = ... ; self.player = ... if (self.player) { [self.player prepareToPlay]; }Copy the code

It is recommended to call prepareToPlay() first at initialization time because the required audio hardware is fetched and the AudioQueue buffer is preloaded before the play() method is called, reducing the delay between calling the play() method and hearing the sound output. If the prepareToPlay() method is not called, a method like prepareToPlay() is also implicitly called when the play() method is called to activate the audio

AVAudioPlayer provides a number of lifecycle control methods for playback, as follows:

  • Play () : Officially plays the audio, and can resume the audio stopped by pause and stop
  • Pause () : Pause audio playback and resume it from play without clearing the content in prepareToPlay
  • Stop () : Stop audio playback, clear prepareToPlay, and resume from play

AVAudioPlayer also provides some property variables for audio control, as shown below:

  • Volume: Changes the volume of the player. The value ranges from 0.0 to 1.0, in floating point values
  • Pan: The sound channel of the player, ranging from -1 (far left) to 1.0 (far right), default is 0.0 (center)
  • Rate: Adjust the playback rate from 0.5 to 2.0
  • NumberOfLoops: numberOfLoops (n > 0) for n loops (n = -1
  • IsMeteringEnabled: whether to enable audio metering, that is, output visual audio metering data
background

When playing audio, a very common scenario is to leave the App out of the foreground and continue playing audio in the background until the user stops.

It’s not hard to play audio in the background. It only takes two steps:

  1. Setting the audio session category to Playback enables the audio session to play even when the phone is silent
  2. You need to add an array of Required background modes in the info.plist file and an item for App Plays Audio or Streams audio/video using AirPlay

With these two steps, you can make the audio play continue to serve in the background.

Interrupt handling

Sometimes the audio is interrupted by a phone call or a Face Time call. When the user rejects the call or the call ends, the audio starts playing again from the paused position. The successful implementation of this sequence of operations relies on AVAudioSession’s interrupt notifications. By listening for interrupt notifications, the system tells the outside world when an interrupt starts or ends. The sample code looks like this:

func setupNotifications() {
    let nc = NotificationCenter.default
    nc.addObserver(self,
                   selector: #selector(handleInterruption),
                   name: AVAudioSession.interruptionNotification,
                   object: AVAudioSession.sharedInstance)
}

@objc func handleInterruption(notification: Notification) {
    
}
Copy the code
  • The interrupt notification contains a userInfo with important information that determines the behavior of the audio, pause or play
  • HandleInterruption (Notification 🙂 : Used to centrally handle interrupt notifications

Example code for handling interrupt notifications in handleInterruption(Notification 🙂 :

@objc func handleInterruption(notification: Notification) {
    guard let userInfo = notification.userInfo,
        let typeValue = userInfo[AVAudioSessionInterruptionTypeKey] as? UInt,
        let type = AVAudioSession.InterruptionType(rawValue: typeValue) else {
            return
    }

    switch type {

    case .began:

    case .ended:
       
        guard let optionsValue = userInfo[AVAudioSessionInterruptionOptionKey] as? UInt else { return }
        let options = AVAudioSession.InterruptionOptions(rawValue: optionsValue)
        if options.contains(.shouldResume) {
           
        } else {
            
        }

    default: ()
    }
}
Copy the code
Route replacement treatment

When using music software, there is often a scene, such as switching from speakers to headphones, or from headphones to speakers and so on. Sometimes it’s dangerous to keep playing the user’s audio while the headset is switched to speaker, because the user may be listening to something very private.

Because of this demand scenarios, AVAudioSession provides line change notification, when the line on the mobile devices (such as speakers switch to headphones) changes, will trigger the AVAudioSession. RouteChangeNotification notification for developers, Developers need to follow the iOS User Experience Guidelines for allowing audio to play or pause.

The sample code for listening for route changes is as follows:

func setupNotifications() {
    let nc = NotificationCenter.default
    nc.addObserver(self,
                   selector: #selector(handleRouteChange),
                   name: AVAudioSession.routeChangeNotification,
                   object: nil)
}

@objc func handleRouteChange(notification: Notification) {
}
Copy the code
  • This notification is issued when the output audio or output device changes
  • Notification contains a userInfo dictionary to get the reason why the notification was sent and the description of the previous line

Example code for handling handleRouteChange(Notification 🙂

@objc func handleRouteChange(notification: Notification) {guard let userInfo = notification.userInfo, let reasonValue = userInfo[AVAudioSessionRouteChangeReasonKey] as? UInt, let reason = AVAudioSession.RouteChangeReason(rawValue: ReasonValue) else {return} switch reason {case.newDeviceAvailable: / / find new equipment let session. = AVAudioSession sharedInstance () headphonesConnected = hasHeadphones (in: Session.currentroute) case.oldDeviceUnavailable :// The old device is disconnected // Obtain the line description if let previousRoute = userInfo[AVAudioSessionRouteChangePreviousRouteKey] as? AVAudioSessionRouteDescription { headphonesConnected = hasHeadphones(in: previousRoute) } default: ()}} func hasHeadphones (in routeDescription: AVAudioSessionRouteDescription) - > Bool {/ / find the first outlet, judge whether the headset entrance return! routeDescription.outputs.filter({$0.portType == .headphones}).isEmpty }Copy the code
Use AVAudioRecorder to record audio

AVAudioRecorder is one of the AV Foundation used for Audio recording interfaces, is an advanced package of Audio Queue Services, using AVAudioRecorder recording is not complex.

AVAudioRecorder creation process is very simple, there are two main steps:

  1. Generate a URL that is appended to AVAudioRecorder as the audio stream write address
  2. Generate a dictionary Settings that regulates the format of the audio stream, also attached to AVAudioRecorder

Example code for the AVAudioRecorder creation process:

do {
    self.recorder = try AVAudioRecorder(url: fileURL, settings: setting)
    self.recorder.delegate = self
    self.recorder.isMeteringEnabled = true
    self.recorder.prepareToRecord()
} catch {
    fatalError(error.localizedDescription)
}
Copy the code
  • The prepareToRecord() method initializes the resources required for the record, including creating files, and minimizes the delay in starting the record
  • The key value information in setting includes audio format, sampling rate, etc
  • The suffix of the URL file path should correspond to the audio format, otherwise there will be problems

Setting is used to specify the recording format of an audio stream. Common key values are:

  • AVFormatIDKey: audio format
  • AVSampleRateKey: sampling rate
  • AVNumberOfChannelsKey: Indicates the number of channels
  • AVEncoderBitDepthHintKey: quantization bit
  • AVEncoderAudioQualityKey: sound quality

When using AVAudioRecorder to record audio, it is necessary to set the audio session classification as playAndRecord, create AVAudioRecorder, and implement AVAudioRecordeDelegate protocol. The content of AVAudioRecordeDelegate is simple and consists mainly of callbacks for recording completion and recording errors. Other methods are largely obsolete.

Text to speech

AV Foundation provides a speech synthesis framework for managing speech and speech synthesis. One of the most common features is text to speech, AVSpeechSynthesisVoice

There are only two steps needed to enable the text-to-speech function in the App:

  1. Create an AVSpeechUtterance object and append content strings, speech parameters such as sound, speed, etc
  2. The AVSpeechUtterance object is attached to the AVSpeechSynthesisVoice instance, which controls the life cycle of the speech

The following is a code example:

// Create AVSpeechUtterance instance and append character content let utterance = AVSpeechUtterance(string: "The quick brown fox jumped over The lazy dog.") utterance. Rate = 0.57 / / rate utterance pitchMultiplier = 0.8 Utterance. PostUtteranceDelay = 0.2 utterance. Volume = 0.8 / / volume let voice = AVSpeechSynthesisVoice (language: "en-GB") utterance.voice = voice let synthesizer = AVSpeechSynthesizer() synthesizer.speak(utterance)Copy the code

AVSpeechUtterance instance also have corresponding a Delegate, namely AVSpeechSynthesizerDelegate, mainly for life cycle in the process of writing voice callback, have time the reader can look at our related API.