IOS Audio and Video (1) AVFoundation core class

IOS Audio and Video (2) AVFoundation video capture

IOS Audio and Video (3) AVFoundation playback and recording

IOS Audio and Video (43) AVFoundation Audio Session

IOS Audio Queue Services for AVFoundation

IOS Audio and Video (45) HTTPS self-signed certificates implement side play

IOS Audio and Video (46) Offline online speech recognition solution

1. Audio session overview

Audio is a hosted service in iOS, tvOS, and watchOS. The system uses audio sessions to manage audio behavior at the application, inter-application, and device levels.

You use an audio session to communicate to the system how you plan to use audio in your application. This audio session acts as a mediator between your application and the operating system, and in turn, between the underlying audio hardware. You can use it to communicate the nature of your application’s audio to the operating system without having to elaborate on specific behavior or the necessary interaction with the audio hardware. Delegating the management of these details to the audio session ensures the best management of the user’s audio experience.

An example of your audio session with your application using AVAudioSession:

  1. Configure the audio session categories and modes to communicate to the system how you intend to use audio in your application
  2. Activate your app’s audio session and configure your categories and modes to action
  3. Subscribe to and respond to important audio session notifications, such as audio interrupts and route changes
  4. Perform advanced audio device configuration, such as setting the sampling rate, I/O buffer duration, and number of channels

  • Audio session management Audio Behavior An audio session is an intermediary between an application and the operating system that configures the audio behavior of an application. At startup, your application will automatically provide a singleton audio session. You can configure it to provide the desired behavior and activate it to turn that behavior into an action.

  • Categories represent audio roles

The primary mechanism for expressing audio behavior is the audio session category. By setting categories, you can indicate whether the application uses input or output paths, whether you want the music to continue playing with the audio, and so on. The behavior you specify should meet the user’s expectations, as described in the iOS Human Interface Guide audio.

AVFoundation defines a number of audio session categories, as well as a set of override and modify switches that allow you to customize audio behavior based on your application’s personality or role. Various categories support playback, recording, and playback with recording. When the system knows the audio role of your application, it will provide you with appropriate access to hardware resources. The system also ensures that other audio on the device works in a way that suits your application and meets user expectations.

Some categories can be further customized by specifying patterns used to specialize the behavior of a given category. For example, when the application uses video recording mode, the system may select a built-in microphone that is different from the default mode. The system may also employ microphone signal processing tuned for video recording use cases.

  • Notification Support interrupt handling Audio interrupts are the deactivation of an application’s audio session, which immediately stops the audio. An interrupt occurs when a competing audio session from an application is activated and the session is not classified by the system to mix with your session. Your application should respond to interruptions by saving state, updating the user interface, and so on. To be notified when audio interrupt the start and end, please register to observe AVAudioSessionInterruptionNotification type of notification.

  • Notification support Audio routing change handling When users initiate audio routing changes by connecting or disconnecting devices, inserting or disconnecting headphones, they have specific expectations. The iOS Human Interface Guide describes these expectations and provides guidance on how to meet them. By registering to observe AVAudioSessionRouteChangeNotification type of notification to handle routing updates.

  • The audio Session control device configuration application does not directly control the device hardware, but the audio session provides you with an interface to request Settings for your preferred hardware device. This interface enables you to perform advanced audio device configurations, such as setting the sampling rate, I/O buffer duration, and the number of audio channels.

  • Audio sessions Protect user privacy Applications that record audio alone or with video require explicit user permission before allowing recording. Until the user grants your app recording permission, your app can only record mute. AVAudioSession provides an interface to request this permission and determine the user’s privacy Settings.

2. Configure an audio session

The audio session category is a key that identifies a set of audio behaviors for an application. By setting a category, you can indicate to the system your audio intentions — for example, whether the audio should continue when the ringer/mute switch is flipped. Several audio session categories, as well as a set of overrides and modify switches, let you customize the audio behavior of your application.

As shown in Table B-1, each audio session category specifies a specific set of responses in response to each of the following behaviors:

  1. Interrupt unmixed application audio: If so, the unmixed application is interrupted when your application activates its audio session.
  2. Mute switch Mute: If yes, your audio will be muted when the user activates the mute switch. (On the iPhone, this switch is called the ringer/Mute switch.)
  3. Audio input support: If supported, app audio input (recording) is allowed.
  4. Audio output support: If so, application audio output (playback) is allowed.

Most apps only need to set categories once at startup, but you can change categories at any time as needed. You can change it, and audio sessions are positive; However, it is usually preferable to disable the audio session before changing the category or other session attributes. Making these changes when the session is disabled prevents unnecessary reconfiguration of the audio system.

2.1 Default behavior of audio sessions

All iOS, tvOS, and watchOS apps have a preset default audio session like this:

  • Audio playback is supported, but audio recording is not allowed.
  • In iOS, setting the ringer/Mute switch to silent mode will mute any audio played by the app.
  • In iOS, the app’s audio is muted when the device is locked.
  • When your application plays audio, any other background audio (such as that played by a music application) will be muted.

The default audio session has useful behavior, but in most cases, you should customize it to better meet the needs of your application. To change the behavior, you need to configure the application’s audio session.

2.2 Configuring An Audio Session

The main way to configure an audio session is to set its category. The audio session category defines a set of audio behaviors. The precise behavior associated with each category is not under the control of the application, but is set by the operating system. Apple will likely refine the category behavior in future versions of its operating system, so your best strategy is to choose the category that most accurately describes the intent of the audio behavior you want. Audio session categories and patterns summarize the behavior details for each category.

Although categories set the basic audio behavior of your application, you can further specialize these behaviors by setting the mode of the category. For example, voice over IP (VoIP) applications will use AVAudioSessionCategoryPlayAndRecord. You can set the audio session mode to AVAudioSessionModeVoiceChat VoIP applications to specialization, this kind of behavior. This mode ensures that the speech signal is optimized through the signal processing provided by the system.

AVAudioSessionCategoryPlayAndRecord model is as follows:

static let playAndRecord: AVAudioSession.Category
Copy the code
  1. Your audio continues with mute switch set to mute and with screen lock. (This switch is called the ringer/Mute switch on the iPhone.) To continue playing the audio when your application transitions to the background (for example, when the screen is locked), add the audio values to the UIBackgroundModes key in your info properties list file.
  2. This category applies to both recording and playing back, as well as applications that record and play back but not simultaneously.
  3. By default, using this category means that your app’s audio is non-mixable — activating your session will interrupt any other non-mixable audio session. To allow mixing of this category, use the mixWithOthers option.
  4. Users must grant permissions for audio recording (see Recording requires user permissions). This category supports mirrored versions of Airplay. If AVAudioSessionModeVoiceChat model used in this category, however, the AirPlay mirroring will be disabled.
  5. Some categories support through setting one or more categories on the session options to override the default behavior (see AVAudioSessionCategoryOptions). For example, when the session is activated, the default behavior associated with AVAudioSessionCategoryPlayback category will interrupt other audio system. In most cases, playback applications require this behavior. However, if you want your audio audio mixing with other systems, you can through the Settings in the dialog AVAudioSessionCategoryOptionMixWithOthers options to override this behavior.
  • Audio session modes include:

Note: In order for your application to continue playing audio when the ringer/Mute switch is set to silent and the screen is locked, make sure that the UIBackgroundModes audio keys have been added to your application’s messages. File. This requirement is except that you use the correct category.

  • Models and related categories:

    To set up the audio session category (and optional modes and options), please call setCategory: mode: options: error: method, using AVFoundation framework set the audio session category code is as follows:

// Access the shared, singleton audio session instance
let session = AVAudioSession.sharedInstance()
do {
    // Configure the audio session for movie playback
    try session.setCategory(AVAudioSessionCategoryPlayback,
                            mode: AVAudioSessionModeMoviePlayback,
                            options: [])
} catch let error as NSError {
    print("Failed to set the audio session category and mode: \(error.localizedDescription)")}Copy the code

2.3 Extending options with Multiple Routing Categories

The multi-routing category works slightly differently than the other categories. All other categories of devices follow the “last-to-network” rule, where the last device to insert an input or output route is the dominant device. However, the multi-routing class allows an application to use all the output ports of the connection, rather than just the last port. For example, if you are listening to audio over the HDMI output path and plug in a set of headphones, your application will continue to play audio over the HDMI output path as well as over the headphones.

Using the Multiroute category, your application can also send different audio streams to different output paths. For example, your application could send one audio stream to the left earphone, another to the right earphone, and a third to an HDMI route. Figure 1-1 shows an example of sending multiple audio streams to different audio routes.

Depending on the device and any connected attachments, the following is a valid combination of output paths:

  • USB and headphone
  • HDMI and headphones
  • Line up for scrimmage and headphones

Multiple routing categories support a single input port.

Important: Use the built-in speaker only when no other suitable output port (USB, HDMI, LineOut) is available.

2.4 Select categories and modes for AirPlay

Only certain categories and modes support AirPlay. The following categories support both mirrored and unmirrored versions of AirPlay:

AVAudioSessionCategorySoloAmbient AVAudioSessionCategoryAmbient AVAudioSessionCategoryPlayback

AVAudioSessionCategoryPlayAndRecord categories and the following model supports only the mirror version of AirPlay:

AVAudioSessionModeDefault AVAudioSessionModeVideoChat AVAudioSessionModeGameChat

Note: since ios10, when you use AVAudioSessionCategoryPlayAndRecord directory, You can through AVAudioSessionCategoryOptionAllowAirPlay options to activate your session to enable the output mirror AirPlay.

AVAudioSessionCategoryOptionAllowAirPlay = 0x40

  1. Only in the audio session category of AVAudioSessionCategoryPlayAndRecord, explicitly set this option. For most other audio session categories, this option is set implicitly. Using AVAudioSessionCategoryMultiRoute or AVAudioSessionCategoryRecord category of audio session implicitly to remove this option.
  2. If you clear this option, the AirPlay device will not display as an available audio output route. If this option is set, these devices are displayed as available output routes.

2.5 Enabling Background Audio

IOS and tvOS applications require you to enable certain features for certain background operations. A common feature required by playback applications is to play background audio. With this feature enabled, the app’s audio can continue when the user switches to another app or locks the iOS device. It is also required to support advanced playback features in iOS, such as AirPlay streaming and playback of pictures within pictures.

The easiest way to configure these features is to use Xcode. Select the target for the application in Xcode and select the Capability TAB. Under the Capability TAB, set the background mode to ON and select the “Audio, AirPlay, and Picture in Picture” option from the list of available modes.

With background mode enabled and audio sessions configured to the appropriate category, your application can start playing background audio

Note: To allow the audio portion of a video presentation to play in the background, see Play Background Audio in the Media Play Programming guide.

3. Activate the audio session

You have configured the audio session by setting its categories, options, and modes. For the configuration to take effect, you now need to activate the audio session.

3.1 How does the system address competing audio requirements

Built-in apps (messaging, music, Safari, phone) may run in the background when your app launches. Each can produce audio: a text message comes in, a podcast you started 10 minutes ago is still playing, and so on.

If you think of a device as an airport, the app is like a taxiing plane, and the system is like a control tower. Your application can make audio requests and claim the priority it wants, but the ultimate authority on what happens “on the tarmac” lies with the system. Use audio sessions to communicate with the Control tower. Figure 2-1 shows a typical scenario – your application requires the use of audio while the music application is already playing. In this scenario, your application interrupts the music application.

In step 1 of the figure, the application requests activation of its audio session. For example, you might make a request like this when the application starts, or when the user taps the Play button in the audio recording and playback application. In Step 2, the system will consider the activation request. Specifically, it takes into account the categories you assign to the audio session. In Figure 2-1, your application uses a category that requires muffling of other audio.

In Steps 3 and 4, the system closes the audio session of the music application and stops its audio playback. Finally, in step 5, the system activates your app’s audio session and playback can begin.

3.2 Activating and closing your audio session

While the AVFoundation playback and recording classes automatically activate an audio session, manually activating it gives you a chance to test whether the activation is successful. However, if your application has a Play/Pause UI element, write your code so that the user must press Play before activating the session. Similarly, when changing the active/inactive status of an audio session, check to make sure the call is successful. Write code to gracefully handle system refusal to activate your session.

The system will close your audio session for a clock or calendar alarm or incoming call. The system allows your session to be activated again when the user deactivates the alarm or chooses to ignore the phone call. Whether to reactivate the session at the end of the interrupt depends on the application type, as described in Audio Guidelines By App Type.

  • The code to activate an audio session is as follows:
let session = AVAudioSession.sharedInstance()
do {
    // 1) Configure your audio session category, options, and mode
    // 2) Activate your audio session to enable your custom configuration
    try session.setActive(true)}catch let error as NSError {
    print("Unable to activate audio session:  \(error.localizedDescription)")}Copy the code

To disable the audio session, pass false to the setActive method.

When audio is played or recorded using AVFoundation objects (AVPlayer, AVAudioRecorder, etc.), the system is responsible for reactivating the audio session when the interruption ends. However, if you register the notification message and explicitly reactivate the audio session, you can verify that the reactivation was successful and you can update the application’s state and user interface. For more information, see Figure 3-1.

Many applications never need to explicitly disable their audio sessions. Important exceptions include VoIP applications, turn-by-turn navigation applications, and in some cases playback and recording applications.

  • Ensure that the VoIP application’s audio session, which typically runs in the background, is active only when the application processes the call. In the background, the audio session of the VoIP application should not be active when you are ready to receive a call.
  • Ensure that the audio session of an application that uses the recording category is active only at the time of recording. Before starting and stopping recording, make sure your session is inactive to allow other sounds, such as incoming message alerts, to play.
  • If the application supports background audio playback or recording, and if the application is not actively using audio (or preparing to use audio), disable its audio session when entering the background. Doing so allows the system to free up audio resources so that other processes can use them. It can also prevent the application’s audio session in the application process by the operating system hangs when failure (see AVAudioSessionInterruptionWasSuspendedKey).

The only appear in the userInfo key AVAudioSession. InterruptionType. The start interrupt event, where the interrupt is a direct result of the operating system suspending the application. Its associated value is a Boolean NSNumber, where a true value indicates that the interrupt was caused by the system suspending the application and not by another audio session.

3.2.1 Audio Guidelines By app type

3.3 Checking whether Other Audio files are being played

When your app is activated, the sound may already be playing on the device. For example, the music app might be playing a song when the user launches the app, or Safari might be playing an audio stream. If your app is a game, it’s especially important to know if other audio is playing. Many games have music tracks and sound effects. In the iOS man-machine Interface guide, it is recommended that you assume that the user expects other audio and game sounds to continue while playing the game.

In your application delegate applicationDidBecomeActive: method, check the audio session secondaryAudioShouldBeSilencedHint attribute, to determine whether the audio is in the air. This value is true when another application with an unmixed audio session is playing audio. Applications should use this property as a prompt to mute audio that is secondary to application functionality. For example, a game using AVAudioSessionCategoryAmbient can use this property to determine whether it should be in keeping the sound effect under the condition of not mute mute audio tracks.

You can also subscribe to the type of AVAudioSessionSilenceSecondaryAudioHintNotification notice, to ensure that in the optional auxiliary audio mute should inform you at either the end or the beginning of the application. This notification is only sent to registered listeners that are currently in the foreground and have active audio sessions.

func setupNotifications(a) {
    NotificationCenter.default.addObserver(self,
                                           selector: #selector(handleSecondaryAudio),
                                           name: .AVAudioSessionSilenceSecondaryAudioHint,
                                           object: AVAudioSession.sharedInstance())
}
 
func handleSecondaryAudio(notification: Notification) {
    // Determine hint type
    guard let userInfo = notification.userInfo,
        let typeValue = userInfo[AVAudioSessionSilenceSecondaryAudioHintTypeKey] as? UInt.let type = AVAudioSessionSilenceSecondaryAudioHintType(rawValue: typeValue) else {
            return
    }
 
    if type == .begin {
        // Other app audio started playing - mute secondary audio
    } else {
        // Other app audio stopped playing - restart secondary audio}}Copy the code

The notice of the userInfo dictionary contains AVAudioSessionSilenceSecondaryAudioHintType AVAudioSessionSilenceSecondaryAudioHintTypeKey value. Use the audio prompt type to determine whether the auxiliary audio mute should start or end

4. The response is interrupted

Add audio session code to handle interrupts, ensuring that your application’s audio continues to function properly when a phone call arrives, a clock or calendar alarm goes off, or another application activates its audio session.

An audio interrupt is the deactivation of an application’s audio session, which immediately stops the audio. An interrupt occurs when a competing audio session from an application is activated and the session is not classified by the system to mix with your session. After your session becomes inactive, the system will send a “you are interrupted” message, which you can respond to by saving the status, updating the user interface, and so on.

Your application may pause after an interruption. This happens when the user answers the phone. If the user ignores a call or cancels an alert, the system will issue an “interruption over” message and your application will continue running. To restore your audio, you must reactivate your audio session.

4.1 Interrupt the life cycle

Figure 3-1 illustrates the sequence of events before, during, and after the audio session break for the playback application.

The interrupt event — in this case, the arrival of the FaceTime request — happens as follows. The numbering steps correspond to the numbers in the diagram.

  1. Your app is active, playing audio.
  2. The FaceTime request arrived. The system activates the audio session of the FaceTime application.
  3. The system will close your audio session. At this point, playback in the application has stopped.
  4. A notification will be sent indicating that your session has been disabled.
  5. Your notification handler will take the appropriate action. For example, it can update the user interface and save the information needed to resume playback when playback is stopped.
  6. If the user rejects the interrupt (ignoring an incoming FaceTime request), a notification is sent indicating that the interrupt is over.
  7. The notification handler takes the appropriate action when the interrupt ends. For example, it can update the user interface, reactivate an audio session, and resume playback.
  8. (Not shown) If the user answers a call instead of canceling the interrupt in step 6, your application will be suspended.

4.2 Audio interrupt processing technology

Handle interrupts by registering to observe interruption notifications published by AVAudioSession. What you do in interrupt code depends on the audio technology you are using, as well as the technology you are using for playback, recording, audio format conversion, reading streaming audio packages, and so on. In general, from the user’s point of view, you want to ensure as few outages as possible and as good a recovery as possible.

Table 3-1 summarizes the appropriate audio session behavior during interruptions. If you use AVFoundation to play back or record objects, some of these steps are handled automatically.

Table 3-1 What should happen during an audio session interruption:

time To deal with
After the interruption starts 1. Save status and context, 2. Updated user interface
After interruption 1. Restore state and context, 2. Update user interface, 3. Reactivate audio session if appropriate application

Table 3-2 summarizes how to handle audio interrupts according to technology:

Audio technology Interrupt handling
1. AVFoundation framework The system automatically pauses playing or recording interrupts and reactivates your audio session when you resume playing or recording. 2. If you want to save and resume playing position between app startup, save playing position in interrupt as well as in app exit.
Audio Queue Services, I/O audio unit These techniques give your application control over handling interrupts. You are responsible for saving the playback or recording location and reactivating the audio session after the interruption ends.
System Sound Services When the interruption starts, the sound played using the system sound service will be muted. If the interruption ends, the sound can be played again. Applications cannot affect the interrupt behavior of sounds using this playback technology.

4.3 Handling Siri Interruption

  • When Siri interrupts your app playback, you’ll have to keep track of any remote control commands Siri issues while the audio session is interrupted. Keep track of all commands Siri issues during the interruption and respond accordingly when the interruption ends. For example, during an outage, the user asked Siri to pause the app’s audio playback. When your application is notified that the interruption has ended, it should not automatically resume playing. Instead, the application’s UI should indicate that the application is in a suspended state.

4.4 Observe audio interruption

  • To deal with audio interrupt, first register AVAudioSessionInterruptionNotification type of notification.
func registerForNotifications(a) {
    NotificationCenter.default.addObserver(self,
                                           selector: #selector(handleInterruption),
                                           name: .AVAudioSessionInterruption,
                                           object: AVAudioSession.sharedInstance())
}
 
func handleInterruption(_ notification: Notification) {
    // Handle interruption
}

Copy the code
  • The published NSNotification instance contains a populated userInfo dictionary that provides details of the interrupt. From the userInfo dictionary retrieve AVAudioSessionInterruptionType values to determine the type of interrupt. The interrupt type indicates whether the interrupt has started or ended.
func handleInterruption(_ notification: Notification) {
    guard let info = notification.userInfo,
        let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt.let type = AVAudioSessionInterruptionType(rawValue: typeValue) else {
            return
    }
    if type == .began {
        // Interruption began, take appropriate actions (save state, update user interface)
    }
    else if type == .ended {
        guard let optionsValue =
            userInfo[AVAudioSessionInterruptionOptionKey] as? UInt else {
                return
        }
        let options = AVAudioSessionInterruptionOptions(rawValue: optionsValue)
        if options.contains(.shouldResume) {
            // Interruption Ended - playback should resume}}}Copy the code
  • If the interrupt type is AVAudioSessionInterruptionTypeEnded the userInfo dictionary may contain AVAudioSessionInterruptionOptions values. AVAudioSessionInterruptionOptionShouldResume an option value is a hint, indicating there is, if the application was interrupted if it should be automatic recovery. Media player applications should always look for this flag before starting playing after an interrupt. If it does not exist, the playback should not start again until initiated by the user. Applications without a playback interface, such as games, can ignore this flag and reactivate and resume playback when the interruption ends.

Note: There is no guarantee that the beginning interrupt will be followed by the end interrupt. Your application needs to know a switch to the foreground running state or when the user presses the play button. In both cases, determine whether your application should reactivate its audio session.

4.5 Responding to media Server reset

Media servers provide audio and other multimedia capabilities through shared server processes. Although rare, the media server can be reset while your application is active. Registered AVAudioSessionMediaServicesWereResetNotification notice to monitor media server reset. After receiving the notification, your app needs to do the following:

  • Handles isolated audio objects (such as players, recorders, converters, or audio queues) and creates new objects
  • Resets any tracked internal audio state, including all properties of AVAudioSession
  • When appropriate, the AVAudioSession instance is reactivated using the setActive:error: method

5. Respond to route changes

When your application is running, the user may plug in or unplug a headset, or use a docking station with an audio connection. The iOS Human Interface Guide describes how applications should respond to such incidents. To implement these recommendations, write audio session code to handle audio hardware routing changes. Some types of applications, such as games, do not always need to respond to route changes. However, other types of applications, such as media players, must respond to all routing changes.

5.1 Various audio hardware routing changes

Audio hardware routing is the wired electronic path of audio signals. When the user of the device inserts or unplugs the headset, the system automatically changes the audio hardware route. If you registered AVAudioSessionRouteChangeNotification type of notification, your application can be notified of these changes.

Figure 4-1 depicts the sequence of events for various routing changes during recording and playback. The bottom of the diagram shows four possible results from the actions taken by the property listener callback function you wrote.

As shown, after your app starts, the system initially determines the audio route. It will continue to monitor active routes while the application is running. Consider first when the user clicks the Record button in the application, which is represented by the Start Record box on the left side of the figure.

During recording, the user can insert or remove the headset, as shown in the diamond-shaped decision element at the bottom left. In response, the system sends a route change notification containing the reason for the change and the previous route. Your application should stop recording.

Playback is similar, but with different results, as shown on the right. If the user unplugs the headset during playback, the app should pause the audio. If the user plugs in a headset during playback, the application should allow playback to continue.

5.2 Observing the Changes of audio routes

  • Audio routing changes can occur for a number of reasons, including users plugging in a pair of headphones, connecting a Bluetooth LE headset, or unplugging the USB audio port. It may be important for your application to know when these changes occur, so you can update its user interface or change its internal state. By registering AVAudioSessionRouteChangeNotification type of notification, can inform you when audio routing updates.
func setupNotifications(a) {
    NotificationCenter.default.addObserver(self,
                                           selector: #selector(handleRouteChange),
                                           name: .AVAudioSessionRouteChange,
                                           object: AVAudioSession.sharedInstance())
}
 
func handleRouteChange(_ notification: Notification){}Copy the code
  • Published notifications contain a populated userInfo dictionary that provides details of route changes. You can retrieve from the userInfo dictionary AVAudioSessionRouteChangeReason values to determine the cause of the change. When connecting a new device, the reason is AVAudioSessionRouteChangeReasonNewDeviceAvailable, when a device is deleted, The reason is avaudiosessionroutechangereasonolddeviceavailable.
func handleRouteChange(_ notification: Notification) {
    guard let userInfo = notification.userInfo,
        let reasonValue = userInfo[AVAudioSessionRouteChangeReasonKey] as? UInt.let reason = AVAudioSessionRouteChangeReason(rawValue:reasonValue) else {
            return
    }
    switch reason {
    case .newDeviceAvailable:
        // Handle new device available.
    case .oldDeviceUnavailable:
        // Handle old device removed.
    default: ()}}Copy the code
  • When a new device becomes available, you can query the currentRoute property of the audio session to determine where the audio output is currently routed. This returns a AVAudioSessionRouteDescription object, the object is a list of all the input and output audio session. When equipment is removed, you can retrieve from the userInfo dictionary on a routing AVAudioSessionRouteDescription object. In both cases, you can query routing describe the output of the attribute, this property returns a AVAudioSessionPortDescription array of objects, and provides the audio output routing details.
func handleRouteChange(notification: NSNotification) {
    guard let userInfo = notification.userInfo,
        let reasonValue = userInfo[AVAudioSessionRouteChangeReasonKey] as? UInt.let reason = AVAudioSessionRouteChangeReason(rawValue:reasonValue) else {
            return
    }
    switch reason {
    case .newDeviceAvailable:
        let session = AVAudioSession.sharedInstance()
        for output in session.currentRoute.outputs where output.portType == AVAudioSessionPortHeadphones {
            headphonesConnected = true
        }
    case .oldDeviceUnavailable:
        if let previousRoute =
            userInfo[AVAudioSessionRouteChangePreviousRouteKey] as? AVAudioSessionRouteDescription {
            for output in previousRoute.outputs where output.portType == AVAudioSessionPortHeadphones {
                headphonesConnected = false}}default: ()}}Copy the code

Important note: media play application should pause playback, if the cause of the route change is AVAudioSessionRouteChangeReasonOldDeviceUnavailable, but should not be, If the cause is AVAudioSessionRouteChangeReasonOverride.

Note: Changes in audio routing may also result in changes in the sampling rate of the audio session, I/O buffer duration, channel count, or other hardware-related values. If these values are important to your application, query them after the route changes to see if their values have changed.

6. Configure the hardware

Using the audio session property, you can optimize the audio behavior of your application at run time for device hardware. Doing so allows your code to adapt to the characteristics of the device on which it is running, as well as to changes the user makes while the application is running (such as plugging in a headset or docking a device on it).

Use the AVAudioSession:

  • Specify preferred hardware Settings for example rate and I/O buffer duration
  • Queries many hardware features, such as input and output latency, input and output channel counts, hardware sampling rates, hardware volume Settings, and audio input availability

6.1 Selecting the preferred Audio hardware Value

Use an audio session to specify your preferred device Settings, such as sampling rate and hardware I/O buffer duration. Table 5-1 describes the benefits and costs of these preferences.

Note: The default audio I/O buffer time (approximately 0.02 seconds for 44.1khz audio) provides adequate responsiveness for most applications. You can set a low I/O duration for delay-critical applications, such as Live Musical Instrument Monitoring, but for most applications, you never need to change this setting.

6.2 Setting the preferred Audio hardware Value

Set the preferred hardware value before activating the audio session. If you are already running an audio session, disable it. The change to the preferred value takes effect after the audio session is activated, and you can verify the change at this point. Listing 5-1 shows how to set the preferred hardware values and validate them.

  • Listing 5-1 sets and validates the audio hardware values
let session = AVAudioSession.sharedInstance()
 
// Configure category and mode
do {
    try session.setCategory(AVAudioSessionCategoryRecord, mode: AVAudioSessionModeDefault)}catch let error as NSError {
    print("Unable to set category:  \(error.localizedDescription)")}// Set preferred sample rate
do {
    try session.setPreferredSampleRate(44 _100)}catch let error as NSError {
    print("Unable to set preferred sample rate:  \(error.localizedDescription)")}// Set preferred I/O buffer duration
do {
    try session.setPreferredIOBufferDuration(0.005)}catch let error as NSError {
    print("Unable to set preferred I/O buffer duration:  \(error.localizedDescription)")}// Activate the audio session
do {
    try session.setActive(true)}catch let error as NSError {
    print("Unable to activate session. \(error.localizedDescription)")}// Query the audio session's ioBufferDuration and sampleRate properties
// to determine if the preferred values were set
print("Audio Session ioBufferDuration: \(session.ioBufferDuration), sampleRate: \(session.sampleRate)")
Copy the code

Note: When competing audio sessions set preferred hardware values, the system gives preference to non-mixed sessions. Use the AVAudioSessionCategoryAmbient category or AVAudioSessionCategoryOptionMixWithOthers option audio session is unlikely to provide its preferred hardware setup.

6.3 Selecting and Configuring the Microphone

On devices with two or more built-in microphones, iOS automatically selects the microphone in audio session mode. Mode specifies the digital signal processing (DSP) and possible routing for input. Optimize input and routing for use cases of each pattern. Setting patterns can also affect other aspects of the route being used.

Developers can also manually select the microphone and even the preferred microphone polarity mode if the hardware supports it.

Important: Before using any of the input selection features, set the audio session category and mode for your application, and then activate the audio session.

  • Set preferred input

To discover built-in or connected input ports, use the availableInputs attribute of the audio session. This property returns AVAudioSessionPortDescription array of objects, these objects description available input port of the equipment. Ports can be identified by their portType attribute. To set the preferred input port (built-in microphone, wired microphone, USB input, and so on), use the setPreferredInput:error: method for an audio session.

  • Setting the preferred data Source
  1. Some ports, such as built-in microphones and some USB accessories, support data sources. Applications can discover available dataSources by querying the dataSources property of the port description. For built-in microphones, the returned data source description object represents each microphone. Different devices return different values for the built-in microphone. For example, the iPhone 4 and iPhone 4S have two microphones: bottom and top. The iPhone 5 has three microphones: bottom, front and back.
  2. A single built-in microphone can be identified by a combination of position properties (up and down) and direction properties (front and back, etc.) described by the data source. An application can use AVAudioSessionPortDescription object setPreferredDataSource: error: method to set the preferred data source.
  • Sets a preferred polar mode

Some iOS devices support setting microphone polarity mode for some built-in microphones. The pole pattern of the microphone determines that its sensitivity to sound depends on the direction of the source. The current iPhone supports the preferred polar mode for the built-in front and rear microphones. Available patterns are returned using the supportedPolarPatterns property of the data source description object. This property returns an array of supported polarity modes (such as Cardioid or OmniDirectional) for the data source, or nil if no alternative mode is available. There are many supported polar mode if the data source, you can through the use of data source description setPreferredPolarPattern: error: method to set the preferred model of polarity.

  • The following code provides a simple example of how to select a specific microphone and set its polarity mode.
// Preferred Mic = Front, Preferred Polar Pattern = Cardioid
let preferredMicOrientation = AVAudioSessionOrientationFront
let preferredPolarPattern = AVAudioSessionPolarPatternCardioid
 
// Retrieve your configured and activated audio session
let session = AVAudioSession.sharedInstance()
 
// Get available inputs
guard let inputs = session.availableInputs else { return }
 
// Find built-in mic
guard let builtInMic = inputs.first(where: {$0.portType == AVAudioSessionPortBuiltInMic
}) else { return }
 
// Find the data source at the specified orientation
guard letdataSource = builtInMic.dataSources? .first (where: {$0.orientation == preferredMicOrientation
}) else { return }
 
// Set data source's polar pattern
do {
    try dataSource.setPreferredPolarPattern(preferredPolarPattern)
} catch let error as NSError {
    print("Unable to preferred polar pattern: \(error.localizedDescription)")}// Set the data source as the input's preferred data source
do {
    try builtInMic.setPreferredDataSource(dataSource)
} catch let error as NSError {
    print("Unable to preferred dataSource: \(error.localizedDescription)")}// Set the built-in mic as the preferred input
// This call will be a no-op if already selected
do {
    try session.setPreferredInput(builtInMic)
} catch let error as NSError {
    print("Unable to preferred input: \(error.localizedDescription)")}// Print Active Configuration
session.currentRoute.inputs.forEach { portDesc in
    print("Port: \(portDesc.portType)")
    if let ds = portDesc.selectedDataSource {
        print("Name: \(ds.dataSourceName)")
        print("Polar Pattern: \(ds.selectedPolarPattern ?? "[none] ")")}}Copy the code

Running this code on an iPhone 6S produces the following console output:

Port: MicrophoneBuiltIn
Name: Front
Polar Pattern: Cardioid

Copy the code

6.4 Running applications in the emulator

When you add audio session support to your application, you can run your application on an emulator or device. However, the emulator cannot simulate most interactions between audio sessions in different processes or audio routing changes. When running your application in the emulator, you cannot:

  • Calling an interrupt
  • Analog plug in or unplug headphones
  • Change the mute switch Settings
  • Analog screen lock
  • Test audio mixing behavior — Audio is played together with audio from another application, such as a music application.

Because of the nature of the emulator, you may want to condition your code to allow partial testing in the emulator. The following code shows how to do this.

#if arch(i386) || arch(x86_64)
    // Execute subset of code that works in the Simulator
#else
    // Execute device-only code as well as the other code
#endif
Copy the code

7. Protect user privacy

To protect user privacy, your application must obtain user permission before recording audio. If the user does not grant permission, only silence is logged. When you use a category that supports records and the application tries to use an input path, the system will automatically prompt the user for permission.

Instead of waiting for the user to be prompted to record permissions, you can use the requestRecordPermission: method to manually request permissions. Using this approach allows your application to be licensed without interrupting the natural flow of your application, resulting in a better user experience.

AVAudioSession.sharedInstance().requestRecordPermission { granted in
    if granted {
        // User granted access. Present recording interface.
    } else {
        // Present message to user indicating that recording
        // can't be performed until they change their preference
        // under Settings -> Privacy -> Microphone}}Copy the code

Important: Starting with iOS 10, all apps that access the device’s microphone must statically declare their intent. To do this, the application must now contains NSMicrophoneUsageDescription key in their information. And provide a destination string for this key. This string is displayed as part of the alert when the system prompts the user to allow access. If an application attempts to access the device’s microphone without this key and value present, the application will terminate.

8. AVAudioSession

AVAudioSession is an object that communicates to the system how you intend to use audio in your application. Its class declaration is as follows:

class AVAudioSession : NSObject
Copy the code

The audio session acts as an intermediary between the application and the operating system and, in turn, between the underlying audio hardware. You use an audio session to communicate the general nature of application audio to the operating system without specifying specific behavior or the interaction required with the audio hardware. You delegate the management of these details to the audio session to ensure that the operating system can best manage the user’s audio experience.

All iOS, tvOS, and watchOS apps have a default audio session, preconfigured with the following behavior:

  • It supports audio playback, but does not allow audio recording (tvOS does not support audio recording).
  • In iOS, setting the ringer/Mute switch to silent mode will mute any audio played by the app.
  • On iOS, locking the device silences the app’s audio.
  • When the app plays the audio, it will mute any other background audio.

While the default audio session provides useful behavior, it generally does not provide the audio behavior required by media applications. To change the default behavior, you need to configure the audio session category for your application.

You can use seven possible categories (see Audio session categories and modes), but playback is the most common one for playback applications. This category indicates that audio playback is a core feature of your application. When you specify this category, your app’s audio will continue with the ringtone/Mute switch set to silent mode (iOS only). Using this category, you can also play background audio if you use audio, AirPlay and pictures in Picture background mode. For more information, see Enabling Background Audio.

Use the AVAudioSession object to configure the application’s audio session. This class is a singleton object used to set the category, mode, and other configurations of the audio session. You can interact with audio sessions throughout the life cycle of your application, but it is often useful to perform this configuration when the application is started, as shown in the following example.

func application(_ application: UIApplication,
                 didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
    
    // Get the singleton instance.
    let audioSession = AVAudioSession.sharedInstance()
    do {
        // Set the audio session category, mode, and options.
        try audioSession.setCategory(.playback, mode: .moviePlayback, options: [])
    } catch {
        print("Failed to set audio session category.")}// Other post-launch configuration.
    return true
}
Copy the code

This configuration is used for audio sessions when you activate sessions using the setActive(:) or setActive(:options:) methods.

Note that you can activate the audio session at any time after setting the category, but it is usually best to defer this call until the application has started audio playback. Deferred calls ensure that you don’t prematurely interrupt any other background audio in progress.

  • AVAudioSession mainly provides the following interfaces

8.1 Obtaining an Audio Session

class func sharedInstance() - >AVAudioSession
//Returns the shared audio session instance.
Copy the code

8.2 Configuring An Audio Session


var category: AVAudioSession.Category
//The current audio session category.
func setCategory(AVAudioSession.Category)
//Sets the audio session’s category.
var availableCategories: [AVAudioSession.Category]
//The audio session categories available on the current device.
struct AVAudioSession.Category
//Audio session category identifiers.
var categoryOptions: AVAudioSession.CategoryOptions
//The set of options associated with the current audio session category.
func setCategory(AVAudioSession.Category.options: AVAudioSession.CategoryOptions) / /Sets the audio session's category with the specified options.
struct AVAudioSession.CategoryOptions
//Constants that specify optional audio behaviors.
var mode: AVAudioSession.Mode
//The current audio session's mode.
func setMode(AVAudioSession.Mode) / /Sets the audio session's mode.
func setCategory(AVAudioSession.Category.mode: AVAudioSession.Mode.options: AVAudioSession.CategoryOptions) / /Sets the audio session's category.mode.and options.
var availableModes: [AVAudioSession.Mode] / /The audio session modes available on the device.
struct AVAudioSession.Mode
//Audio session mode identifiers.
var routeSharingPolicy: AVAudioSession.RouteSharingPolicy
//The current route-sharing policy.
func setCategory(AVAudioSession.Category.mode: AVAudioSession.Mode.policy: AVAudioSession.RouteSharingPolicy.options: AVAudioSession.CategoryOptions) / /Sets the session category.mode.route-sharing policy.and options.
enum AVAudioSession.RouteSharingPolicy
//Cases that indicate the possible route-sharing policies for an audio session.
Copy the code

8.3 Activating an Audio Session

// Activate or disable the application's audio session with the specified options.
func setActive(Bool, options: AVAudioSession.SetActiveOptions)

// Activate the audio session asynchronously on watchOS.
func activate(options: AVAudioSessionActivationOptions, completionHandler: (Bool, Error?) -> Void)

Copy the code

8.4 Requesting the Recording Right

// Current recording license status.
var recordPermission: AVAudioSession.RecordPermission

// Request user permission to record audio.
func requestRecordPermission(PermissionBlock)

Copy the code

8.5 Mix with other audio


var isOtherAudioPlaying: Bool
//A Boolean value that indicates whether another app is playing audio.
var secondaryAudioShouldBeSilencedHint: Bool
//A Boolean value that indicates whether another app, with a nonmixable audio session, is playing audio.
var allowHapticsAndSystemSoundsDuringRecording: Bool
//A Boolean value that indicates whether system sounds and haptics play while recording from audio input.
func setAllowHapticsAndSystemSoundsDuringRecording(Bool)
//Sets a Boolean value that indicates whether system sounds and haptics play while recording from audio input.
var promptStyle: AVAudioSession.PromptStyle
//A hint to audio sessions that use voice prompt mode to alter the type of prompts they issue in response to other system audio, such as Siri and phone calls.
enum AVAudioSession.PromptStyle
//Constants that indicate the prompt style to use.
Copy the code

8.6 Responding to Audio Session Notifications

class let interruptionNotification: NSNotification.Name
//A notification that's posted when an audio interruption occurs.
class let routeChangeNotification: NSNotification.Name
//A notification that's posted when the system's audio route changes.
class let silenceSecondaryAudioHintNotification: NSNotification.Name
//A notification that's posted when the primary audio from other applications starts and stops.
class let mediaServicesWereLostNotification: NSNotification.Name
//A notification that's posted when the system terminates the media server.
class let mediaServicesWereResetNotification: NSNotification.Name
//A notification that's posted when the media server restarts.
Copy the code

8.7 Using Audio Routes

// A description of the input and output ports of the current audio route.
var currentRoute: AVAudioSessionRouteDescription

// An object that describes the input and output ports associated with the audio route of the session.
class AVAudioSessionRouteDescription// A Boolean value indicating whether the audio input path is available.var isInputAvailable: Bool// An array of input ports that can be used for audio routing.var availableInputs: [AVAudioSessionPortDescription]? // Preferred input port for audio routing.var preferredInput: AVAudioSessionPortDescription? // Sets the preferred input port for the audio route.func setPreferredInput(AVAudioSessionPortDescription?). // Information about port functionality and the hardware channels it supports.class AVAudioSessionPortDescription// The currently selected input data source.var inputDataSource: AVAudioSessionDataSourceDescription? // Array of available data sources for the current input port of the audio session.var inputDataSources: [AVAudioSessionDataSourceDescription]? // Select the data source for the current input port of the audio session.func setInputDataSource(AVAudioSessionDataSourceDescription?). // An array of available output data sources for the current audio route.var outputDataSources: [AVAudioSessionDataSourceDescription]? // The currently selected output data source.var outputDataSource: AVAudioSessionDataSourceDescription? // Sets the output data source for the audio session.func setOutputDataSource(AVAudioSessionDataSourceDescription?). // An object that defines a data source for audio input or output, providing information such as the name, location, and direction of the data source.class AVAudioSessionDataSourceDescription// Temporarily change the current audio route.func overrideOutputAudioPort(AVAudioSession.PortOverride)

Copy the code

8.8 Using the Audio Channel

// The number of currently routed audio input channels.
var inputNumberOfChannels: Int

// The maximum number of input channels available for the current audio route.
var maximumInputNumberOfChannels: Int

// Number of preferred input channels of the current route.
var preferredInputNumberOfChannels: Int

// Set the number of preferred channels for the current route.
func setPreferredInputNumberOfChannels(Int)

// Number of audio output channels.
var outputNumberOfChannels: Int

// The maximum number of output channels available for the current audio route.
var maximumOutputNumberOfChannels: Int

// Priority number of output channels of the current route.
var preferredOutputNumberOfChannels: Int

// Sets the preferred number of output channels for the current route.
func setPreferredOutputNumberOfChannels(Int)

Copy the code

8.9 Handling Audio Device Settings



// The gain applied to the session-dependent input.
var inputGain: Float

// A Boolean value indicating whether the input gain can be set.
var isInputGainSettable: Bool

// Change the input gain to the specified value.
func setInputGain(Float)

// System-wide output volume set by the user.
var outputVolume: Float

// Current audio sampling rate, in Hertz.
var sampleRate: Double

// Preferred sampling rate, in Hertz.
var preferredSampleRate: Double

// Sets the preferred sampling rates for audio input and output.
func setPreferredSampleRate(Double)

// Audio input delay, in seconds.
var inputLatency: TimeInterval

// Audio output delay, in seconds.
var outputLatency: TimeInterval

// Current I/O buffer duration in seconds.
var ioBufferDuration: TimeInterval

// Preferred I/O buffer duration in seconds.
var preferredIOBufferDuration: TimeInterval

// Sets the preferred audio I/O buffer duration.
func setPreferredIOBufferDuration(TimeInterval)
Copy the code

8.10 Setting I/O Preferences for Aggregation

// Sets aggregate I/O configuration preferences for audio sessions.
func setAggregatedIOPreference(AVAudioSession.IOType)

Copy the code

See the Audio Sessions section of the Apple documentation