1. The AVAudioSession overview

I have been working on the iOS client development of IPC Camera for the last year. AVAudioSession is one of the most important ways to deal with audio. Take a look at apple’s official image:

It can be seen that AVAudioSession is used to manage the resource usage of audio hardware devices (microphone, speaker) by multiple apps.

For example, AVAudioSession can do these things

• Set whether your APP and other APP audio exist at the same time, or interrupt other APP audio
• When the mobile phone is set to silent mode, whether the audio of your APP can play sound
• Phone or other APP interrupts its APP’s audio event handling
• A device that specifies audio input and output (for example, whether a headset or a speaker outputs sound)
• Whether to support recording and audio playback at the same time

2. AVAudioSession Category

The interface to AVAudioSession is relatively simple. AVAudioSession will be automatically activated when the APP starts. Of course, we can manually activate the code as follows.

// Import the header file#import <AVFoundation/AVFoundation.h>//AVAudioSession is a singleton class AVAudioSession *session = [AVAudioSession sharedInstance]; / / AVAudioSessionCategorySoloAmbient is the category of the system's default [sessionsetCategory:AVAudioSessionCategorySoloAmbient error:nil]; // Activate AVAudioSession [session]setActive:YES error:nil];
Copy the code

You can see that there are two parameters in setting session: category and Options category. There are currently seven categories in iOS, and each category supports the following four capabilities

• Interrupts non-mixable apps Audio: Whether to interrupt an APP that does not support mixing playback
• Silenced by the Silent switch: Whether the phone will respond to the mute button switch
• Supports Audio Input: Supports audio recording
• Supports Audio Output: Supports audio playback

Let’s use the chart to visually look at the specific capability sets of each category

• AVAudioSessionCategoryAmbient, only support audio playback. In this Category, the audio is silenced by the Mute button and the lock button. And it doesn’t interrupt other apps’ audio playback.

• AVAudioSessionCategorySoloAmbient, this is the default Category, system only support audio playback. The audio is muted by the Mute button and the lock button. Unlike AVAudioSessionCategoryAmbient, this will interrupt other audio playback applications

• AVAudioSessionCategoryPlayback, only support audio playback. Your audio will not be muted by the Mute and lock buttons. It is applicable to apps where audio is the main function, such as netease Cloud music apps, which can still play after the screen is locked.

Note that you must enable background audio support in the application when you select to support audio Category even when the Mute button is switched to mute and the lock button is switched to lock. See UIBackgroundModes for details.

• AVAudioSessionCategoryRecord, only support audio recording. Playback is not supported.
• AVAudioSessionCategoryPlayAndRecord to support audio playback and recording. Audio input and output do not need to be synchronized, but can be synchronized. If you want an audio calling application, you can use this Category.
• AVAudioSessionCategoryAudioProcessing, support local audio codec processing only. It does not support playback or recording.
• AVAudioSessionCategoryMultiRoute to support audio playback and recording. Allows simultaneous input and output of multiple audio streams. (For example, special requirements such as USB connection to external speaker output audio and Bluetooth headset playing another audio at the same time)

We can also read the categories supported by the current device through the AVAudioSession property

@property(readonly) NSArray<NSString *> *availableCategories;
Copy the code

This ensures device compatibility.

The following is an example of the code for setting a Category

NSError *setCategoryError = nil;
BOOL isSuccess = [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:&setCategoryError];
if(! Success) {// This can be readsetCategoryError. LocalizedDescription to see the error reason}Copy the code

3. AVAudioSession Mode&&Options

The Category we just introduced defines seven main scenarios, and it sometimes needs to be tweaked slightly in actual development requirements. We found that this interface also has two parameters: Mode and Options.

/ *set session category and mode with options */
- (BOOL)setCategory:(NSString *)category mode:(NSString *)mode options:(AVAudioSessionCategoryOptions)options error:(NSError * *) outError API_AVAILABLE (ios (10.0), watchos (3.0), tvos (10.0));Copy the code

AVAudioSession Mode

We get the Mode supported by the current device by reading the following property

@property(readonly) NSArray<NSString *> *availableModes;
Copy the code

There are seven modes in iOS to customize our Category behavior

Let’s look at each Mode one by one

• AVAudioSessionModeDefault, the default mode, compatible with all the Category

• AVAudioSessionModeVoiceChat, suitable for VoIP type of application. Can only be AVAudioSessionCategoryPlayAndRecord Category. In this mode the system will automatically configure AVAudioSessionCategoryOptionAllowBluetooth this option. The system will automatically select the best combination of built-in microphones to support voice chat.

• AVAudioSessionModeVideoChat for video chat application type, can only be AVAudioSessionCategoryPlayAndRecord Category. Ed in this mode the system will automatically configure AVAudioSessionCategoryOptionAllowBluetooth and AVAudioSessionCategoryOptionDefaultToSpeaker options. The system will automatically select the best combination of built-in microphones to support video chat.

• AVAudioSessionModeGameChat, suitable for gaming applications. The application of using GKVoiceChat object will automatically set this mode and AVAudioSessionCategoryPlayAndRecord Category. Parameters and AVAudioSessionModeVideoChat

• AVAudioSessionModeVideoRecording, applicable to use video camera acquisition application. Can only be AVAudioSessionCategoryPlayAndRecord and AVAudioSessionCategoryRecord under the two Category. This pattern is used in conjunction with the AVCaptureSession API to better control the audio and video input and output paths. (for example, set automaticallyConfiguresApplicationAudioSession properties, the system will automatically choose the best output path.

• AVAudioSessionModeMeasurement, minimize system. Only used for AVAudioSessionCategoryPlayAndRecord, AVAudioSessionCategoryRecord, AVAudioSessionCategoryPlayback these Category.

• The application of AVAudioSessionModeMoviePlayback, suitable for broadcast video. Only for AVAudioSessionCategoryPlayback this Category.

AVAudioSession Options

We can also use options to fine-tune the Category behavior, as shown in the following table

Tune our Category

We can tune our effects by using categories and appropriate modes and Options. Here are two application scenarios:

Those of you who have used Gaudammap know that when playing QQ music in the background, if the navigation voice comes out, QQ music will not stop, but will be intelligently lowered and mixed. After the navigation voice is broadcast, QQ music will play normally. Here we need to play music in the background. So use AVAudioSessionCategoryPlayback Category, need to mix and intelligent push other APP volume, So the Options chosen AVAudioSessionCategoryOptionMixWithOthers and AVAudioSessionCategoryOptionDuckOthers

An example of this code is as follows

 BOOL isSuccess = [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryOptionMixWithOthers | AVAudioSessionCategoryOptionDuckOthers error:&setCategoryError];
Copy the code

Or I hope AVAudioSessionCategoryPlayAndRecord the default audio Category, which is shown by a speaker so you can call this interface to adjust the Category

- (BOOL)setCategory:(NSString *)category withOptions:(AVAudioSessionCategoryOptions)options error:(NSError **)outError
Copy the code

By selecting appropriate and Category, mode, and options, you can tune the audio input and output to meet your daily development needs. That is, a Category supports certain modes and options, as you can see from the table above.

4. Audio interrupt processing

Other apps or phone calls will interrupt our APP audio, so we have to deal with it accordingly. We can get audio interrupt by listening AVAudioSessionInterruptionNotification this key event

The callback returns that Userinfo has key values

• AVAudioSessionInterruptionTypeKey: Values AVAudioSessionInterruptionTypeBegan said interrupts start values AVAudioSessionInterruptionTypeEnded said over

Interrupt start: What we need to do is to save the playback state, context, update the user interface, etc. Interrupt end: What we need to do is to restore the state and context, update the user interface, and choose whether to activate our session after we are ready according to the requirements.

Select different audio playback technology, processing interrupt mode is also different, specific as follows:

• System Sound Services: When you use System Sound Services to broadcast audio, the System will automatically process the audio playback, which is not controlled by the APP. When the interruption occurs, the audio playback will be muted. When the interruption ends, the audio playback will be resumed.

• AV Foundation Framework: The AVAudioPlayer and AVAudioRecorder classes provide Delegate callback methods to start and end interrupts. When the interruption occurs, the system will automatically stop playing. What you need to do is to record the status such as the playing time and update the user interface. After the interruption ends, you can call the playing method again and the system will automatically activate the session.

• Audio Queue Services, I/O Audio Unit: Using aduio unit, you need to handle interrupts. What you need to do is to record the position of playback or recording, and restore the Audio session after the interruption.

• OpenAL: When using OpenAL to play, you also need to listen for interrupts. Manage OpenAL context and restore audio Session after user interrupt.

It should be noted that: 1. There is an interrupt start event, but not necessarily an interrupt end event. Therefore, it is necessary to save the playback state and Audio Session management when the user enters the foreground and clicks the UI operation, so as not to affect the Audio function of the APP. 2. In the competition of audio resources, telephone must be the priority. 3. AVAudioSession can also monitor the audio status of peripherals, such as earphone unplugging. No elaboration here

5. AVAudioSession summary

The role of AVAudioSession is to manage the allocation of audio, the only hardware resource, and adapt to the functional requirements of our APP for audio by tuning the appropriate AVAudioSession. When switching audio scenes, the corresponding switching AVAudioSession is required.

Reference: Audio Session Programming Guide