1. Introduction

IOS provides audio processing plug-ins that support mixing, balancing, format conversion, and real-time INPUT/output for recording, playback, offline rendering, and real-time conversations, such as Voice over Internet Protocol (VoIP). You can load and use (that is, host) these powerful and flexible plug-ins, audio Units, on the fly from your iOS application.

Audio units typically work within the context of a closed object called the Audio Processing graph, as shown in the figure. In this example, your app sends the audio to the first audio unit in the diagram through one or more callbacks, and controls each audio unit individually. The output of the I/O unit (the last audio unit in this or any audio processing diagram) is directly connected to the output hardware.

Because audio units are at the bottom of the iOS audio programming stack, you need to understand them more deeply than other iOS audio technologies.

Unless you need real-time playback of synthetic sound, low latency I/O(input and output), or specific audio unit features, Otherwise, the first thing to consider is Media Player, AV Foundation, OpenAL, or Audio Toolbox frameworks. These higher-level techniques can replace your use of audio units and provide important additional functionality, as described in the Multimedia Programming Guide

Audio Units Provide Fast, Modular Audio Processing

The two biggest advantages of using audio units directly are:

Excellent responsiveness. Because you have access to the real-time priority thread in the audio unit render callback function, your audio code is as close to metal as possible. Synthetic instruments and real-time synchronous voice I/O benefit most from using audio units directly.
Dynamic reconfiguration. The Audio processing graphics API is built around the AUGraph opaque type, which allows you to dynamically assemble, reconfigure, and rearrange complex audio processing chains in a thread-safe manner while working with audio. This is the only audio API in iOS that does this.

The life cycle of an audio unit is as follows:

At run time, get a reference to a dynamically linkable library that defines the audio unit you want to use
Instantiate the audio unit
Configure the audio units according to the type required to meet your needs
Initialize the audio unit so that it is ready to process audio
Start audio stream
Control audio unit
When finished, release the audio unit

The audio unit provides very useful personal functions such as stereo panning, mixing, volume control and audio level measurement. Hosted audio units allow you to add these features to your applications. However, in order to reap these benefits, you must understand some basic concepts, including audio data stream formats, render callback functions, and audio unit architectures.

Audio Unit Hosting Fundamentals

Choosing a Design Pattern and Constructing Your App

The Audio Unit hosted design pattern provides a flexible blueprint for customizing application details. Each pattern represents:

How to configure I/O units. An I/O unit has two separate elements, one that receives audio from the input hardware and one that sends audio to the output hardware. Each design pattern indicates which elements or elements should be enabled.
Where, in the audio processing diagram, the audio data stream format must be specified. You must correctly specify the format to support audio streams.
Where to make audio unit connections and where to add render callback functions. An audio unit connection is a formal construct that propagates a stream format from the output of one audio unit to the input of another. Callbacks allow you to input audio into the Graph or manipulate audio at a single example level within the Graph

No matter which design pattern you choose, the steps for building an audio unit hosting application are basically the same:

Configure application audio sessions to ensure that applications work correctly in the system and device hardware environment
Construct an Audio processing graph. This multi-step process leverages everything you’ve learned in audio unit hosting basics
Provides a user interface to control graphic audio units

Familiarize yourself with these steps so that you can apply them to your own projects.

Constructing Audio Unit Apps

Get the Most Out of Each Audio Unit

Most of this article will tell you that all iOS audio units have important attributes in common. For example, your application needs to specify and load the audio unit at run time, and then correctly specify its audio stream format.

At the same time, each audio unit has certain unique functions and requirements, from the correct type of audio sample data to be used to the configuration required for correct behavior. Understand the usage details and specific functions of each audio unit so that you know, for example, when to use a 3D Mixer unit and when to use a Multichannel Mixer.

Related chapters: Using Specific Audio Units

How to Use This Document

If you want to have a solid conceptual foundation before starting your project, read the Audio Unit Hosting Fundamentals. This chapter explains the concepts behind apis. Read on for more information on Constructing Audio Unit Apps to learn how to choose design patterns and build an application workflow for your project.

If you have some experience with Audio Units and just want Specific types of detail, you can start with Using Specific Audio Units

The basic reference documents include the following:

Audio Unit Properties Reference describes the Properties that can be used to configure each Audio Unit type
Audio Unit Parameters Reference describes the Parameters that can be used to control each Audio Unit type.
The Audio Unit Component Services Reference describes the APIS used to access the Audio Unit parameters and properties and describes the various Audio Unit callback functions.
The Audio Component Services Reference describes apis for accessing and managing Audio unit instances at run time.
Audio Unit Processing Graph Services Reference describes apis for constructing and manipulating Audio Processing graphs, which are dynamically reconfigurable Audio Processing chains.
The Core Audio Data Types Reference describes the Data structures and Types required to host the Audio unit.

2. Audio Unit Hosting Foundation

All of the audio technology in iOS is built on top of the audio unit, as shown here. The advanced technologies shown here — Media Player, AV Foundation, OpenAL, and Audio Toolbox — are packaged Audio units that provide dedicated apis for specific tasks

Only when you need the highest degree of control, performance, or flexibility in a project, or when you need a specific feature (such as echo cancellation) that can only be obtained by using the audio unit directly, use another API

Audio Units Provide Fast, Modular Audio Processing

Use the audio unit directly, rather than through a higher-level API, when you need one of the following:

Low-latency synchronous audio I/O(input and output), such as Voice over Internet Protocol (VoIP) applications
Responsive playback of synthetic sounds, such as music games or synthetic instruments
Use a specific audio unit feature, such as echo cancellation, mixing, or tone equalization
A processing chain architecture that lets you assemble audio processing modules into flexible networks. This is the only audio API in iOS that does this.

Audio Units in iOS

IOS provides seven audio units, divided into four categories by purpose

Purpose	Audio units
Effect	iPod Equalizer
Mixing	3D Mixer Multichannel Mixer
I/O	Remote I/O Voice-Processing I/O Generic Output
Format conversion	Format Converter

Note: The iOS Dynamic plugin architecture does not support third-party audio units. That is, the only audio unit available for dynamic loading is provided by the operating system.

1. Effect Unit

IOS 4 provides an effects unit, iPod Equalizer, the same equalizer that uses the built-in iPod app. To see the iPod app’s user interface for this audio unit, go to Settings > iPod > EQ. When using this audio unit, you must provide your own UI. This audio unit provides a preset set of balanced curves, such as bass boosters, pop, and verbal words.

2. Mixer Units

IOS provides two mixer units. The 3D Mixer unit is the basis on which OpenAL is built. In most cases, if you need the power of a 3D Mixer unit, your best bet is to use OpenAL, which provides a more advanced API that is perfect for gaming applications.

The multi-channel mixer unit provides any number of mono or stereo streams mixed with stereo output. You can turn each input on or off, set its input gain, and set its stereo pan position. For a demonstration of how to use this audio unit, see the Sample code project Audio Mixer (MixerHost).

3. I/O Units

IOS provides three I/O units. Remote I/O units are the most commonly used. It connects to input and output audio hardware, providing you with low latency access to individual incoming and outgoing audio sample values. It provides format conversion between hardware audio formats and application audio formats through an included format converter unit

Voice-processing I/O units extend remote I/O units by adding echo cancellation for VoIP or Voice chat applications. It also provides automatic gain correction, voice processing quality adjustment and mute.

The Generic Output unit does not connect to the audio hardware, but rather provides a mechanism for sending the Output of the processing chain to the application. You typically use Generic Output units for offline audio processing.

4. Format Converter Unit

IOS 4 provides a format converter unit, which is usually used indirectly through I/O units.

Use the Two Audio Unit APIs in Concert

IOS has an API for processing audio Units directly and another API for processing Audio Processing Graphs. When you use audio units in your application, you use both apis.

To use audio Units directly (configure and control them), useAudio Unit Component Services ReferenceThe methods described in
To create and configure the Audio Processing Graph, use theAudio Unit Processing Graph Services ReferenceThe methods described in

There is some overlap between the two apis, and you are free to mix and match according to your programming style. The Audio Unit API and the Audio Processing Graph API provide the following capabilities, respectively:

Gets a reference to a dynamic link library that defines an audio unit
Instantiate the audio unit
Connect audio unit and attach render callback function
Start and stop audio streams

This document provides code examples that use both apis, but focuses on the Audio Processing Graph API. If you need to choose between two apis in your code, use the Graph API unless you have a specific reason not to. Your code will be more compact, easier to read, and easier to support dynamic reconfiguration (see Audio Processing Graphs Provide Thread Safety)

Use Identifiers to Specify and Obtain Audio Units

To find the audio unit at run time, first specify its Type, subtype, and Manufacturer Keys in the audio component description data structure. Do this using either the Audio Unit or the Audio Processing Graph API

Listing 1-1 Creating an audio component description to identify an audio unit

AudioComponentDescription ioUnitDescription;
 
ioUnitDescription.componentType          = kAudioUnitType_Output;
ioUnitDescription.componentSubType       = kAudioUnitSubType_RemoteIO;
ioUnitDescription.componentManufacturer  = kAudioUnitManufacturer_Apple;
ioUnitDescription.componentFlags         = 0;
ioUnitDescription.componentFlagsMask     = 0;
Copy the code

This description specifies exactly one audio unit — the Remote I/O Unit. This key and other Audio unit Keys in iOS are listed in the Identifier Keys for Audio Units. Note that all iOS audio units use kAudioUnitManufacturer_Apple in the componentManufacturer field.

To create wildcard descriptions, set one or more type/subTYPE fields to 0. For example, to match all I/O units, set componentSubType = 0

Once you have a description, you can use either API to get a library reference for a given audio unit (or set of audio units). The audio unit API is shown below

Listing 1-2 Obtaining an audio unit instance using the audio unit API

AudioComponent foundIoUnitReference = AudioComponentFindNext (
                                          NULL,
                                          &ioUnitDescription
                                      );
AudioUnit ioUnitInstance;
AudioComponentInstanceNew (
    foundIoUnitReference,
    &ioUnitInstance
);
Copy the code

NULL is passed to the first argument of AudioComponentFindNext, telling the function to find the first system audio unit that matches the description using the system-defined order. If a previously found audio unit reference is passed in this parameter, the function locates the next audio unit that matches the description. For example, this usage allows you to get references to all I/O units by repeatedly calling AudioComponentFindNext
The second argument to the AudioComponentFindNext call refers to the audio unit description defined in Listing 1-1.
The result of the AudioComponentFindNext function is a reference to a dynamically linkable library that defines an audio unit. Pass reference to AudioComponentInstanceNew function to instantiate the audio unit, as shown in listing 1-2.

You can instantiate audio units using the Audio Processing Graph API. Listing 1-3 shows how to do this

Listing 1-3 Obtaining an audio unit instance using the audio processing graph API

// Declare and instantiate an audio processing graph
AUGraph processingGraph;
NewAUGraph (&processingGraph);
 
// Add an audio unit node to the graph, then instantiate the audio unit
AUNode ioNode;
AUGraphAddNode (
    processingGraph,
    &ioUnitDescription,
    &ioNode
);
AUGraphOpen (processingGraph); // indirectly performs audio unit instantiation
 
// Obtain a reference to the newly-instantiated I/O unit
AudioUnit ioUnit;
AUGraphNodeInfo (
    processingGraph,
    ioNode,
    NULL,
    &ioUnit
);
Copy the code

This code listing introduces AUNode, an opaque type that represents the audio unit in the audio Processing Graph context. In the output of the AUGraphNodeInfo function call, you receive a reference to the new audio unit instance in the ioUnit parameter

The second parameter to the AUGraphAddNode call refers to the audio unit description defined in Listing 1-1.

Once you have an audio unit instance, you can configure it. To do this, you need to understand the features of two audio units, scopes and Elements

Use Scopes and Elements to Specify Parts of Audio Units

Each part of an audio unit is organized into scopes and Elements, as shown in Figure 1-2. When calling a function to configure or control an audio unit, you can specify scopes and Elements to identify the specific goal of the function.

Figure 1-2 Audio unit scopes and elements

Scope is the programming context in the audio unit. Although the name of the Global Scope might imply otherwise, these contexts are never nested. Specify the target scope by using constants in the Audio Unit Scopes enumeration.

Element is a programming context nested within the Audio Unit scope. When an element is part of an input or output scope, it is similar to a signal bus in a physical audio device-hence sometimes called a bus. These two terms — element and bus — refer to exactly the same thing in audio unit programming. This document uses BUS when emphasizing signal flows and elements when emphasizing specific functions of audio Units, such as input and output elements for I/O Units (see Essential Characteristics of I/O Units)

Element (or bus) can be specified by a zero-indexed integer value. If you set a property or parameter that applies to the entire scope, specify an element value of 0.

Figure 1-2 illustrates a common architecture for an audio unit where the input and output elements are the same. However, different audio units use different architectures. For example, a mixer unit may have several input elements but only one output element. You can extend what you learn here about Scope and Elements to any audio unit, although there are some architectural changes

Global Scope, shown at the bottom of Figure 1-2, applies to the audio unit as a whole and is not associated with any particular audio stream. It has only one element, Element 0. Some properties, such as the maximum number of frames per slice (kAudioUnitProperty_MaximumFramesPerSlice), only apply to Global Scope.

The input and output scopes directly participate in moving one or more audio streams through the audio unit. As you might expect, audio enters from the input scope and leaves from the output scope. Properties or parameters can be applied to input or output scopes as a whole, such as the Element count property (kAudioUnitProperty_ElementCount). Other properties and parameters, such as the enable I/O property (kAudioOutputUnitProperty_EnableIO) or volume parameter (kMultiChannelMixerParam_Volume), apply to specific elements within the scope

Use Properties to Configure Audio Units

The audio unit property is a key-value pair that can be used to configure the audio unit. The key of the property is a unique integer with an associated mnemonic identifier, such as kAudioUnitProperty_MaximumFramesPerSlice = 14. Apple keeps a property key from 0 to 63999. In Mac OS X, third-party audio units use keys above this range

The value of each property is the specified data type and has the specified read/write access, as described in the Audio Unit Properties Reference. To set any properties on any audio unit, you can use a flexible function: AudioUnitSetProperty. Listing 1-4 shows a typical use of this function, with comments highlighting how to specify scope and Element, as well as keys and values indicating properties.

Listing 1-4 Using scope and element when setting a property

UInt32 busCount = 2;
 
OSStatus result = AudioUnitSetProperty (
    mixerUnit,
    kAudioUnitProperty_ElementCount,   // the property key
    kAudioUnitScope_Input,             // the scope to set the property on
    0.// the element to set the property on
    &busCount,                         // the property value
    sizeof (busCount)
);
Copy the code

The following are some of the attributes that you will often use in audio unit development. Familiarize yourself with them by reading its reference documentation:

KAudioOutputUnitProperty_EnableIO, used to enable or disable I/O input or output. By default, output is enabled and input is disabled.
KAudioUnitProperty_ElementCount, for example, is used to configure the number of input elements on a Mixer unit
KAudioUnitProperty_MaximumFramesPerSlice, in order to specify the maximum number of frames for audio data, the audio unit should be prepared to be generated in response to a render call. For most audio units, in most cases you must set this property as described in the reference documentation. If you don’t, your audio will stop when the screen locks
KAudioUnitProperty_StreamFormat, used to specify the audio stream data format for the input or output bus of a particular audio unit

Most attribute values can only be set when the audio unit is not initialized. The user does not intend to change these properties. However, some, such as the kAudioUnitProperty_PresentPreset property for the iPod EQ unit, and the kAUVoiceIOProperty_MuteOutput property for the Voice-processing I/O unit, It was intended to be changed while the audio was playing

To discover the availability of a property, access its value, and monitor changes to its value, use the following functions:

AudioUnitGetPropertyInfo to find out if a property is available If so, the data size of its value is given and whether the value can be changed
AudioUnitGetProperty, AudioUnitSetProperty, gets or sets the value of the property
AudioUnitAddPropertyListener, AudioUnitRemovePropertyListenerWithUserData, install or remove back DiaoHan monitoring of changes on the value of an attribute

Use Parameters and UIKit to Give Users Control

Audio unit parameters the user is configurable and the setting can be changed when the audio unit produces audio. In fact, the purpose of most parameters (such as volume or stereo pan position) is to adjust in real time what processing the audio unit is performing.

Like audio unit properties, audio unit parameters are key-value pairs. The key is defined by the audio unit to which it applies. It is always an enumerated constant, such as kMultiChannelMixerParam_Pan = 2, which is unique to the audio unit, but not globally unique.

Unlike attribute values, each parameter value has the same type: a 32-bit floating-point number. The permissible range of a value, as well as the units of measure it represents, is determined by the implementation of the parameter’s audio unit. These and other Parameters are in the Audio Unit Parameters Reference

To get or set parameter values, use one of the following functions, which are described in detail in the Audio Unit Component Services Reference:

AudioUnitGetProperty
AudioUnitSetProperty

To give the user control over an audio unit, give them access to its parameters through the user interface. Start by selecting an appropriate class from the UIKit framework to represent the parameters. For an ON /off feature, such as the kMultiChannelMixerParam_Enable parameter for a Multichannel Mixer unit, you can use a UISwitch object. For a constantly changing feature, such as the stereoscopic pan position provided by the kMultiChannelMixerParam_Pan parameter, you can use a UISlider object

Essential Characteristics of I/O Units

An I/O unit contains two elements, as shown in Figure 1-3.

Figure 1-3 The architecture of an I/O unit

Although these elements are part of an audio unit, your application treats them as separate entities. For example, you can use the EnableI /O property (kAudioOutputUnitProperty_EnableIO) to enable or disable each element independently, depending on the needs of your application.

Element 1 of the I/O unit connects directly to the audio input hardware on the device, represented by the microphone in the figure. This hardware connection (Element 1’s Input scope) is opaque to you. The first access to the audio data input from the input hardware is within Element 1’s Output scope.

Similarly, Element 0 of the I/O unit connects directly to the audio output hardware on the device, as shown in Figure 1-3, represented by a speaker. You can pass audio to Element 0’s Input scope, but its Output scope is opaque.

When dealing with audio units, you’ll often hear the two elements of an I/O unit described not by numbers but by names:

Input element is Element 1 (the letter I in Input looks like the number 1)
Output element is element 0 (the letter O of the word Output looks like the number 0)

As shown in Figure 1-3, each element itself has an Input scope and an Output scope. For this reason, describing these parts of an I/O unit can be a bit confusing. For example, you could say that in a synchronous I/O application, you receive audio from the Output scope of the Input Element and send the audio to the Input scope of the Output Element

Finally, the I/O unit is the only audio unit in the Audio Processing Graph that can start and stop an audio stream. In this way, the I/O unit is responsible for the audio stream in the audio unit application

Audio Processing Graphs Manage Audio Units

The Audio Processing Graph is a Core Foundation style opaque type, AUGraph, that you can use to build and manage audio unit processing chains. Graph can leverage the power of multiple audio units and multiple render callback functions, allowing you to create almost any audio processing solution you can think of

The AUGraph type adds thread-safety to the audio unit: it allows you to dynamically reconfigure the processing chain. For example, you can safely insert an equalizer while the audio is playing, or even swap a different render callback function for a mix input. In fact, the AUGraph type provides the only API in iOS for performing this dynamic reconfiguration in audio applications.

The Audio Processing Graph API uses another opaque type, AUNode, to represent individual audio units within the Graph context. When using Graph, you typically interact with Nodes as a proxy for the audio units it contains, rather than directly with the audio units

However, when putting graphs together, you have to configure each audio unit, and to do that you have to interact directly with the audio unit through the Audio Unit API. The audio cell node itself is not configurable. In this way, constructing a graph requires you to Use both APIs, as explained in Use the Two Audio Unit APIs in Concert

You can also represent a complete audio processing subgraph by defining node, using an AUNode instance as an element in a complex graph. In this case, the I/O unit at the end of the Subgraph must be the Generic Output unit — a type of I/O unit that is not connected to the device hardware.

In general, building the Audio Processing Graph requires three tasks:

Add Nodes to graph
Directly configures the audio unit represented by Nodes
Connect the nodes

For more information on these tasks and the rest of the Audio Processing Graph’s life cycle, see Constructing Audio Unit Apps. For a complete description of the API, see the Audio Unit Processing Graph Services Reference

An Audio Processing Graph Has Exactly One I/O Unit

Each Audio Processing Graph has an I/O unit, whether you are recording, playing, or simultaneously I/O. The I/O unit can be any available in iOS, depending on the needs of your application. See Start by Choosing a Design Pattern for how I/O units fit into the Audio Processing graph’s architecture in a variety of usage scenarios

Graphs let you start and stop the audio stream with the AUGraphStart and AUGraphStop functions. These functions, in turn, pass start or stop messages to the I/O unit by calling their AudioOutputUnitStart or AudioOutputUnitStop functions. Thus, the GRAPH’s I/O unit is responsible for the audio stream within the Graph

Audio Processing Graphs Provide Thread Safety

The Audio Processing Graph API uses the metaphor of a “to-do list” to provide thread-safety. Some functions in this API add a unit of work to the change list for later execution. After you specify a complete set of changes, you can ask Graph to implement them.

Here are some common reconfigurations supported by the Audio Processing Graph API and their associated capabilities:

Add or remove audio Unit Nodes (AUGraphAddNode, AUGraphRemoveNode)
Add or delete the connection between the nodes (AUGraphConnectNodeInput AUGraphDisconnectNodeInput)
Connection to render the callback function to the audio unit input bus (AUGraphDisconnectNodeInput)

Let’s look at an example of reconfiguring a running Audio Processing Graph. For example, you’ve built a graph containing a Multichannel Mixer unit and a Remote I/O unit for the mixed playback of two synthetic sounds. You put the sound into the mixer’s two input buses. Mixer output goes to the OUTPUT element of the I/O unit, and then to the output audio hardware. Figure 1-4 shows the architecture.

Figure 1-4 A simple audio processing graph for playback

Now, suppose the user wants to insert an equalizer into one of two audio streams. To do this, add an iPod EQ unit between the input of one of the sounds and the mixer input it goes into, as shown in Figure 1-5.

Figure 1-5 The same graph after inserting an equalizer

The steps to complete this real-time reconfiguration are as follows:

By calling the AUGraphDisconnectNodeInput from mixer unit of Input 1 disconnect “beats sound” callback
Add node, the audio unit containing the iPod EQ unit, to graph. By using AudioComponentDescription structure specifies the iPod EQ unit, and then call AUGraphAddNode to make this happen. At this point, the iPod EQ unit is instantiated, but not initialized. It belongs to Graph, but is not yet involved in audio streaming.
Configure and initialize the iPod EQ unit. In this case, this requires a few things:
- Call the AudioUnitGetProperty function to retrieve the stream format from the Mixer input (kAudioUnitProperty_StreamFormat)
- The AudioUnitSetProperty function is called twice, once to set the stream format on the input of the iPod EQ unit and the second time to set the stream format on the output. (For a complete description of how to configure iPod EQ Units, see Using Effect Units.)
- Call the AudioUnitInitialize function to allocate resources to the iPod EQ unit and prepare it to process audio. This function call is not thread-safe, but you can (and must) perform it at this point when the iPod EQ unit is not yet actively involved in audio processing graphics (because you have not yet called the AUGraphUpdate function).
By calling AUGraphSetNodeInputCallback will “beats sound” callback function is attached to the input of iPod EQ.

In the previous list, steps 1, 2, and 4 — all AUGraph* function calls — were added to the graph graph’s “to-do” list. Call AUGraphUpdate to perform these pending tasks. When the AUGraphUpdate function is successfully returned, the graph has been dynamically reconfigured and the iPod EQ is in place to process the audio.

Audio Flows Through a Graph Using “Pull”

In the Audio Processing Graph, consumers call the provider when they need more audio data. The request for audio data has a stream, which is in the opposite direction to the audio stream. Figure 1-6 shows the working principle.

Figure 1-6 The pull mechanism of audio data flow

Each request for a set of data is called a render call, or pull. The rendering call is represented by a gray “Control Flow” arrow. The data requested by the render call is more accurately a set of audio sample frames (see Frame)

In turn, a set of audio sample frames supplied in response to a render call is called a slice. (See slice). The code that provides slice is called the Render Callback function and is described in Render Callback Functions Feed Audio to Audio Units.

In Figure 1-6, the pull process is as follows:

After you call the AUGraphStart function, the virtual Output device calls the callback of the Output Element of the Remote I/O unit. This call requests a slice of the processed audio data frame
The Remote I/O unit’s callback function looks for audio data to process in its input buffer to satisfy the call. If there is data waiting to be processed, the Remote I/O unit uses it. Otherwise, as shown, it calls the callback function that the application connects to the input. In this example, the input of the Remote I/O unit is connected to the output of the Effect unit. So, the I/O unit branch effect unit, requests slice audio frames.
An Effect unit behaves just like a Remote I/O unit. When it needs audio data, it gets it from the input connection. In this case, the effect unit pull applies the render callback function.
Your application’s render callback is the ultimate recipient of pull. It feeds the requested frame to the Effect unit.
The Effect unit processes the slice provided by the application’s render callback. The Effect unit then provides the processed data requested earlier (in Step 2) to the Remote I/O unit
The Remote I/O unit processes the slice provided by the Effect unit. The Remote I/O unit then feeds the processed slice of the original request (in Step 1) to the virtual output device. This completes a pull loop

Render Callback Functions Feed Audio to Audio Units

To provide the audio to audio unit input bus from disk or memory, pass it using a render callback function that conforms to the AURenderCallback prototype. When the Audio unit input needs another sample frame slice, it calls your callback function, as described in Audio Flows Through a Graph Using Pull

The process of writing callback functions is probably the most creative aspect of designing and building an audio unit application. This is your chance to create or change sound in any way you can imagine and code.

At the same time, render callbacks have strict performance requirements that must be followed. Render callbacks exist on a real-time priority thread where subsequent render callbacks arrive asynchronously. There is a time limit to what you do in render callbacks. If you are still working on the previous render call when the next callback comes, you will get a gap in the sound. Therefore, you cannot use locks, allocate memory, access file systems or network connections, or otherwise perform time-consuming tasks in render callbacks.

Understanding the Audio Unit Render Callback Function

Listing 1-5 shows the header of the render callback function that matches the AURenderCallback prototype. This section describes the usage and usage of each parameter.

Listing 1-5 A render callback function header

static OSStatus MyAURenderCallback (
    void                        *inRefCon,
    AudioUnitRenderActionFlags  *ioActionFlags,
    const AudioTimeStamp        *inTimeStamp,
    UInt32                      inBusNumber,
    UInt32                      inNumberFrames,
    AudioBufferList             *ioData
) { /* callback body */ }
Copy the code

The inRefCon parameter points to the programming context you specify when you Attach the Callback to the audio unit input (see Write and Attach Render Callback Functions). The purpose of this context is to provide the callback function with any audio input data or state information it needs to calculate the output audio for a given render call
The ioActionFlags parameter allows callbacks to provide a hint for the audio unit when there is no audio to process. For example, if your application is a synthetic guitar and the user is not currently playing a note, use the following statement in the body of the callback function:
```
*ioActionFlags |= kAudioUnitRenderAction_OutputIsSilence;
Copy the code
```
The inBusNumber parameter represents the audio unit bus that invokes the callback, allowing you to branch in the callback based on this value. In addition, when attaching callbacks to an audio unit, you can specify a different context for each bus (inRefCon).
The inNumberFrames parameter indicates the number of audio sample frames requested for the callback in the current call. You supply these frames to the buffer in the ioData argument
The ioData argument points to the audio data buffer that must be filled when the callback function is called. The audio put into these buffers must conform to the audio stream format of the bus calling the callback.

These buffers are explicitly set to 0 if a specific call to the callback is kept silent, for example using the memset function.

Figure 1-7 illustrates a pair of non-staggered stereo buffers in the ioData parameter. Use element in the diagram to visualize the details of the ioData buffer that the callback needs to fill

Figure 1-7 The ioData buffers for a stereo render callback function

Audio Stream Formats Enable Data Flow

When working with audio data at a single example level, as with audio units, it is not enough to specify the correct data type to represent the audio. The layout of bits in a single audio sample value is meaningful, so data types like Float32 or UInt16 are not expressive enough. In this section, you’ll learn about Core Audio’s solution.

Working with the AudioStreamBasicDescription structure (using AudioStreamBasicDescription structure)

Listing 1-6 The AudioStreamBasicDescription structure

struct AudioStreamBasicDescription {
    Float64 mSampleRate;
    UInt32  mFormatID;
    UInt32  mFormatFlags;
    UInt32  mBytesPerPacket;
    UInt32  mFramesPerPacket;
    UInt32  mBytesPerFrame;
    UInt32  mChannelsPerFrame;
    UInt32  mBitsPerChannel;
    UInt32  mReserved;
};
typedef struct AudioStreamBasicDescription  AudioStreamBasicDescription;
Copy the code

Because the name AudioStreamBasicDescription is very long, so it is usually abbreviated ASBD. To define values for ASBD fields, write code similar to listing 1-7.

Listing 1-7 Defining an ASBD for a stereo stream

size_t bytesPerSample = sizeof (AudioUnitSampleType);
AudioStreamBasicDescription stereoStreamFormat = {0};
 
stereoStreamFormat.mFormatID          = kAudioFormatLinearPCM;
stereoStreamFormat.mFormatFlags       = kAudioFormatFlagsAudioUnitCanonical;
stereoStreamFormat.mBytesPerPacket    = bytesPerSample;
stereoStreamFormat.mBytesPerFrame     = bytesPerSample;
stereoStreamFormat.mFramesPerPacket   = 1;
stereoStreamFormat.mBitsPerChannel    = 8 * bytesPerSample;
stereoStreamFormat.mChannelsPerFrame  = 2;           // 2 indicates stereo
stereoStreamFormat.mSampleRate        = graphSampleRate;
Copy the code

First, determine the data type that represents the value of an audio sample. This example uses the type defined by AudioUnitSampleType, which is the recommended data type for most audio units. In iOS, AudioUnitSampleType is defined as an 8.24 fixed-point integer. The first line in Listing 1-7 counts the number of bytes in the type; This number is needed when defining some field values for ASBD, as you can see in the listing.

Next, still refer to listing 1-7, declare a AudioStreamBasicDescription types of variables, and the field is initialized to zero, to make sure that no fields contain garbage data. Do not skip the zero step

Now define the ASBD field value. Specify kAudioFormatLinearPCM for the mFormatID field. Audio units use uncompressed audio data, so this is the correct format identifier to use when working with audio units

Next, for the majority of audio unit for mFormatFlags specifies kAudioFormatFlagsAudioUnitCanonical metaflag. The sign in CoreAudio) framework/CoreAudioTypes) h are defined as follows:

kAudioFormatFlagsAudioUnitCanonical = kAudioFormatFlagIsFloat | kAudioFormatFlagsNativeEndian | kAudioFormatFlagIsPacked  | kAudioFormatFlagIsNonInterleavedCopy the code

This metaFlag is responsible for specifying all the layout details for the bits in the linear PCM sample values of type AudioUnitSampleType.

Some audio units use an atypical audio data format; the sample requires a different data type, and the mFormatFlags field requires a different set of flags. For example, 3 d Mixer unit need audio samples UInt16 data types, and set up for kAudioFormatFlagsCanonical need ASBD mFormatFlags field. When using a particular audio unit, be careful to use the correct data format and the correct format flags. (See Using Specific Audio Units)

Moving on to Listings 1-7, the next four fields further specify the organization and meaning of the bits in the sample frame. Set these fields – mBytesperPacket, mBytesPerFrame, mFramesPerPacket and mBitsPerChannel fields – depending on the nature of the audio stream you are using. To understand the meaning of these fields, please refer to the AudioStreamBasicDescription structure of the document

Set the ASBD’s mChannelsPerFrame field based on the number of channels in the stream, 1 for mono audio, 2 for stereo, and so on.

Finally, set the mSampleRate field based on the sampling rate you use throughout your application. Understand the importance of Where and How to Set Stream Formats. Configure Your Audio Session explains how to ensure that Your application’s sampling rate matches the Audio hardware sampling rate.

You can use CAStreamBasicDescription. H file (/ Developer/Extras/CoreAudio/PublicUtility) provided in the c + + utility methods, rather than one at a time like you see here specifies ASBD fields. In particular, check out the setaucanical and SetCanonical c++ methods. These specify the correct way to get the value of an ASBD field given three factors:

Is the stream used for I/O (SetCanonical) or audio processing (Setaucanical)?
How many channels do you want the stream format to represent
Whether you want interleaved or non-interleaved

Whether you in your project contains CAStreamBasicDescription. H file to use it directly, the method of apple suggest you study the document to learn the right way to use AudioStreamBasicDescription structure.

For information on how to fix problems related to audio data stream formats, see Troubleshooting Tips

Understanding Where and How to Set Stream Formats

You must format the audio data stream at a critical point in the Audio Processing Graph. In other areas, the system sets the format. Other times, audio unit connections propagate a stream format from one audio unit to another.

Audio input and output hardware on iOS devices has a system-determined audio stream format. These formats are always uncompressed, linear PCM format and interleaved. The system adds these formats to the outward side of the I/O unit in the Audio Processing graph, as shown in Figure 1-8

Figure 1-8 Where to set audio data stream formats

In the figure, the microphone represents the input audio hardware. The system determines the audio stream format of the Input hardware and applies it to the Input scope of the Input Element of the Remote I/O unit.

Again, the speakers in the figure represent the output audio hardware. The system determines the stream format of the Output hardware and applies it to the Output scope of the Output Element of the Remote I/O unit.

Your application is responsible for formatting the audio stream on the internal-facing side of the ELEMENT of the I/O unit. The I/O unit performs any necessary transformations between the application format and the hardware format. Your application is also responsible for formatting the streams wherever they are needed in the Graph. In some cases, such as the output of the Multichannel Mixer unit in Figure 1-8, you only need to set part of the format — specifically the sampling rate. Starting with selecting a design pattern, you will learn how to format streams for various types of audio unit applications. Using Specific Audio Units lists the streaming format requirements for each iOS Audio unit.

A key feature of the audio unit connection, as shown in Figure 1-8, is that the connection propagates the audio data stream format from the output of the source audio unit to the input of the target audio unit. This is a key point, so it needs to be emphasized: the propagation of the stream format occurs through the audio unit connection and in only one direction — from the output of the source audio unit to the input of the target audio unit.

Use format propagation. It can significantly reduce the amount of code you need to write. For example, when connecting the output of a multi-channel Mixer unit to a Remote I/O unit for playback, you do not need to set the flow format of the I/O unit. It is appropriately set up by connections between audio units, based on the output stream format of the Mixer (see Figure 1-8).

Streaming format propagation occurs at a specific point in the Audio Processing Graph lifecycle — at initialization. See Initialize and Start the Audio Processing Graph

You can be very flexible in defining application audio stream formats. However, whenever possible, use the sample rate that the hardware is using. When you do this, the I/O unit does not need to perform a sampling rate conversion. This minimizes energy consumption (an important consideration for mobile devices) and maximizes audio quality. To learn how to use hardware sampling rates, see Configure Your Audio Session

3. Build Audio Unit Apps

Now that you understand how the Audio Unit works, as explained in the Audio Unit Hosting Fundamentals section, you are ready to build the Audio Unit part of your application. The main step is to choose a design pattern and then write code to implement that pattern.

Start by Choosing a Design Pattern

There are six basic design patterns for audio units in iOS apps. Start by choosing the mode that best represents how you want your app to handle audio. As you study each pattern, notice the common characteristics. Each pattern:

There is only one I/O unit
A single audio stream format is used throughout the Audio Processing Graph — although there can be variations, such as mono and stereo stream input mixing units
A section that requires you to format a stream or stream at a specific location

The correct format of the stream is a prerequisite for creating an audio data stream. Most of these patterns rely on the automatic propagation of audio stream formats from source to target, as provided by audio cell connections. Take advantage of this propagation whenever possible because it reduces the amount of code to write and maintain. In the meantime, make sure you understand where you need to format the stream. For example, you must set up the full stream format on the input and output of the iPod EQ unit. For all iOS Audio unit stream format requirements, refer to Using Specific Audio Units

In most cases, the design pattern in this chapter uses the Audio Processing Graph (type AUGraph). You can implement any of these patterns without using graph, but using graph simplifies the code and enables dynamic reconfiguration, as described in Audio Processing Graphs Manage Audio Units

1. I/O Pass Through

The I/O pass-through mode sends incoming audio directly to the output hardware, with no option to process the audio data. Although not of practical value, building audio unit hosting applications based on this pattern is a good way to validate and solidify your understanding of audio unit concepts. Figure 2-1 shows this mode.

Figure 2-1 Simultaneous I/O pass through

As you can see in the figure, the audio Input hardware applies its stream format to the outward end of the Input element of the Remote I/O cell. In turn, you can specify the format you want to use on the inwards end of the element. The audio unit is formatted as needed. To avoid unnecessary sample rate conversions, be sure to use the audio hardware sample rate when defining the stream format

Input Element is disabled by default, so make sure it is enabled

The pattern shown in Figure 2-1 leverages the audio unit connection between two Remote I/O elements. Specifically, you do not need to format the stream on the input scope of the output element of the audio unit. The audio unit connection will pass the format you specified for the input Element

The outward side of the Output element adopts the stream format of the audio Output hardware, and the Output element performs format conversion for the Output audio as needed

With this mode, you do not need to configure any audio data buffers

2. I/O Without a Render Callback Function

Adding one or more other audio units between elements of a Remote I/O unit allows you to build a more interesting application. For example, you can use a Multichannel Mixer unit to locate incoming microphone audio in the stereo field or to provide output volume control. In this design mode, there is still no callback function, as shown in Figure 2-2. This simplifies the schema but limits its utility. Without rendering callbacks, you have no way to manipulate audio directly.

Figure 2-2 Simultaneous I/O without a render callback function

In this mode, as in pass-Through mode, two elements of the Remote I/O unit are configured. To set up the Multichannel Mixer unit, you must set the sampling rate of the flow format on the Mixer output, as shown in Figure 2-2.

The mixer’s Input stream format is automatically established through the audio unit connection through the output of the Input element of the Remote I/O unit. Similarly, the stream format of the input scope of the Remote I/O unit Output element is established through audio unit connections, thanks to the passing of the Mixer unit Output.

In any instance of this pattern, you actually have to set the kAudioUnitProperty_MaximumFramesPerSlice property whenever you use audio units other than I/O units, As described in the Audio Unit Properties Reference

3. I/O with a Render Callback Function

By placing callbacks between the input and output elements of the Remote I/O unit, you can manipulate incoming audio before it reaches the output hardware. In a very simple case, you can use the render callback function to adjust the output volume. However, you can add vibrato, ringtone modulation, echo, or other effects. By taking advantage of the Fourier transform and convolution functions provided in the Accelerate Framework (see *Accelerate Framework Reference*), your possibilities are endless. Figure 2-3 shows this mode.

Figure 2-3 Simultaneous I/O with a render callback function

As you can see from the figure, this pattern uses two elements of the Remote I/O unit, just like the previous pattern in this chapter. Append the render callback to the input scope of the Output Element. When the Element needs another set of audio sample values, it calls a callback. In turn, your callback gets the new example by calling the render callback of the Input Element of the Remote I/O unit.

As with other I/O modes, you must explicitly turn on the input module on the Remote I/O unit because input is disabled by default. For other I/O modes, you do not need to configure any audio data buffers.

Note that when you use the render callback function to establish an audio path from one audio unit to another, as in this mode, the callback replaces the audio unit connection

4. Output-Only with a Render Callback Function

Choose this mode for music games and synthesizers — applications where you are generating sound and need maximum response. At its simplest, this pattern involves a render callback function that connects directly to the Input scope of the Remote I/O unit output element, as shown in Figure 2-4.

Figure 2-4 Output-only with a render callback function

You can use the same pattern to build applications with more complex audio structures. For example, you might want to generate several sounds, mix them together, and then play them through the device’s output hardware. See Figure 2-5. Here, the pattern employs an Audio Processing graph and two additional audio units, a Multichannel Mixer and an iPod EQ.

Figure 2-5 A more complex example of output-only with a render callback function In the diagram, notice that the iPod EQ requires you to set the full stream format on the input and output. On the other hand, the Multichannel Mixer only needs to set the correct sampling rate on its output. The complete stream format is then propagated through the audio unit connection from the output of the mixer to the input range of the output element of the remote I/O unit. These usage details and other details about using various iOS audio units will be found inUsing Specific Audio UnitsIs described in

For each Multichannel Mixer unit input, the complete stream formatting is shown in Figure 2-5. For input 0, you set it explicitly. For input 1 format the audio unit is propagated through the output connection of the iPod EQ unit. Typically, you must consider the streaming format requirements of each audio unit separately

5. Other Audio Unit Hosting Design Patterns

There are two other main audio unit design patterns. To record or analyze audio, create an application with only input and render callbacks. The callback function is called by your application, which in turn calls the render method of the Input Element of the Remote I/O unit. In most cases, however, an application like this would be better off using an input audio queue object (of type AudioQueueRef, instantiated using the AudioQueueNewInput function), As explained in the Audio Queue Services Programming Guide. Using an audio queue object provides greater flexibility because its render callback function is not on a live thread.

To perform offline audio processing, use the Generic Output Unit. Unlike the Remote I/O unit, this audio unit is not connected to the device’s audio hardware. When you use it to send audio to your application, it is up to your application to invoke its render method.

Constructing Your App

No matter which design pattern you choose, the steps for building an audio unit hosting application are basically the same:

Configure your Audio Session
Specify the audio units
Create the Audio Processing graph and get the Audio Units
Configure audio units
Connect audio Units nodes
Provide the UI
Initialize and start the Audio Processing Graph

1. Configure Your Audio Session

The first step in building an audio unit application is the same as in any iOS audio application: configuring an audio session. The nature of an audio session largely determines the audio functionality of an application and its interaction with the rest of the system. First, specify the sampling rate you want to use in your application, as follows:

Self. GraphSampleRate = 44100.0; // HertzCopy the code

Next, use the audio session object to ask the system to use your preferred sampling rate as the device hardware sampling rate, as shown in Listing 2-1. The goal here is to avoid sampling rate conversions between hardware and applications. This maximizes CPU performance and sound quality and minimizes battery consumption

Listing 2-1 Configuring an audio session

NSError *audioSessionError = nil; AVAudioSession *mySession = [AVAudioSession sharedInstance]; // 1 [mySession setPreferredHardwareSampleRate: graphSampleRate // 2 error: &audioSessionError]; [mySession setCategory: AVAudioSessionCategoryPlayAndRecord // 3 error: &audioSessionError]; [mySession setActive: YES // 4 error: &audioSessionError]; self.graphSampleRate = [mySession currentHardwareSampleRate]; / / 5Copy the code

The above lines of code do the following:

Gets a reference to a singleton audio session object for your application
Request the sampling rate of the hardware. The system may or may not process the request, depending on other audio activity on the device
Request the audio session category you want. The “Play and Record” category specified here supports audio input and output
Request activation of your audio session
After the audio session is activated, it updates its own sampling rate variable based on the actual sampling rate provided by the system

You may need to configure another hardware feature: audio hardware I/O buffer times. At 44.1khz sampling rate, the default duration is about 23 ms, which is equivalent to the slice size of 1024 samples. If I/O latency is important in your application, you can request a shorter duration, reduced to 0.005 ms (equivalent to 256 samples), as shown below:

Self. IoBufferDuration = 0.005; [mySession setPreferredIOBufferDuration: ioBufferDuration error: &audioSessionError];Copy the code

For complete instructions on how to configure and use Audio Session objects, see Audio Session Programming Guide

2. Specify the Audio Units You Want

At runtime, after your audio session configuration code runs, your application does not get the audio unit. You can use AudioComponentDescription structure to specify that you want to each component. See Use Identifiers to Specify and Obtain Audio Units to learn how to do this. The Identifier Keys for each iOS Audio unit are listed in the Identifier Keys for Audio Units

If you have the audio unit specifier, you can build the Audio Processing Graph based on the pattern you choose

3. Build an Audio Processing Graph

In this step, you will create a framework for the design patterns explained in the first part of this chapter. specifically

Instantiate an AUGraph opaque type. This example represents the Audio Processing graph
Instantiate one or more AUNode opaque types, each representing an audio unit in the Graph
Add Nodes to the graph
Open graph and instantiate the audio unit
Gets a reference to the audio unit

Listing 2-2 shows how to perform these steps for a graph containing a Remote I/O unit and a Multichannel Mixer unit. It assumes that you have for each of the audio element defines a AudioComponentDescription structure

Listing 2-2 Building an audio processing graph

AUGraph processingGraph;
NewAUGraph (&processingGraph);
 
AUNode ioNode;
AUNode mixerNode;
 
AUGraphAddNode (processingGraph, &ioUnitDesc, &ioNode);
AUGraphAddNode (processingGraph, &mixerDesc, &mixerNode);
Copy the code

The AUGraphAddNode function call uses the audio unit specifiers ioUnitDesc and mixerDesc. At this point, the Graph will be instantiated and have nodes that you will use in your application. To open graph and instantiate the audio unit, call AUGraphOpen:

AUGraphOpen (processingGraph);
Copy the code

Then, we get a reference to the audio unit instance through the AUGraphNodeInfo function, as follows:

AudioUnit ioUnit;
AudioUnit mixerUnit;
 
AUGraphNodeInfo (processingGraph, ioNode, NULL, &ioUnit);
AUGraphNodeInfo (processingGraph, mixerNode, NULL, &mixerUnit);
Copy the code

The ioUnit and mixerUnit variables now contain references to instances of audio units in the Graph, allowing you to configure and interconnect audio units.

4. Configure the Audio Units

Each iOS Audio unit requires its own configuration, as described in Using Specific Audio Units. However, some configurations are so common that all iOS audio developers should be familiar with them.

By default, the Remote I/O unit has output enabled and input disabled. If your application performs I/O simultaneously, or uses input only, you must reconfigure the I/O unit accordingly. For details, see the kAudioOutputUnitProperty_EnableIO property in the Audio Unit Properties Reference.

All iOS audio units, except Remote I/O and Voice-processing I/O units, require their kAudioUnitProperty_MaximumFramesPerSlice property configuration. This property ensures that the audio unit is ready to produce a sufficient number of audio data frames in response to the render call. For more information, see kAudioUnitProperty_MaximumFramesPerSlice in the AudioUnit Properties Reference

All audio units require their audio stream format to be defined in input, output, or both. For an explanation of Audio Stream Formats, see Audio Stream Formats Enable Data Flow. See Using Specific Audio Units for Specific streaming format requirements for different iOS Audio Units

5. Write and Attach Render Callback Functions

For a design pattern that uses callback functions, you must write these functions and then append them to the correct locations. Render Callback Functions Feed Audio to Audio Units describes what these Callback Functions do and explains how they work

When the audio is not flowing, you can attach a callback immediately by using the audio unit API, as shown in Listing 2-3.

Listing 2-3 Attaching a render callback immediately

AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc        = &renderCallback;
callbackStruct.inputProcRefCon  = soundStructArray;
 
AudioUnitSetProperty (
    myIOUnit,
    kAudioUnitProperty_SetRenderCallback,
    kAudioUnitScope_Input,
    0.// output element
    &callbackStruct,
    sizeof (callbackStruct)
);
Copy the code

You can use the Audio Processing Graph API to attach render callbacks in a thread-safe manner, even if the audio is flowing. Listing 2-4 shows how to do this.

Listing 2-4 Attaching a render callback in a thread-safe manner

AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc        = &renderCallback;
callbackStruct.inputProcRefCon  = soundStructArray;
 
AUGraphSetNodeInputCallback (
    processingGraph,
    myIONode,
    0.// output element
    &callbackStruct
);
// ... some time later
Boolean graphUpdated;
AUGraphUpdate (processingGraph, &graphUpdated);
Copy the code

6. Connect the Audio Unit Nodes

In most cases, the use of audio processing in the graph API AUGraphConnectNodeInput and disconnected AUGraphDisconnectNodeInput function between the audio unit set up or is the best, It’s also easier. These functions are thread-safe and avoid the coding overhead of explicitly defining the join (which is necessary without Graph)

Listing 2-5 shows how to connect the output of a Mixer node to the input of an I/O unit output element using the audio processing diagram API

Listing 2-5 Connecting two audio unit nodes using the audio processing graph API

AudioUnitElement mixerUnitOutputBus  = 0;
AudioUnitElement ioUnitOutputElement = 0;
 
AUGraphConnectNodeInput (
    processingGraph,
    mixerNode,           // source node
    mixerUnitOutputBus,  // source node bus
    iONode,              // destination node
    ioUnitOutputElement  // desinatation node element
);
Copy the code

You can also establish and disconnect connections between audio units directly through the audio unit properties mechanism. To do this, use the AudioUnitSetProperty function and the kAudioUnitProperty_MakeConnection property, as shown in Listing 2-6. This approach requires defining an AudioUnitConnection structure for each connection as its property value.

Listing 2-6 Connecting two audio units directly

AudioUnitElement mixerUnitOutputBus  = 0;
AudioUnitElement ioUnitOutputElement = 0;
 
AudioUnitConnection mixerOutToIoUnitIn;
mixerOutToIoUnitIn.sourceAudioUnit    = mixerUnitInstance;
mixerOutToIoUnitIn.sourceOutputNumber = mixerUnitOutputBus;
mixerOutToIoUnitIn.destInputNumber    = ioUnitOutputElement;
 
AudioUnitSetProperty (
    ioUnitInstance,                     // connection destination
    kAudioUnitProperty_MakeConnection,  // property key
    kAudioUnitScope_Input,              // destination scope
    ioUnitOutputElement,                // destination element
    &mixerOutToIoUnitIn,                // connection definition
    sizeof (mixerOutToIoUnitIn)
);
Copy the code

7. Provide a User Interface

At this point in building the application, the audio unit (typically an audio processing diagram) is completely built and configured. In many cases, you’ll want to provide a user interface that lets your users adjust the audio behavior. You can customize the user interface to allow users to adjust specific audio unit parameters and, in some special cases, audio unit properties. In either case, the user interface should provide visual feedback about the current setup

Use Parameters and UIKit to Give Users Control explains the basics of building a user interface that lets Users Control parameter values

8. Initialize and Start the Audio Processing Graph

Before starting the audio stream, the audio processing diagram must be initialized by calling the AUGraphInitialize function. The key steps are as follows:

Initialize the audio units owned by the chart by automatically calling the AudioUnitInitialize function separately for each audio unit. (If you want to build a processing chain without diagrams, you must explicitly initialize each audio unit in turn.)
Verify graph’s connection and audio data stream format
Transfer stream format across audio unit connections

Listing 2-7 shows how to use AUGraphInitialize

Listing 2-7 Initializing and starting an audio processing graph

OSStatus result = AUGraphInitialize (processingGraph);
// Check for error. On successful initialization, start the graph...
AUGraphStart (processingGraph);
 
// Some time later
AUGraphStop (processingGraph);
Copy the code

Troubleshooting Tips

When the Core Audio function provides a return value, the value is captured and checked for success or failure. If a fault occurs, you can use Xcode’s Debugging feature, as shown in the Xcode Debugging Guide. If you use objective-C methods in your application, such as configuring audio sessions, you can also use the error parameter.

Note the dependencies between function calls. For example, the Audio Processing Graph cannot be started until it has been successfully initialized. Check the return value of AUGraphInitialize. If the function returns successfully, graph can be started. If you fail, determine what went wrong. Check that all audio unit function calls prior to initialization return successfully

Second, if graph initialization fails, use the CAShow function. This function prints the graph’s state to the Xcode console.

Make sure that you will each AudioStreamBasicDescription structure initialized to zero, as shown below:

AudioStreamBasicDescription stereoStreamFormat = {0};
Copy the code

Initializing ASBD’s fields to 0 ensures that no fields contain junk data. (When a data structure is declared in external storage — for example, as an instance variable in a class declaration — its fields are automatically initialized to 0, without requiring you to initialize them yourself.)

AudioStreamBasicDescription structure to the field values of the print to the Xcode console (this is very useful in the development process), please use the code shown in listing 2 to 8.

Listing 2-8 A utility method to print field values for an AudioStreamBasicDescription structure

- (void) printASBD: (AudioStreamBasicDescription) asbd { char formatIDString[5]; UInt32 formatID = CFSwapInt32HostToBig (asbd.mFormatID); bcopy (&formatID, formatIDString, 4); formatIDString[4] = '\0'; NSLog (@" SampleRate: %10.0f", asbd.msamplerate); NSLog (@" Format ID: %10s", formatIDString); NSLog (@" Format Flags: %10X", asbd.mFormatFlags); NSLog (@" Bytes per Packet: %10d", asbd.mBytesPerPacket); NSLog (@" Frames per Packet: %10d", asbd.mFramesPerPacket); NSLog (@" Bytes per Frame: %10d", asbd.mBytesPerFrame); NSLog (@" Channels per Frame: %10d", asbd.mChannelsPerFrame); NSLog (@" Bits per Channel: %10d", asbd.mBitsPerChannel); }Copy the code

This practical method can quickly find problems in ASBD

When defining asBDs for Audio unit stream formats, make sure you follow the “Recommended Stream Format Attributes” and “Stream Format Specifications” in Using Specific Audio Units. Don’t deviate from these tips unless you have a special reason.

The official documentation

Audio Unit Hosting Fundamentals
Constructing Audio Unit Apps
Using Specific Audio Units
The Demo address

IOS Audio -AudioUnit (official documentation)

1. Introduction

Audio Units Provide Fast, Modular Audio Processing

Choosing a Design Pattern and Constructing Your App

Get the Most Out of Each Audio Unit

How to Use This Document

2. Audio Unit Hosting Foundation

Audio Units Provide Fast, Modular Audio Processing

Audio Units in iOS

1. Effect Unit

2. Mixer Units

3. I/O Units

4. Format Converter Unit

Use the Two Audio Unit APIs in Concert

Use Identifiers to Specify and Obtain Audio Units

Use Scopes and Elements to Specify Parts of Audio Units

Use Properties to Configure Audio Units

Use Parameters and UIKit to Give Users Control

Essential Characteristics of I/O Units

Audio Processing Graphs Manage Audio Units

An Audio Processing Graph Has Exactly One I/O Unit

Audio Processing Graphs Provide Thread Safety

Audio Flows Through a Graph Using “Pull”

Render Callback Functions Feed Audio to Audio Units

Understanding the Audio Unit Render Callback Function

Audio Stream Formats Enable Data Flow

Working with the AudioStreamBasicDescription structure (using AudioStreamBasicDescription structure)

Understanding Where and How to Set Stream Formats

3. Build Audio Unit Apps

Start by Choosing a Design Pattern

1. I/O Pass Through

2. I/O Without a Render Callback Function

3. I/O with a Render Callback Function

4. Output-Only with a Render Callback Function

5. Other Audio Unit Hosting Design Patterns

Constructing Your App

1. Configure Your Audio Session

2. Specify the Audio Units You Want

3. Build an Audio Processing Graph

4. Configure the Audio Units

5. Write and Attach Render Callback Functions

6. Connect the Audio Unit Nodes

7. Provide a User Interface

8. Initialize and Start the Audio Processing Graph

Troubleshooting Tips

The official documentation

Related Posts

Swift refactoring: Reduces code by default view styles

IOS bottom layer – Analysis of message forwarding process

Swift- How to optimize using Print